#jython IRC Log

Index

IRC Log for 2014-03-17

Timestamps are in GMT/BST.

[0:04] * enebo (~enebo@c-75-73-8-169.hsd1.mn.comcast.net) Quit (Quit: enebo)
[0:16] * lheuer (~Adium@unaffiliated/lheuer) Quit (Quit: Leaving.)
[3:31] * sinistersnare is now known as not-sinistersnar
[3:31] * not-sinistersnar is now known as notsinistersnare
[3:31] * notsinistersnare is now known as sinistersnare
[4:32] * sinistersnare is now known as sinsnare|zzZZzz
[6:24] * lheuer (~Adium@f048035249.adsl.alicedsl.de) has joined #jython
[6:24] * lheuer (~Adium@f048035249.adsl.alicedsl.de) Quit (Changing host)
[6:24] * lheuer (~Adium@unaffiliated/lheuer) has joined #jython
[10:17] * lopex (uid4272@gateway/web/irccloud.com/x-jrtbegoushwrzgeq) Quit (Write error: Connection reset by peer)
[10:18] * lopex (uid4272@gateway/web/irccloud.com/x-rrxaasmiuwrgtxcz) has joined #jython
[11:00] * vext01 (~edd@88-106-250-174.dynamic.dsl.as9105.com) has joined #jython
[11:00] <vext01> morning all
[11:18] <topi`> what's up
[11:58] <vext01> im just having a look around in the Jython source code
[11:59] <vext01> lots of methods like:
[11:59] <vext01> public static final PyType TYPE = PyType.fromClass(PyObject.class);
[12:01] <topi`> fromClass looks like an alternate constructor
[12:01] <vext01> yeh it is
[12:01] <vext01> i think it's lazily creating python types
[12:02] <topi`> so it generates a Type of PyObject
[12:03] <vext01> this is the place we spend a lot of time in our jython programs
[12:08] <vext01> topi`: so it looks like, given a java class, it makes a python type instance
[12:09] <topi`> so it's a method that would be suitable target for some optimization?
[12:27] <vext01> well perhaps
[14:04] <vext01> what determines if cProfile is available on a given platform?
[14:10] * clajo04 (~clajo04@pool-108-21-222-14.nycmny.fios.verizon.net) Quit (Quit: clajo04)
[14:17] * enebo (~enebo@c-75-73-8-169.hsd1.mn.comcast.net) has joined #jython
[14:25] <jimbaker> sinsnare|zzZZzz, makes perfect sense, UMD is a great school
[14:25] <vext01> hi jimbaker
[14:26] <jimbaker> vext01, right this initialization would be part of the expense of startup
[14:26] <jimbaker> there have been some minor work to optimize startup time, but it's never been a priority
[14:27] <vext01> we are trying to find where most time is spent in our language compositions
[14:27] <jimbaker> vext01, re cProfile - it would be easy to enough to write the java equiv
[14:27] <jimbaker> and make sure it's fast - there was a good related talk on this at the jvm lang summit by jeremy manson
[14:27] <vext01> jimbaker: i have alreayd been profiled with the built in java profiler
[14:28] <vext01> but we want to try at the python level also
[14:28] <jimbaker> exactly - that's why you need cProfile - but written in java
[14:28] <vext01> i see
[14:28] <vext01> i dont think we have time to write a profiler :P
[14:28] <jimbaker> cStringIO is another example where we kept the name
[14:28] <vext01> submission deadline next week
[14:29] <jimbaker> ahh, yes, that's a little tough
[14:29] <vext01> so i was hoping to get cProfile working atleast
[14:29] <jimbaker> i was talking with jeremy siek on the ski lift saturday about this very question
[14:29] <jimbaker> as in, the nature of academic deadlines and writing stuff to the last moment
[14:29] <vext01> we don't like to
[14:30] <jimbaker> sure
[14:30] <vext01> so, what determines whether cProfile is built?
[14:31] <jimbaker> anyway - cProfile doesn't exist in a form suitable for jython. it is possible that jyni could make that feasible
[14:31] <jimbaker> but obviously not available for real usage yet
[14:31] <vext01> oh i see
[14:31] <vext01> oh i was confused
[14:31] <vext01> http://www.jython.org/docs/library/profile.html
[14:32] <vext01> ^ not jython specific?
[14:32] <vext01> nope
[14:32] <vext01> that explains the confusion
[14:33] <jimbaker> vext01, right, the docs in there do say, we haven't ported the docs completely
[14:33] <jimbaker> sorry about the confusion
[14:34] <vext01> no problems
[14:34] <vext01> so i guess 'import profile' is my best bet for now
[14:35] <jimbaker> vext01, yes
[14:35] <jimbaker> it would be very straightforward to write cProfile in java however
[14:35] <vext01> im not much of a java hotshot to be honest
[14:35] <vext01> if i had more time, i would be tempted
[14:36] <jimbaker> vext01, no worries
[14:36] <vext01> mayeb after the deadline
[14:36] <vext01> *maybe
[14:36] <jimbaker> it's probably one of the easiest things that could be done, since it interacts at most in one or two places
[14:36] <jimbaker> w/ the rest of the runtime
[14:37] <vext01> cool
[14:37] <vext01> oh while i am here, which stuff in the os module depends upon jffi?
[14:37] <jimbaker> just want to make that interaction has light weight as possible. basically the hook is through http://docs.python.org/2/library/sys.html#sys.setprofile
[14:37] <jimbaker> you mean, jnr?
[14:38] <vext01> as in jffi-x86_64-OpenBSD.jar
[14:38] <jimbaker> right
[14:38] <jimbaker> that's the c-facing part of jnr. it's going to be things like os.chmod
[14:39] <jimbaker> pretty straightforward to identify - one moment
[14:44] * thereisnospoon_ (~thereisno@113-61-86-28.static.qld.dsl.net.au) has joined #jython
[14:46] <jimbaker> vext01, so the os module on posix systems will import this bridge module, https://bitbucket.org/jython/jython/src/6e438088c0e3e624465cc0747761650da3eb280f/src/org/python/modules/posix/PosixModule.java?at=default
[14:46] * thereisnospoon (~thereisno@113-61-86-28.static.qld.dsl.net.au) Quit (Ping timeout: 264 seconds)
[14:47] <vext01> thanks
[14:47] <vext01> ok, so os.stat for example
[14:47] <jimbaker> the key thing to look for is the use of posix, https://bitbucket.org/jython/jython/src/6e438088c0e3e624465cc0747761650da3eb280f/src/org/python/modules/posix/PosixModule.java?at=default#cl-60
[14:47] <jimbaker> which in turns calls against jnr-posix, https://github.com/jnr/jnr-posix/blob/master/src/main/java/jnr/posix/POSIX.java
[14:48] <jimbaker> jnr-posix in turn will try fallbacks to regular java if it cannot use jffi to access
[14:49] <vext01> i'm im surprised i can call os.stat even though the jffi jar is empty for openbsd
[14:49] <jimbaker> which could also be because it's running on some system that it hasn't been built against. i believe there's os 390 support, but if not, that would be a good example :)
[14:49] <vext01> ah
[14:49] <vext01> yes, i see
[14:49] <jimbaker> vext01, so there you go
[14:49] <vext01> gotcha
[14:49] <vext01> did that change recently?
[14:49] <jimbaker> right, we had this discussion way back about an empty jar
[14:49] <vext01> ye, we did
[14:50] <jimbaker> this fallback mechanism has been in place for quite a while
[14:50] <vext01> i didn't realise there were fallbacks, that's al
[14:50] <vext01> cool
[14:50] <jimbaker> yeah, it's pretty important for cases like running in an app server
[14:52] <jimbaker> anyway, this has been a great collaboration with jruby. basically they do the work, we use it. thanks enebo ! ;)
[14:52] <vext01> :)
[14:52] <enebo> jimbaker: :)
[14:54] <jimbaker> looking at https://github.com/jnr/jnr-posix/blob/master/src/main/java/jnr/posix/POSIX.java, it does make me think we should look at processMaker and some related process mgmt, at the very least for better twisted support
[14:55] <jimbaker> topi`, ^^^
[14:55] <jimbaker> vext01, it looks like you can test with POSIX.isNative() whether you get fallbacks or not
[14:58] <vext01> neat
[14:59] * q5p4k0 (~q5p4k0@unaffiliated/q5p4k0) has joined #jython
[14:59] <jimbaker> another thing we should probably revisit is our signal implementation - we only have a basic mapping against that support from sun.misc
[15:11] <vext01> jimbaker: thanks for your help by the way
[15:11] <jimbaker> vext01, np
[15:11] <jimbaker> i took a look at sys.profile a bit more just to capture what's going on
[15:12] <jimbaker> right now, it does a fairly expensive synchronization due to the use of PythonTraceFunction, which wraps the profile function
[15:17] <jimbaker> one could directly set the underlying threadstate variable, profilefunc, to an object extending the TraceFunction class
[15:17] <jimbaker> and the only overhead would then be the actual profiling being done. this include extra profiling of the JVM per what jeremy manson discussed
[15:17] <jimbaker> sys.setprofile, rather
[15:17] <jimbaker> an obvious optimization for us to do is add one extra check in sys.setprofile - if the user specifies such an object extending TraceFunction, we directly install it, vs wrapping it
[15:22] <jimbaker> vext01, temporary network blip, so resending just in case
[15:22] <jimbaker> [09:09:37] <jimbaker> i took a look at sys.profile a bit more just to capture what's going on
[15:22] <jimbaker> [09:10:24] <jimbaker> right now, it does a fairly expensive synchronization due to the use of PythonTraceFunction, which wraps the profile function
[15:23] <jimbaker> [09:12:09] <jimbaker> one could directly set the underlying threadstate variable, profilefunc, to an object extending the TraceFunction class
[15:23] <jimbaker> [09:13:25] <jimbaker> and the only overhead would then be the actual profiling being done. this include extra profiling of the JVM per what jeremy manson discussed
[15:23] <jimbaker> [09:13:48] <jimbaker> sys.setprofile, rather
[15:23] <jimbaker> [09:14:50] <jimbaker> an obvious optimization for us to do is add one extra check in sys.setprofile - if the user specifies such an object extending TraceFunction, we directly install it, vs wrapping it
[15:23] <jimbaker> we can also take a look at this approach to profiling
[15:24] <jimbaker> http://www.oracle.com/technetwork/java/jvmls2013manson-2013920.pdf, p29
[15:24] <vext01> right
[15:51] <jimbaker> vext01, so two locks that would not be necessary for java code, assuming it's not calling python (in which case it needs to acquire the module import lock and do a safe call as well). the cost of the module import lock is probably minimal (likely elided), but the safe calling is definitely relatively expensive, beyond have to do additional dynamic lookups
[15:53] <vext01> sounds like you have a much better idea of what needs to be done tbh
[16:01] <jimbaker> vext01, that's too often the case :(
[16:01] <jimbaker> still doesn't mean i have time for it this moment
[16:03] <vext01> :)
[16:03] <vext01> tell me about it
[16:04] <jimbaker> ok, that reminds me i need to finish up work on socket-reboot
[16:05] <jimbaker> i made some nice progress on this last week, but there are several test suites to go!
[16:39] * vext01 (~edd@88-106-250-174.dynamic.dsl.as9105.com) Quit (Ping timeout: 264 seconds)
[17:06] * vext01 (~edd@host-92-23-230-118.as13285.net) has joined #jython
[18:35] * oscar_toro (~Thunderbi@h-17-170.a328.priv.bahnhof.se) has joined #jython
[18:49] * srcerer (~chatzilla@dns2.klsairexpress.com) has joined #jython
[21:28] * ArcTanSusan (~susantan@50-203-159-62-static.hfc.comcastbusiness.net) has joined #jython
[21:34] * clajo04 (~clajo04@pool-96-232-190-28.nycmny.fios.verizon.net) has joined #jython
[21:42] * ArcTanSusan (~susantan@50-203-159-62-static.hfc.comcastbusiness.net) Quit (Quit: ArcTanSusan)
[21:53] * ArcTanSusan (~susantan@50-203-159-62-static.hfc.comcastbusiness.net) has joined #jython
[22:24] * oscar_toro (~Thunderbi@h-17-170.a328.priv.bahnhof.se) Quit (Ping timeout: 240 seconds)
[23:29] * Arfrever (~Arfrever@apache/committer/Arfrever) Quit (Ping timeout: 265 seconds)
[23:35] * enebo (~enebo@c-75-73-8-169.hsd1.mn.comcast.net) Quit (Quit: enebo)

Index

These logs were automatically created by JythonLogBot on irc.freenode.net using a slightly modified version of the Java IRC LogBot.