#jython IRC Log (v0.9)

Index

IRC Log for 2011-11-30

Timestamps are in GMT/BST.

[2:11] * shashank (~shashank@ucb-np2-229.colorado.edu) Quit (Ping timeout: 244 seconds)
[2:34] * juneau001_ (~juneau@50-103-7-74.dklb.il.frontiernet.net) has joined #jython
[2:36] * juneau001 (~juneau@50-103-7-37.dklb.il.frontiernet.net) Quit (Ping timeout: 244 seconds)
[2:37] * juneau001 (~juneau@50-103-29-192.dklb.il.frontiernet.net) has joined #jython
[2:38] * juneau001_ (~juneau@50-103-7-74.dklb.il.frontiernet.net) Quit (Ping timeout: 240 seconds)
[2:51] * headius (~headius@71-210-151-185.mpls.qwest.net) has joined #jython
[2:51] <headius> hiya hiya
[2:52] <jimbaker> headius, hey. thanks for joining us here. let me just repaste from a pm
[2:52] <jimbaker> so this arose in our weekly jython performance meeting at univ of colorado. the context is, we would like to consider generating accessors mapping global names to a namespace (just global variables). and the concern is that this might not actually be a net benefit compared to just doing a ConcurrentHashMap
[2:53] <jimbaker> this would be via indy of course
[2:53] <jimbaker> so i probably have to get a little more into what is going on here
[2:54] <headius> ok
[2:54] <headius> you could probably make something more direct atop indy that would perform a lot better than hitting a CHM
[2:54] <jimbaker> for a given scope, say a module, python has a single namespace for names - variables/functions/classes
[2:56] <headius> I'm with you so far :)
[2:56] <jimbaker> in the past, due to python's rules for how globals are looked up, it was always done through a series of steps. the frame would be used to look up the namespace. this is because the code object is separable from a function object
[2:56] <jimbaker> so a function basically binds a global context with the code
[2:57] <jimbaker> now it's extraordinarily rare for code objects to be executed in this way
[2:58] <headius> ok, I think I follow still
[2:58] <jimbaker> so an obvious optimization is for functions to know the modules they live in (which constitute their globals) and fast path the desired namespace
[2:58] <jimbaker> no need to go through the frame
[2:58] <jimbaker> which is actually also necessary if we are to implement frameless pragmas
[3:00] <headius> ok
[3:00] <headius> and this is for making invocations of those functions fast, presumably?
[3:00] <headius> rather than doing the namespaced lookup every time?
[3:00] <jimbaker> some other details that matter here. globals in a module come about because of usage in the module, or through import. in general, imports are explicit in python, so even that's amenable to static analysis
[3:01] <jimbaker> both functions and more generally names used as variables
[3:02] <jimbaker> the only difference is that for a function, it's getglobal+call, the others are getglobal/setglobal in terms of the ops used
[3:03] <jimbaker> so we want to optimize function calls that nearly always resolve to the same function (standard indy stuff), as well as optimize get/set of names that are used as variables
[3:04] <jimbaker> global "variables" vs global names are not so common, but it's certainly common enough that we cannot make slower (and ideally want to make faster of course)
[3:04] <jimbaker> sorry, meant to say
[3:04] <jimbaker> global names used as "functions"
[3:05] <jimbaker> headius, just wanted to clarify this larger problem seems to be the one we need to solve
[3:05] <headius> ok
[3:05] <headius> well, the case of functions seems simple enough...you cache the functions inside some guarded series of handles at the call site, and presumably it never changes
[3:06] <headius> depending on what can cause it to change, you might have the guard simply be a SwitchPoint that when invalidated causes the call sites to re-cache
[3:06] <jimbaker> i'm pretty certain that there's a double guard required here. 1) is this code object used in the correct namespace? should be a simple checkcast 2) switchpoint to invalidate names used as functions, as you mention
[3:07] <headius> are not the call sites always within the same namespace?
[3:08] <jimbaker> by virtue of python import semantics, yes, they would be in that namespace
[3:08] <headius> then I don't see the need for guard #1
[3:08] <jimbaker> but i could take the co_code attribute of a function, then exec with a completely different global namespace
[3:08] <jimbaker> :)
[3:09] <headius> oh, I see...so the code object might execute under a different scope that looks up functions along a different path
[3:09] <jimbaker> to be honest, this is a bit of functionality i never felt the need to use, but it's there. probably for debugging or related uses
[3:10] <jimbaker> so the checkcast guard would invalid a switchpoint for that namespace (or more granular, probably not necessary though) to ensure callsites get rebound
[3:11] <jimbaker> so i think this part for functions is pretty straightforward
[3:11] <headius> yeah, dynamic scoping is a bitch
[3:11] <jimbaker> ;)
[3:11] <headius> that's definitely trickier
[3:12] <headius> presumably you can distinguish between calling via a normal function operation and calling against a co_code object directly
[3:12] <headius> you could have different entry points, or even different code
[3:12] * shashank (~shashank@174-29-75-118.hlrn.qwest.net) has joined #jython
[3:12] <jimbaker> yes, it's possible to make this distinction
[3:12] <jimbaker> shashank, you might want to look at the log, we're talking about your work right now :)
[3:12] <headius> you say this is rare, so maybe it doesn't matter...but if someone does use a co_code directly you don't necessarily want every invocation of it to invalidate everything
[3:13] <headius> in fact, what about this...instead of invalidating at all, you have at each call site a GWT where the fallback path is the slow always-look-up-in-namespace logic
[3:14] <headius> so that's some additional state passed into each invokedynamic that indicates whether to use the cached fast logic or the slow logic
[3:14] <jimbaker> sounds like a good plan, and no extra cost as i would understand it
[3:16] <headius> not much extra cost, at least
[3:16] <headius> the GWT guard would be 100% method handles
[3:16] <headius> that's *almost* free :)
[3:16] <jimbaker> right, and should inline well. extra cost in terms of space, i don't care too much. the other piece is support for global variables. so we are thinking of building a indyified version of __dict__
[3:17] <headius> sure
[3:17] <headius> so variable-wise, you have to decide if the variables are read-mostly or read-write
[3:17] <headius> in the read-mostly case, you can cache the value directly and invalidate through some mechanism. We do this for Ruby "constants"
[3:17] <jimbaker> so a global variable would simply be an accessor onto a suitable agglomeration of fields with volatile access
[3:18] <headius> you probably saw my blog post
[3:18] <headius> invalidation could be as simple as a single switchpoint, if as in Ruby you just invalidate everything...or could be some check that your'e still in the same namespace, as before
[3:18] <headius> are globals considered read-mostly?
[3:19] <jimbaker> only globals that are used as functions would have this characteristic w/o typing/profiling
[3:20] <jimbaker> that's maybe not quite true
[3:20] <jimbaker> there's actually some marker info that we could use here as a heuristic
[3:21] <jimbaker> in python, one knows if one is using a global variable to write to, and has to declare it
[3:22] <jimbaker> whereas globals that are read (ex: they refer to a dict or is just a constant) don't have that
[3:22] <jimbaker> also we certainly know all globals, outside of __dict__ ops - are they are read to/written to
[3:23] <jimbaker> headius, so i guess the answer is, we actually know more about their use
[3:24] <jimbaker> so it would be pretty easy to make those types of read-mostly globals work faster, as you note, and maybe care less about the read-write variant
[3:25] <headius> sure
[3:25] <headius> that seems just fine then
[3:32] <jimbaker> headius, cool. it sounds like we just use the same constant support for both names that are read ("read-mostly variables") and names that are called ("functions"); we can support "read-write" globals separately in the future . this is a nice simplification, even if has to take in account some different aspects of python semantics
[3:33] <headius> yeah, it sounds good
[4:35] * headius (~headius@71-210-151-185.mpls.qwest.net) Quit (Quit: headius)
[4:47] * SinZ|offline is now known as SinZ
[6:00] * shashank (~shashank@174-29-75-118.hlrn.qwest.net) Quit (Quit: Leaving.)
[6:02] * shashank (~shashank@174-29-75-118.hlrn.qwest.net) has joined #jython
[7:14] * Oti (~ohumbel@adsl-89-217-94-187.adslplus.ch) Quit (Quit: Oti)
[9:27] * Trundle (~andy@python/site-packages/trundle) has joined #jython
[9:56] * jimbaker (~jbaker@canonical/jimbaker) Quit (Ping timeout: 255 seconds)
[10:07] * jimbaker (~jbaker@c-75-71-80-146.hsd1.co.comcast.net) has joined #jython
[10:07] * jimbaker (~jbaker@c-75-71-80-146.hsd1.co.comcast.net) Quit (Changing host)
[10:07] * jimbaker (~jbaker@canonical/jimbaker) has joined #jython
[10:32] * sungji (~unknown@31.164.0.187) Quit (Quit: Leaving)
[10:58] * wainersm (~wainersm@201.82.225.3) has joined #jython
[11:16] * juneau001 (~juneau@50-103-29-192.dklb.il.frontiernet.net) Quit (Quit: Take care...)
[11:27] * stakkars_ (~tismer@p5DDB7E68.dip.t-dialin.net) Quit (Quit: schnarch)
[11:30] * SinZ is now known as SinZ|offline
[12:06] * juneau001 (~juneau@fess-116326.dhcp.fnal.gov) has joined #jython
[13:41] * stakkars__ (~tismer@g225039037.adsl.alicedsl.de) has joined #jython
[14:04] * T34Calliope (~chatzilla@80.31.98.86) has joined #jython
[14:05] * T34Calliope (~chatzilla@80.31.98.86) Quit (Client Quit)
[14:19] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has joined #jython
[14:38] * stakkars__ (~tismer@g225039037.adsl.alicedsl.de) Quit (Quit: schnarch)
[15:16] * robbyoconnor (~wakawaka@guifications/user/r0bby) Quit (Ping timeout: 248 seconds)
[15:19] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) Quit (Ping timeout: 258 seconds)
[15:20] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has joined #jython
[15:49] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) Quit (Quit: Ex-Chat)
[15:49] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has joined #jython
[15:52] * stakkars_ (~tismer@82.113.106.113) has joined #jython
[16:52] * fwierzbicki_ (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has joined #jython
[16:59] * sungji (~unknown@31.164.0.187) has joined #jython
[17:04] * Trundle (~andy@python/site-packages/trundle) Quit (Remote host closed the connection)
[17:09] * fwierzbicki_ (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has left #jython
[17:55] * fwierzbicki_ (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has joined #jython
[18:04] * robbyoconnor (~wakawaka@guifications/user/r0bby) has joined #jython
[18:05] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) Quit (Quit: Ex-Chat)
[18:05] * fwierzbicki_ is now known as fwierzbicki
[18:07] * r0bby (~wakawaka@guifications/user/r0bby) has joined #jython
[18:10] * robbyoconnor (~wakawaka@guifications/user/r0bby) Quit (Ping timeout: 258 seconds)
[18:19] * r0bby (~wakawaka@guifications/user/r0bby) Quit (Max SendQ exceeded)
[18:20] * r0bby (~wakawaka@guifications/user/r0bby) has joined #jython
[18:25] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) Quit (Remote host closed the connection)
[18:25] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has joined #jython
[18:39] * stakkars_ (~tismer@82.113.106.113) Quit (Quit: schnarch)
[18:48] * r0bby (~wakawaka@guifications/user/r0bby) Quit (Ping timeout: 260 seconds)
[18:58] * r0bby (~wakawaka@guifications/user/r0bby) has joined #jython
[19:15] * shashank (~shashank@174-29-75-118.hlrn.qwest.net) Quit (Ping timeout: 252 seconds)
[19:45] * stakkars_ (~tismer@88.130.164.59) has joined #jython
[20:15] * juneau001 (~juneau@fess-116326.dhcp.fnal.gov) Quit (Quit: juneau001)
[20:44] * r0bby (~wakawaka@guifications/user/r0bby) Quit (Ping timeout: 260 seconds)
[20:58] * shashank (~shashank@ucb-np2-115.colorado.edu) has joined #jython
[21:48] * juneau001 (~juneau@50-103-29-192.dklb.il.frontiernet.net) has joined #jython
[22:19] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has left #jython
[22:57] * shashank (~shashank@ucb-np2-115.colorado.edu) Quit (Quit: Leaving.)
[22:58] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has joined #jython
[23:06] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) Quit (Quit: Ex-Chat)
[23:14] * sungji (~unknown@31.164.0.187) Quit (Read error: Connection reset by peer)
[23:14] * sungji (~unknown@31.164.0.187) has joined #jython
[23:21] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has joined #jython
[23:21] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) Quit (Client Quit)
[23:22] * fwierzbicki (~frank@99-106-170-105.lightspeed.sntcca.sbcglobal.net) has joined #jython
[23:34] * diminoten (~diminoten@reddit/operator/diminoten) Quit (Quit: ZNC - http://znc.sourceforge.net)
[23:42] * diminoten (~diminoten@reddit/operator/diminoten) has joined #jython

Index

These logs were automatically created by JythonLogBot_ on irc.freenode.net using a slightly modified version of the Java IRC LogBot (github).