#jython IRC Log (v0.9)


IRC Log for 2010-09-04

Timestamps are in GMT/BST.

[0:14] <jimbaker> headius: does jruby use String#intern ? we have a bug report, http://bugs.jython.org/issue1482, which states that our use of interning is filling up permgen when executing scripts repeatedly (presumably different string literals). seems plausible, except i can't reproduce
[0:15] <jimbaker> this is not something there seems to be a consistent answer to in my research
[0:16] <jimbaker> i do have a small patch, which i will attach to that bug, which replaces intern with a weak map. regrtest passes, and it might be the right solution anyway
[0:22] <headius> interned strings don't go into permgen
[0:22] <headius> what are you interning?
[0:23] <jimbaker> we are interning literals
[0:23] <jimbaker> so this could be a variable name, or a string literal
[0:23] <headius> hmm, that doesn't seem like it would be a lot of stuff
[0:23] <jimbaker> my reading of this is that intern works just like a weak map
[0:24] <jimbaker> well, i think this arises when someone executes a script file repeatedly
[0:24] <jimbaker> but of course with different literals each time :)
[0:25] <jimbaker> now, it's not exactly the way i would recommend writing code these days
[0:27] <headius> jimbaker: I've got a script running in a loop interning numbers from 0 to 10B and permgen is holding steady around 20MB
[0:28] <jimbaker> headius: what's a good way to dump out that number?
[0:28] <jimbaker> (in terms of current permgen usage)
[0:29] <headius> I'll let it run for a while
[0:29] <headius> I'm just watching it in jconsole
[0:31] <jimbaker> thanks, i'll look at that. artificially limiting heap doesn't seem to do anything. i wonder if the original poster is confusing issues here - we can't solve his memory leaks
[0:31] <jimbaker> so if he is polluting the global namespace, this could happen
[0:33] <headius> ../bsd-port/hotspot/agent/src/share/classes/sun/jvm/hotspot/memory/StringTable.java
[0:34] <headius> looking for intern
[0:34] <jimbaker> i did find that -Xmx16k is supposedly possible. runs real slow
[0:35] <jimbaker> probably ignores a max heap size below a certain amount however
[0:36] <headius> jimbaker: looks like it might all be in native memory
[0:37] <headius> so it would leak if it were interning literal strings, but not in permgen
[0:37] <headius> I don't think
[0:37] <headius> anyway, why are you interning literal strings?
[0:37] <headius> you need to get him to give you a memory dump and examine that
[0:38] <jimbaker> headius: i believe we intern string literals, much like java; but i have to check on that
[0:39] <headius> doesn't seem like you need to do that, but I don't know how they're used internally
[0:39] <jimbaker> i was able to cause an out of memory error when i attached w/ jconsole
[0:39] <headius> mmm
[0:39] <jimbaker> good heisenbug quality there
[0:39] <headius> my example is holding steady and is probably well into the millions of strings right now
[0:39] <headius> I think interned strings didn't used to GC, but they fixed that
[0:39] <jimbaker> correct on GCing interned strings
[0:39] <jimbaker> certainly by java 5
[0:39] <headius> yeah
[0:40] <headius> seems like something else is leaking, especially if the error actually mentions permgen
[0:40] <jimbaker> the out of memory is only on my ridiculous heap size max
[0:40] <headius> like class references
[0:40] <jimbaker> which i believe i just fixed
[0:40] <jimbaker> so probably just confounded
[0:41] <jimbaker> the beta 2 blockers (sounds like heart medicine ;) are anything that cause resource leaks
[0:41] <headius> heheh
[0:46] <jimbaker> headius: another question for you - have you had any luck in using tools like proguard to strip down jars? it seems good for apps, harder for what are essentially libraries like jython
[0:48] <jimbaker> (thanks for reminding me to use jmx here... we probably should add support for it to our regrtest suite)
[0:49] <headius> no, I haven't tried proguard yet...seems like it would be pretty tricky to get right with all the classes we have
[0:51] <jimbaker> that's my concern too
[0:52] <jimbaker> basically need something that says, keep on exposing all of these public APIs - along w/ all the dynamic stuff too
[1:06] <jimbaker> ok, i closed out http://bugs.jython.org/issue1482 - i think the jmx results are pretty definitive here
[1:06] <jimbaker> headius: thanks for your help!
[1:06] <headius> jimbaker: no problem!
[1:07] <headius> and yeah, agreed on what's needed from proguard
[1:07] <jimbaker> sounds like a good joint project
[1:07] <jimbaker> although, for now put it on the back burner!
[1:09] * verterok (~ggonzalez@unaffiliated/verterok) Quit (Ping timeout: 272 seconds)
[1:10] <headius> yeah :)
[1:20] * jimbaker (~jbaker@ Quit (Quit: jimbaker)
[1:25] * verterok (~ggonzalez@unaffiliated/verterok) has joined #jython
[1:46] * verterok (~ggonzalez@unaffiliated/verterok) Quit (Ping timeout: 240 seconds)
[1:59] * stakkars_ (~tismer@i59F7D801.versanet.de) Quit (Quit: stakkars_)
[2:27] * jimbaker (~jbaker@c-24-8-39-78.hsd1.co.comcast.net) has joined #jython
[2:28] <ropoctl> how do you import a jar file from python interpreter
[2:28] <ropoctl> jython i mean
[2:34] <jimbaker> ropoctl: just add it to your path, either classpath at startup or sys.path
[2:34] <ropoctl> ah
[3:16] * robbyoconnor (~wakawaka@guifications/user/r0bby) Quit (Ping timeout: 245 seconds)
[3:20] * robbyoconnor (~wakawaka@guifications/user/r0bby) has joined #jython
[3:59] * verterok (~ggonzalez@unaffiliated/verterok) has joined #jython
[4:10] * verterok (~ggonzalez@unaffiliated/verterok) Quit (Ping timeout: 260 seconds)
[5:31] * jimbaker (~jbaker@c-24-8-39-78.hsd1.co.comcast.net) Quit (Quit: jimbaker)
[5:52] * jimbaker (~jbaker@c-24-8-39-78.hsd1.co.comcast.net) has joined #jython
[5:58] * stever (~stever@dsl78-143-205-231.in-addr.fast.co.uk) has joined #jython
[6:03] * stever (~stever@dsl78-143-205-231.in-addr.fast.co.uk) Quit (Quit: stever)
[6:15] * jabley (~jabley@cpc1-farn4-0-0-cust318.6-2.cable.virginmedia.com) has joined #jython
[6:20] * stever (~stever@dsl78-143-205-231.in-addr.fast.co.uk) has joined #jython
[6:21] * stever (~stever@dsl78-143-205-231.in-addr.fast.co.uk) Quit (Client Quit)
[6:38] * jabley (~jabley@cpc1-farn4-0-0-cust318.6-2.cable.virginmedia.com) Quit (Quit: jabley)
[6:44] * jimbaker (~jbaker@c-24-8-39-78.hsd1.co.comcast.net) Quit (Quit: jimbaker)
[7:08] * stakkars (~tismer@i59F7D801.versanet.de) Quit (Ping timeout: 265 seconds)
[7:13] * stakkars (~tismer@i577B6E1F.versanet.de) has joined #jython
[7:46] * jabley (~jabley@cpc1-farn4-0-0-cust318.6-2.cable.virginmedia.com) has joined #jython
[9:12] * robbyoconnor (~wakawaka@guifications/user/r0bby) Quit (Ping timeout: 276 seconds)
[9:16] * robbyoconnor (~wakawaka@guifications/user/r0bby) has joined #jython
[9:18] * r0bby (~wakawaka@guifications/user/r0bby) has joined #jython
[9:21] * robbyoconnor (~wakawaka@guifications/user/r0bby) Quit (Ping timeout: 245 seconds)
[9:22] * juneau001_ (~juneau@pool-72-69-213-50.chi01.dsl-w.verizon.net) has joined #jython
[9:23] * r0bby (~wakawaka@guifications/user/r0bby) Quit (Ping timeout: 272 seconds)
[9:25] * juneau001 (~juneau@pool-72-69-84-97.chi01.dsl-w.verizon.net) Quit (Ping timeout: 265 seconds)
[9:25] * juneau001_ is now known as juneau001
[12:43] * lopex (lopex@chello089076044027.chello.pl) Quit ()
[12:58] * juneau001_ (~juneau@pool-72-69-207-41.chi01.dsl-w.verizon.net) has joined #jython
[13:00] * juneau001 (~juneau@pool-72-69-213-50.chi01.dsl-w.verizon.net) Quit (Ping timeout: 265 seconds)
[13:00] * juneau001_ is now known as juneau001
[13:43] * stakkars_ (~tismer@i577B6E1F.versanet.de) has joined #jython
[14:19] * verterok (~ggonzalez@unaffiliated/verterok) has joined #jython
[17:33] <slestak> can someone soundboard this idea. i struggled withthi9s yestrday and wonder if the plan is flawed
[17:33] <slestak> i want to run a DocXMLRPCServer in jython
[17:34] <slestak> runs well when run in its own script
[17:34] <slestak> however, I tried to serve it up via modjy on tomcat and no such luck
[17:35] <slestak> i tried the wsgi_xmlrpc package (which is close, it doesnt do the self documenting server) but is the idea basically flawed before i put another day or two into it
[17:36] <slestak> i have really mature java bindings for my db, but none for python, so im hoping jython will help me jailbreak this thing
[18:39] * pigletto (~pigletto@dud16.neoplus.adsl.tpnet.pl) has joined #jython
[18:39] * jabley (~jabley@cpc1-farn4-0-0-cust318.6-2.cable.virginmedia.com) Quit (Quit: jabley)
[19:09] <agronholm> why is modjy a requirement
[19:09] <agronholm> I'd just serve it via reverse proxy with apache or nginx
[19:12] <agronholm> slestak: besides, you never explained what the problem was with modjy+tomcat
[19:41] * stever (~stever@dsl78-143-205-231.in-addr.fast.co.uk) has joined #jython
[19:56] * aljungberg (~siker@pc-6-47-104-200.cm.vtr.net) has joined #jython
[19:57] <aljungberg> Hey guys. Is there a way to toggle the respectJavaAccessibility flag without command line options or a registry file?
[19:59] * verterok (~ggonzalez@unaffiliated/verterok) Quit (Ping timeout: 276 seconds)
[20:05] <agronholm> don't do it
[20:05] <agronholm> it doesn't really work
[20:06] <agronholm> it makes your program execution unreliable
[20:14] * lopex (lopex@chello089076044027.chello.pl) has joined #jython
[20:15] <aljungberg> The flag doesn't work or toggling it doesn't work?
[20:52] * pigletto (~pigletto@dud16.neoplus.adsl.tpnet.pl) Quit (Remote host closed the connection)
[21:30] <agronholm> toggling works but the interpreter goes haywire
[21:31] <agronholm> it's a very bad idea to enable it
[21:59] <aljungberg> Seems to work alright so far. Can't really avoid it, this library is riddled with protected classes, so I'll have to take my chances.
[22:01] <aljungberg> Thanks for the warning.
[22:20] <agronholm> wait, protected?
[22:20] <agronholm> why do you need that for protected stuff
[22:21] <agronholm> what library is it that you are talking about
[22:21] <aljungberg> Default visibility classes can only be accessed from the same package otherwise. This is Google's Closure Compiler.
[22:22] <agronholm> that just means you aren't supposed to access them in your application code
[22:23] <aljungberg> Not in this case.
[22:23] <agronholm> how is your case different
[22:25] <aljungberg> I'm hooking into the core to extend it but I can't live in the same package being a Jython app.
[22:26] <agronholm> extending it with jython will get hairy
[22:26] <agronholm> sounds like a bad idea
[22:26] <agronholm> the workaround is to place java adapters in the proper package and just call python code from there
[22:31] <aljungberg> You mean calling the Jython code from Java code? I do that for another job and it's unfortunately a little bit on the cumbersome side - through no fault of Jython, just because of Java itself.
[22:31] <aljungberg> Anyhow, it works right now so no worries. :)
[22:33] <agronholm> you may end up regretting turning on that flag
[22:34] <aljungberg> Yeah I tried to avoid it. For one it makes it harder to see what's private in the debugger with the dir() command.
[22:36] <ropoctl> this is the essence of a problem I'm having
[22:36] <ropoctl> http://pastebin.com/6JZ8AmQu
[22:36] <ropoctl> i made it work by making a new constructor in java, is that the only way to do it?
[22:39] <agronholm> your code has an error
[22:39] <agronholm> all methods must have self as the first argument
[22:40] <ropoctl> i dont remember if i forgot self in the actual example
[22:40] <ropoctl> but if i did, it means i've been doing too much java
[22:40] <ropoctl> lol
[22:41] <ropoctl> no im pretty sure i had self in there when i did it
[22:41] <agronholm> which version of jython
[22:41] <ropoctl> because i did not get a "wtf is 'self' " error
[22:41] <ropoctl> 2.5.2
[22:42] <agronholm> the java class looks like pseudocode
[22:43] <agronholm> can you give me an actual example that compiles?
[22:43] <ropoctl> ok i will work on making one
[22:54] <ropoctl> http://pastebin.com/YEYeDmnh
[23:09] <agronholm> what if you replace line 13 with xyz = Foo.bar(self, other)?
[23:09] <agronholm> I never use super() myself
[23:09] <ropoctl> yah actually that one works
[23:10] <ropoctl> but then xyz is a Foo object and I can't "cast" it as a PyFoo object so that I can modify it
[23:10] <agronholm> http://fuhm.net/super-harmful/
[23:11] <agronholm> of course it's a Foo object
[23:11] <agronholm> since that's what the superclass bar() returns
[23:11] <ropoctl> yah, obviously it would have to be
[23:11] <ropoctl> how would i cast it into PyFoo?
[23:12] <agronholm> cast?
[23:12] <ropoctl> change the java class somehow?
[23:12] <agronholm> err, I think you're a bit lost here
[23:12] <agronholm> why not just construct a PyFoo in the subclass?
[23:13] <ropoctl> the "bar" function in java is something i want to keep in java
[23:13] <agronholm> yeah but obviously since it returns a java instance, it will stay a java instance
[23:14] <agronholm> it's not like you could cast the end result to the subclass in java either
[23:14] <ropoctl> right, so that's why im all confused
[23:15] <agronholm> what you want is impossible, and it's not even a Jython problem
[23:15] <agronholm> bar does not seem to do anything useful
[23:16] <agronholm> obviously you have a real world problem but I can't figure it out from this simplistic example
[23:16] <agronholm> I could maybe suggest a different approach
[23:16] <ropoctl> in my actual example, it does a computationally intensive method on the two "Foo" objects and returns a new one
[23:17] <agronholm> why not just create the new instance in the subclass method
[23:17] <agronholm> and only leave the computation to the superclass method
[23:18] <ropoctl> yah actually that's what i ended up doing
[23:18] <ropoctl> it just seemed kludgy


These logs were automatically created by JythonLogBot_ on irc.freenode.net using a slightly modified version of the Java IRC LogBot (github).