#jython IRC Log

Index

IRC Log for 2016-11-18

Timestamps are in GMT/BST.

[0:36] * nickmbai_ (~nickmbail@2605:6000:e8ce:3100:a1bb:b392:6c35:f14a) Quit (Quit: later)
[3:17] * lopex (uid4272@gateway/web/irccloud.com/x-hgfigeezfevhdavs) Quit (Quit: Connection closed for inactivity)
[3:50] * lopex (uid4272@gateway/web/irccloud.com/x-jtheiiwvbzdxowtf) has joined #jython
[9:55] * girish946 (~gsjoshi@182.70.51.4) has joined #jython
[11:57] * girish946 (~gsjoshi@182.70.51.4) Quit (Remote host closed the connection)
[12:15] * girish946 (~gsjoshi@182.70.51.4) has joined #jython
[13:31] * girish946 (~gsjoshi@182.70.51.4) Quit (Quit: Leaving)
[13:59] * humon (~humon@rrcs-74-143-21-178.central.biz.rr.com) has joined #jython
[14:01] <humon> i have a fresh install of jython 2.7..... I try 'pip install requests' and get "RuntimeException: java.lang.RuntimeException: Method code too large!" Help?
[14:26] <humon> is this tool ready for primetime or still just a pipe dream?
[14:52] * maucar (~maurizioc@c-24-62-42-67.hsd1.ma.comcast.net) has joined #jython
[14:53] * maucar (~maurizioc@c-24-62-42-67.hsd1.ma.comcast.net) Quit (Remote host closed the connection)
[14:53] * maucar (~maurizioc@c-24-62-42-67.hsd1.ma.comcast.net) has joined #jython
[14:58] * maucar (~maurizioc@c-24-62-42-67.hsd1.ma.comcast.net) Quit (Remote host closed the connection)
[15:22] * xemdetia (xemdetia@nat/ibm/x-eqsoshtlesntmxpn) has joined #jython
[15:47] * stewori (~stefan@5.146.129.76) has joined #jython
[16:01] <stewori> humon: This is a known issue: http://bugs.jython.org/issue1891
[16:01] <stewori> although AFAIK requests used to work (didn't try it myself)
[16:02] <stewori> maybe an older version of requests still works, if this is because they made some method even larger in a new version
[16:06] <jimbaker> i haven't tried requests in the last month or so. sorry to hear about the problem, maybe we can get a fix in at some point. but almost always this is a vendored library they use
[16:08] <jimbaker> humon, re pipe dream - it sometimes feels like an uphill battle. we get major libraries to work. but since it's relatively rare for people to test against jython, stuff gets broken
[16:08] <jimbaker> one of my colleagues on my team is a requests committer, so maybe we can get this fixed :)
[16:08] <agronholm> humon: what is your motivation for trying jython?
[16:13] * humon (~humon@rrcs-74-143-21-178.central.biz.rr.com) Quit (Read error: Connection reset by peer)
[16:14] * humon (~humon@rrcs-74-143-21-178.central.biz.rr.com) has joined #jython
[16:19] <stewori> we should really get this pyc-bytecode support going. Would fix this constantly annoying issue 1891 and also advance Android support.
[16:22] <stewori> jimbaker: What is actually required for this? IIRC you once explained that import already works and code for compiling to pyc is also more or less there. It would be mostly a work of integration, gluing and fine-tuning. Maybe you could lay out some steps to get started here?
[16:22] <jimbaker> stewori, yeah, it's just a question of doing the python bytecode compilation
[16:23] <jimbaker> it would be a fun project
[16:24] <jimbaker> unlike most of the bug fixes we deal with
[16:25] <stewori> wouldn't it be too slow to use pyc-bytecode? (The fact that this would be applied on extra-large methods doesn't make it better I suppose)
[16:25] <jimbaker> stewori, it depends on specifics
[16:26] <stewori> I wondered whether it would be possible to auto-split methods. E.g. by passing down locals
[16:27] <jimbaker> the simplest approach would be to build a PyCode that uses python bytecode in the $py.class file
[16:27] <jimbaker> so it's just a constant in the constant pool
[16:27] <stewori> going the pyc-way is probably easier and also more valuable because of Android I agree
[16:27] <jimbaker> re large methods - they are almost always some initialization that's automatically generated
[16:28] <jimbaker> so speed is a minor concern
[16:28] <stewori> what would be wrong about storing in a pyc-file? Could be cross-used with CPython
[16:29] <jimbaker> pyc is also fine, and this is a reasonable workaround. one concern: there's no guarantee on pyc across major versions
[16:30] <jimbaker> so 2.5 is not compatible with 2.7
[16:30] <jimbaker> etc
[16:30] <stewori> hmm it would be a per-.py-file decision whether to use Python- or Java-bytecode, wouldn't it? So this would not only slowdown the single large method, but maybe the whole module
[16:30] <jimbaker> in this particular case, i don't think there has been intentional breakage
[16:31] <jimbaker> stewori, that's why i suggest using PyCode instead
[16:31] <jimbaker> and just construct in a $py.class
[16:31] <jimbaker> as needed
[16:32] <jimbaker> actually, it's PyCode for java bytecode; PyBytecode for python bytecode; PyBaseCode is the common base class
[16:32] <stewori> I heard of these incompatibilities. Isn't there a pyc-internal bit or so encoding the targeted python version? Somehow also CPython needs to know
[16:34] <jimbaker> stewori, don't know
[16:35] <jimbaker> also in 3.6, there's the new wordcode support
[16:36] <stewori> Anyway, your points for using $py.class are convincing
[16:36] <jimbaker> yeah, it would be super easy to do
[16:36] <jimbaker> from a packaging perspective
[16:39] <jimbaker> so the logic would be: attempt to compile a method; if it fails because too large, set up a PyBytecode instead. the only tricky part will be the fixups for the constant pool, but cpython and java are pretty similar here
[16:40] <jimbaker> this is related to the comments about intern names...
[16:41] <stewori> Is it straightforward to insert a foreign bytecode (partly) into a class-file? I guess it would take a special classloader to sort this out
[16:45] <jimbaker> the classloader would not care
[16:46] <jimbaker> this is just another PyObject initialization
[16:46] <jimbaker> it's worthwhile decompiling a $py.class file to see what actually goes on
[16:47] <jimbaker> i like procyon as a simple decompiler. but intellij has it built in as well now
[16:48] <jimbaker> a good example to try is by generating a large simple list/dict (maybe also set) initialization. you will see how that gets pulled out of the generated method
[16:48] <jimbaker> out of the *compiled* method, as compiled by jython
[16:50] <jimbaker> so a key part of our compiler is building up appropriate initializations of PyObjects. these are the actual constants we work with in jython
[16:50] <jimbaker> including of course PyCode, which ultimately links to java bytecode
[16:51] <jimbaker> even that is somewhat abstracted
[16:51] <jimbaker> it's quite an interesting setup to be honest
[16:51] <jimbaker> most of the optimizations around switchpoints and invokedynamic are about removing some of this overhead
[17:18] * clajo04 (~clajo04@pool-108-46-224-11.nycmny.fios.verizon.net) Quit (Quit: clajo04)
[17:19] <stewori> I just took a look at pycimport.py. So far I wasn't aware that a .pyc-file simply contains a serialized PyObject (i.e. marshalled)
[17:22] <stewori> Just a consistency-qustion: If we have an ordinary PyModule loaded with Jython (without an oversized method for now). The pipe this PyModule through _marshal.Marshaller.dump, would this result in a proper CPython 2.7 pyc-file?
[17:26] <jimbaker> stewori, no, _marshal cannot dump PyCode
[17:28] <jimbaker> _marshal does have support for PyBytecode however
[17:32] <stewori> I see. This essentially means only reading works.
[17:33] <jimbaker> because of how bytecode loaders work, there's also no access of the Java bytecode from PyCode
[17:34] <jimbaker> one can always go back to the $py.class file, but that's a basic JVM restriction
[17:34] <jimbaker> (assuming $py.class is constructed - it's not if compiled on the fly)
[17:35] <jimbaker> anyway... that's not the problem we started with
[17:35] <jimbaker> :)
[17:39] * clajo04 (~clajo04@pool-108-46-224-11.nycmny.fios.verizon.net) has joined #jython
[17:45] <stewori> right. I'm just getting an overview...
[17:46] <stewori> Unlike described in doc-comment of py_compile.py (even the one in Lib), py_compile does not create pyc-files, does it?
[17:52] * humon (~humon@rrcs-74-143-21-178.central.biz.rr.com) Quit (Read error: Connection reset by peer)
[17:52] * humon (~humon@rrcs-74-143-21-178.central.biz.rr.com) has joined #jython
[18:14] * humon (~humon@rrcs-74-143-21-178.central.biz.rr.com) Quit (Read error: Connection reset by peer)
[18:14] * humon (~humon@rrcs-74-143-21-178.central.biz.rr.com) has joined #jython
[18:15] * TomA (~TomA@2601:402:500:8e98:45a4:38df:9591:508b) has joined #jython
[18:17] <stewori> So how would we get a PyBytecode-object for an oversized method? Will we have to port this part from CPython? Or can we get Jython to do this? I.e. by adding a new class to org.python.compiler
[18:18] * TomA (~TomA@2601:402:500:8e98:45a4:38df:9591:508b) Quit (Remote host closed the connection)
[18:18] * TomA (~TomA@2601:402:500:8e98:45a4:38df:9591:508b) has joined #jython
[18:27] * girish946 (~gsjoshi@27.58.20.67) has joined #jython
[18:29] <stewori> jimbaker: Having to leave now, will continue to read async
[18:30] * stewori (~stefan@5.146.129.76) has left #jython
[19:09] <jimbaker> stewori, we will just want to create a python bytecode compiler. but we can experiment by simply using cpython 2.7 as a compiler backend for us :)
[19:21] * girish946 (~gsjoshi@27.58.20.67) Quit (Quit: Leaving)
[21:51] * TomA (~TomA@2601:402:500:8e98:45a4:38df:9591:508b) Quit (Remote host closed the connection)
[22:36] * xemdetia (xemdetia@nat/ibm/x-eqsoshtlesntmxpn) Quit (Ping timeout: 260 seconds)
[23:06] * humon (~humon@rrcs-74-143-21-178.central.biz.rr.com) Quit (Ping timeout: 260 seconds)

Index

These logs were automatically created by JythonLogBot on irc.freenode.net using a slightly modified version of the Java IRC LogBot.