#jython IRC Log

Index

IRC Log for 2016-08-16

Timestamps are in GMT/BST.

[0:02] * jimbaker (~jbaker@python/psf/jimbaker) Quit (Ping timeout: 265 seconds)
[0:06] * jimbaker (~jbaker@8.44.156.98) has joined #jython
[0:06] * jimbaker (~jbaker@8.44.156.98) Quit (Changing host)
[0:06] * jimbaker (~jbaker@python/psf/jimbaker) has joined #jython
[0:06] * ChanServ sets mode +o jimbaker
[0:20] <jimbaker> _MylesC, something like the following: https://gist.github.com/jimbaker/7f260df8605fb156ae155800a383e357
[0:21] <jimbaker> just need to rework in java. there are several ways to do this
[0:26] <_MylesC> jimbaker: yeah i'm just stuck on how to do that, there's not many docs on using the java side. I've tried __set__, __setitem__ on PySystemState, and also tried making a globals object and passing it into the interpreter.
[0:28] <jimbaker> _MylesC, so i rarely use the java api myself, but http://www.jython.org/javadoc/org/python/util/PythonInterpreter.html is the simplest way to go
[0:28] <jimbaker> you should be able to do arbitrary python stuff, including setting stuff
[0:29] <jimbaker> PySystemState itself is not a real python module, due to bootstrapping issues. there is an outstanding bug about that
[0:30] <jimbaker> it's complex enough that no one has made it happen. and every time we touch PySystemState, we seem to break some corner case someone is depending on...
[0:31] <jimbaker> so the net of this is: don't try doing against PySystemState directly. use python code to make changes, because we fake the module aspects better from python space
[0:31] <jimbaker> PythonInterpreter lets you do that
[0:32] <_MylesC> So is there a way java side without using .exec etc to set sys.exit to a function?
[0:32] <jimbaker> not in a deep sense
[0:33] <jimbaker> maybe an equivalent usage that's more efficient
[0:34] <jimbaker> right, i was just looking at the PythonInterpreter API again. do use a code object, so you avoid compilation overhead
[0:34] <jimbaker> eg http://www.jython.org/javadoc/org/python/util/PythonInterpreter.html#exec(org.python.core.PyObject) works better than http://www.jython.org/javadoc/org/python/util/PythonInterpreter.html#exec(java.lang.String)
[0:35] <jimbaker> or http://www.jython.org/javadoc/org/python/util/PythonInterpreter.html#set(java.lang.String, java.lang.Object)
[0:35] <jimbaker> all reasonable
[0:39] <jimbaker> you might also get some insight looking at https://github.com/jythontools/jython/blob/master/src/org/python/util/jython.java, which builds on PythonInterpreter to get the REPL
[0:41] <_MylesC> I'll have a look into that and get back to you, I appreciate the help. Is the 2.7 branch still actively being developed?
[0:41] <jimbaker> lots of ideas on possible hooks here. for example: https://github.com/jythontools/jython/blob/master/src/org/python/util/jython.java#L343
[0:41] <jimbaker> _MylesC, yeah
[0:41] <jimbaker> we hit a roadblock, so we basically went into resting mode
[0:42] <_MylesC> I reported a bug earlier (i'm not sure if you guys have a format or anything) but essentially just that PyType.__subclasses__() order is random
[0:42] <jimbaker> but darjus recently found a solution to our object publication bug. as soon as we all get back from our respective vacations and find some common time, we will get some more progress in
[0:43] <jimbaker> and get 2.7.1, long delayed, out
[0:43] <jimbaker> _MylesC, yeah, i saw that on __subclasses__
[0:45] <jimbaker> in general if it's not tested by the cpython test suite that we use (cpython is the ref implementation)... well it can be different
[0:45] <jimbaker> so i see minimal testing of __subclasses__ in that regrtest
[0:46] <jimbaker> i have no idea of what it would take to fix. i very much doubt a fix would be in 2.7.1 since it should get shipped ASAP
[0:47] <jimbaker> i personally have spent too much time on PyType... it's quite possible in the same code we have been struggling with, it would seem to be likely
[0:50] <jimbaker> also i see after a 3 month summer hiatus (vacations!), stewori made a commit. nice
[0:51] <_MylesC> Appreciate the help man, will be around again hopefully tomorrow (when i've figured it out)
[0:53] <jimbaker> _MylesC, thanks for asking. good luck!
[1:23] * jimbaker (~jbaker@python/psf/jimbaker) Quit (Ping timeout: 252 seconds)
[1:27] * jimbaker (~jbaker@8.44.156.98) has joined #jython
[1:27] * jimbaker (~jbaker@8.44.156.98) Quit (Changing host)
[1:27] * jimbaker (~jbaker@python/psf/jimbaker) has joined #jython
[1:27] * ChanServ sets mode +o jimbaker
[2:58] * jimbaker (~jbaker@python/psf/jimbaker) Quit (Ping timeout: 244 seconds)
[3:02] * jimbaker (~jbaker@8.44.156.98) has joined #jython
[3:02] * jimbaker (~jbaker@8.44.156.98) Quit (Changing host)
[3:02] * jimbaker (~jbaker@python/psf/jimbaker) has joined #jython
[3:02] * ChanServ sets mode +o jimbaker
[3:32] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) Quit (Remote host closed the connection)
[3:33] * nickmbailey (~nickmbail@2605:6000:e8ce:3100:b0ca:d985:136b:1039) has joined #jython
[3:37] * nickmbailey (~nickmbail@2605:6000:e8ce:3100:b0ca:d985:136b:1039) Quit (Ping timeout: 250 seconds)
[4:25] * jimbaker (~jbaker@python/psf/jimbaker) Quit (Ping timeout: 265 seconds)
[4:28] * jimbaker (~jbaker@8.44.156.98) has joined #jython
[4:29] * jimbaker (~jbaker@8.44.156.98) Quit (Changing host)
[4:29] * jimbaker (~jbaker@python/psf/jimbaker) has joined #jython
[4:29] * ChanServ sets mode +o jimbaker
[4:35] * nickmbailey (~nickmbail@2605:6000:e8ce:3100:65b1:c333:6891:edd2) has joined #jython
[4:42] * nickmbailey (~nickmbail@2605:6000:e8ce:3100:65b1:c333:6891:edd2) Quit (Ping timeout: 250 seconds)
[5:05] * jimbaker (~jbaker@python/psf/jimbaker) Quit (Ping timeout: 276 seconds)
[5:09] * jimbaker (~jbaker@8.44.156.98) has joined #jython
[5:09] * jimbaker (~jbaker@8.44.156.98) Quit (Changing host)
[5:09] * jimbaker (~jbaker@python/psf/jimbaker) has joined #jython
[5:09] * ChanServ sets mode +o jimbaker
[6:18] * jimbaker (~jbaker@python/psf/jimbaker) Quit (Ping timeout: 240 seconds)
[6:22] * jimbaker (~jbaker@8.44.156.98) has joined #jython
[6:22] * jimbaker (~jbaker@8.44.156.98) Quit (Changing host)
[6:22] * jimbaker (~jbaker@python/psf/jimbaker) has joined #jython
[6:22] * ChanServ sets mode +o jimbaker
[6:58] * ohumbel (5390f784@gateway/web/freenode/ip.83.144.247.132) Quit (Quit: Page closed)
[7:04] * ohumbel (5390f784@gateway/web/freenode/ip.83.144.247.132) has joined #jython
[7:40] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) has joined #jython
[7:44] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) Quit (Ping timeout: 244 seconds)
[8:40] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) has joined #jython
[8:45] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) Quit (Ping timeout: 265 seconds)
[9:41] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) has joined #jython
[9:46] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) Quit (Ping timeout: 252 seconds)
[10:42] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) has joined #jython
[10:46] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) Quit (Ping timeout: 240 seconds)
[11:43] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) has joined #jython
[11:47] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) Quit (Ping timeout: 250 seconds)
[12:44] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) has joined #jython
[12:48] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) Quit (Ping timeout: 244 seconds)
[13:44] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) has joined #jython
[13:49] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) Quit (Ping timeout: 244 seconds)
[14:05] * jimbaker (~jbaker@python/psf/jimbaker) Quit (Ping timeout: 260 seconds)
[14:09] * jimbaker (~jbaker@8.44.156.98) has joined #jython
[14:09] * jimbaker (~jbaker@8.44.156.98) Quit (Changing host)
[14:09] * jimbaker (~jbaker@python/psf/jimbaker) has joined #jython
[14:09] * ChanServ sets mode +o jimbaker
[14:22] * Arfrever (~Arfrever@apache/committer/Arfrever) has joined #jython
[14:22] * ChanServ sets mode +o Arfrever
[14:38] * nickmbailey (~nickmbail@aus.internal.datastax.com) has joined #jython
[15:04] * jimbaker (~jbaker@python/psf/jimbaker) Quit (Ping timeout: 244 seconds)
[15:08] * jimbaker (~jbaker@8.44.156.98) has joined #jython
[15:08] * jimbaker (~jbaker@8.44.156.98) Quit (Changing host)
[15:08] * jimbaker (~jbaker@python/psf/jimbaker) has joined #jython
[15:08] * ChanServ sets mode +o jimbaker
[18:02] * choochter (choochter@nat/ibm/x-rpjcgqlynywtmddh) Quit (Read error: Connection reset by peer)
[18:29] * stewori (~stefan@5.146.129.76) has joined #jython
[19:23] <_MylesC> So i'm running the interpreter via java code, and the python code opens a file, and appends to it some data, when it finishes running java interepreter.execfile, it hasn't finished writing? and the only solution to getting the output is putting a Thread.sleep(10000) on it in java. Don't suppose there's any locks jython wise?
[19:39] * nickmbailey (~nickmbail@aus.internal.datastax.com) Quit (Remote host closed the connection)
[20:00] * nickmbailey (~nickmbail@cpe-70-117-83-204.austin.res.rr.com) has joined #jython
[20:18] <stewori> _MylesC: Can this somehow rely on a finalizer to flush the file-output? In that case Jython does not guarantee when finalizers are executed (or if at all), because finalization is triggered by Java-GC.
[20:19] <_MylesC> stewori: so there's no way to force all files to close?
[20:19] <stewori> (Maybe we should hook into end of execfile and run Python-finalizers, but I am spontaneously not sure if this is feasible)
[20:20] <stewori> I am just guessing
[20:20] <_MylesC> I mean I tried, interpreter = null then System.gc()
[20:20] <_MylesC> but I don't think that works that way
[20:20] <stewori> System.gc is inherently async in Java
[20:20] <stewori> usually it takes 1-2 seconds to complete
[20:21] <_MylesC> ah okay
[20:21] <stewori> Did you put the file access into a with-statement?
[20:21] <stewori> so it can be auto-closed
[20:21] <_MylesC> I just close it using output.close
[20:23] <stewori> I am not sure how Python semantic is here, but I would think this should flush the file. Maybe Jython gets something wrong here
[20:24] <_MylesC> stewori: Do you work on the project / know it really well then?
[20:26] <stewori> Not exactly File-stuff. However, does the flushing happen at least on JVM-exit? I know there is a mechanism that ensures file-closing on exit, but this might focus on JVM-exit rather than execfile-exit.
[20:27] <_MylesC> So my workflow is, java runs python asking it to put output in a temp.json, python runs, then I want to delete the file but I can't because it's still locked. I've tried a http://stackoverflow.com/a/6915221 to force it. When trying file.deleteonexit from java it also fails.
[20:37] <stewori> Could you try while not yourFile.closed(): yourFile.close()? (Not a nice solution, but I'm just curious if that works or not
[20:37] <stewori> at end of your python code
[20:39] <_MylesC> The while loop finishes but java is still unable to delete the file
[20:39] <_MylesC> all my code is open source on git, though it's a bit of an overhead to show you :p
[20:48] <stewori> what error message do you get when the JVM refuses to delete the file?
[20:48] <_MylesC> well f.delete() returns true but fails
[20:49] <_MylesC> then I try FileUtils.forceDelete (apache commons) and it just states can't delete
[21:02] <stewori> Looking at the sources underlying PyFile (which are quite involved for some reason) the actualy Jython class that wraps Java-nio API is core.io.FileIO. It uses nio.FileChannel. I'd suggest to retrive the FileChannel object and interrogate its state, but that seems to be non-trivial, only possible by acessing private fields
[21:05] <stewori> I guess you tried to delete the file by creating a java.io.File object from Java. I don't know JVM internals, but I suspect io.File and nio.FileChannel are fairly seperate APIs that might conflict if you mix them. Maybe JVM internally blocks interferring access to the same file by these seperate APIs at the same time
[21:07] <stewori> Still, calling close in Python should forward this to the underlying FileChannel, i.e. FileChannel.close.
[21:09] <stewori> Could you setup a simple Java-only test-example to see if it works when you create a FileChannel, write something, call close and then delete the file? If that succeeds it is confirmed that something in Jython causes this
[21:17] <stewori> Did you try to delete the file from python-code (just for testing)? Did you try to delete using java.nio.Files?
[21:18] <_MylesC> well I need to read it in java then delete :p
[21:18] <_MylesC> reading works fine just deleting sucks
[21:18] <_MylesC> gonna try and see if I can grab the filelock to get more info :)
[21:18] <_MylesC> (haven't tried what you suggested above)
[21:28] <_MylesC> Good news, I located the problem. It came across that the filereader we use in java was never closed. It wasn't, I apologise for any time waste I caused. (Simple mistakes destroy us all)
[21:29] <_MylesC> In other news, you wouldn't know how to reassign sys.exit use pure java would you? (I don't really wanna make a script to replace it)
[21:29] <_MylesC> *using
[22:05] * Arfrever (~Arfrever@apache/committer/Arfrever) Quit (Quit: 御出で)
[22:28] <jimbaker> _MylesC, re http://bugs.jython.org/issue2514, should be straightforward change
[22:28] <jimbaker> one line to be precise
[22:28] <jimbaker> (plus an import)
[22:28] <stewori> I suspected it is nothing with a filelock, at least not explicitly, because FileIO doesn't use any locking
[22:30] <stewori> Did you try it via Py.getSystemState().__dict__ ?
[22:30] <stewori> (regarding sys.exit)
[22:30] <_MylesC> Is there a specific set operation to use?
[22:31] <_MylesC> jimbaker: thanks so much for looking into it, it's really appreciated
[22:31] <stewori> the difficult part is to get the new method in Java as a PyObject
[22:32] <jimbaker> _MylesC, just need to construct a LinkedHashSet instead of Generic.set()
[22:32] <stewori> let's say you have a PyObject taht is callable containing your replacement for sys.exit
[22:33] <jimbaker> _MylesC, we could add that as a method to Generic.java, but you can experiment for now to see if it works
[22:33] <stewori> then try Py.getSystemState().__dict__.put("exit", replacementPyObject)
[22:33] <_MylesC> __dict__ is an instance of PyObject, so there's no put method?
[22:34] <jimbaker> stewori, i talked with darjus re that obj publication bug, and he's going to push his fix in
[22:34] <jimbaker> (http://bugs.jython.org/issue2487)
[22:34] <jimbaker> so hopefully later today
[22:35] <jimbaker> will unblock 2.7.1, which will make us all very happy i think
[22:35] <stewori> _MylesC: Use Py.newJavaFunc to get a Java-Method as a PyObject
[22:35] <stewori> Jim: Sound good
[22:35] <jimbaker> ok, just wanted to update, be back later
[22:35] <_MylesC> stewori: i'm still stuck on setting it in the dict, using __set__ gives "AttributeError: object internal __set__ impl is abstract"
[22:36] <_MylesC> jimbaker: take care :)
[22:37] <stewori> I maybe should fix platform.uname to work like before I added os.uname. This might break some things out there otherwise (not sure about the actual impact)
[22:46] <_MylesC> Tried poking at a few things, still stuck on assigning sys.exit, (tried a wrapper class but couldn't work out how to hook onto get as it never got called when I printed stuff out to test)
[22:51] <stewori> _MylesC: Try __setitem__ rather than __set__
[22:52] <stewori> Py.getSystemState().__dict__.__setitem__("exit", replacementPyObject)
[22:56] <stewori> Py.getSystemState().__setattr__("exit", replacementPyObject) might also work
[22:58] <stewori> Oh, you're using a PythonInterpreter. Call getSystemState on the interpreter object rather than on Py
[22:59] <_MylesC> Yeah I am dw
[22:59] <_MylesC> I tried the dict example and interpreter.getSystemState().__setattr__("sys", new PyObject());
[22:59] <_MylesC> (I'm just setting it to nothing to test)
[22:59] <_MylesC> They compile but they don't prevent sys.exit
[22:59] <_MylesC> I mean I could take the lazy java method and just use a security manager
[23:00] <_MylesC> but it would be nice to know how to override system functions
[23:00] <jimbaker> _MylesC, so i will just mention: PySystemState is not a normal module
[23:00] <jimbaker> so you can get this to work with the java api
[23:01] <jimbaker> but you will have to look at some specific workarounds done in PySystemState
[23:01] <jimbaker> doing this with python code is so much easier :)
[23:01] <jimbaker> that's why i recommended using PythonInterpreter#eval or #exec, with precompiled code objects
[23:02] <jimbaker> but it's a worthwhile exploration
[23:02] <jimbaker> anyway... i'm heading out, ttyl
[23:47] <_MylesC> I found a way to do it, not perfect but hey. Simply used "interpreter.getSystemState().__setattr__("exitfunc", Py.newJavaFunc(ExitCaller.class, "exit"));" then throw a RuntimeException from that and catch it when I execfile
[23:48] <_MylesC> oh wait that still doesn't cancel the sys exit o.o rip
[23:48] <_MylesC> Gonna head to bed aha, will try more tomorow

Index

These logs were automatically created by JythonLogBot on irc.freenode.net using a slightly modified version of the Java IRC LogBot.