Timestamps are in GMT/BST.
[0:19] * xemdetia (xemdetia@nat/ibm/x-rhjaacetlzrvpego) Quit (Ping timeout: 245 seconds)
[8:33] * Trundle (~andy@python/site-packages/trundle) has joined #jython
[13:59] * xemdetia (xemdetia@nat/ibm/x-hhgpusqrmfjnzwjz) has joined #jython
[14:03] * enebo (~enebo@c-75-73-8-169.hsd1.mn.comcast.net) has joined #jython
[15:05] * ELFrederich (~ELFrederi@63.87.61.77) has joined #jython
[15:07] <ELFrederich> hey guys... I'm using RPyC to use Jython from CPython. Things are working nicely except some things with coercion. When I pass a CPython list as an argument over RPyC to Jython which calls a Java method it is not being coerced into an array.
[15:08] <ELFrederich> If I use Jython directly without RPyC or CPython Jython does the coercion correctly... but I think when it is coming from RPyC it is getting a netref object instead of a list object so it isn't doing the coercion
[15:08] <agronholm> ELFrederich: may I ask why you're using jython from cpython?
[15:09] <ELFrederich> Any idea where in the Jython code this Python list to Java array coercion takes place? I'm wondering if it would be a simple change to allow either lists or netrefs to lists
[15:09] <ELFrederich> agronholm, sure... no problem...
[15:09] <agronholm> I mean, is there a reason you're not using py4j, or did you not know about it?
[15:09] <ELFrederich> agronholm, We're using Jython because there are Java libraries that integrate with an application we're using.
[15:09] <agronholm> there can be valid use cases for this too of course
[15:10] <agronholm> but most people would get by with py4j which is a lot simpler than what you're doing now
[15:10] <ELFrederich> agronholm, there are also C++ libraries but then we'd have to use boost to be able to use them from CPython. With Jython and RPyC combined we get to use the application without having to create anything (like boost bindings)
[15:10] <agronholm> I use jasperreports from my cpython app via py4j
[15:10] <ELFrederich> agronholm, I don't know about py4g
[15:10] <ELFrederich> *py4j
[15:12] <ELFrederich> agronholm, what is the state of py4j?... is it new, is it old, is it mature, maintined?
[15:12] <agronholm> it's maintained
[15:13] <agronholm> I've only had one problem which I reported and it was fixed
[15:16] <ELFrederich> agronholm, so I'd have to write some Java code to explicitly expose my classes?
[15:16] <agronholm> no
[15:16] <agronholm> I've written exactly zero lines of Java for my project
[15:16] <agronholm> you can just use the JVM from the python end
[15:16] <agronholm> no need to write any java code. it's optional.
[15:18] <ELFrederich> agronholm, what kind of gateway do I start in Java?
[15:18] <agronholm> maybe I don't understand the question
[15:19] <agronholm> the py4j library has utility functions to launch a JVM and it will run the py4j java part
[15:19] <agronholm> which will enable your python app to harness the JVM to its own use
[15:19] <ELFrederich> agronholm, I have some jars that I need to use from CPython... what code do I write in CPython?
[15:19] <ELFrederich> http://py4j.sourceforge.net/
[15:19] <ELFrederich> when I run that example I get py4j.protocol.Py4JNetworkError: An error occurred while trying to connect to the Java server
[15:20] <agronholm> use launch_gateway()
[15:20] <ELFrederich> agronholm, okay... thats what I need. What is the utility code to launch the JVM? The docs even start off with "writing the Java program" and "writing the Python program"
[15:20] <agronholm> from py4j.java_gateway
[15:21] <agronholm> the return value will be the port to connect to
[15:21] <agronholm> https://bpaste.net/show/7a3c64421719
[15:21] <agronholm> this is what I do in my app
[15:22] <ELFrederich> agronholm, thanks a bunch
[15:22] <ELFrederich> agronholm, I'll give it a try
[15:22] <agronholm> remember to add your jars in the class path
[15:23] <ELFrederich> agronholm, you're not using ctx in that function?
[15:23] <agronholm> this is a callback
[15:23] <agronholm> never mind that
[15:23] <ELFrederich> agronholm, okay... do I need to set classpath on launch_gateway or can I just use CLASSPATH env variable?
[15:24] <agronholm> I'm not sure if it propagates to the subprocess
[15:25] <ELFrederich> agronholm, I'll check it out
[15:25] <ELFrederich> what should I set start_callback_server and auto_convert to:?
[15:25] <agronholm> you should look in the docs what they do
[15:25] <ELFrederich> ;-)
[15:25] <agronholm> it'll give you a better idea than my explanations
[15:25] <agronholm> you may or may not need a callback server
[15:25] <agronholm> my app does
[15:26] <agronholm> I would enable auto_convert
[15:42] <ELFrederich> agronholm, to use this 3rd party Java application you need to login. The Java API for logging in requires that you pass it an object that subclasses CredentialManager. With Jython I can implement this class in Jython. Is something like this possible with py4j?
[15:58] <agronholm> ELFrederich: yes. that's what you need the callback server for.
[16:03] * Arfrever (~Arfrever@apache/committer/Arfrever) Quit (Ping timeout: 250 seconds)
[16:04] <ELFrederich> agronholm, hmm... how do I subclass a Java class? If I need to subclass CredentialManager where do I get CredentialManager from if the gateway hasn't been started yet/
[16:06] <agronholm> ELFrederich: ahm, sorry I misread -- I'm not entirely sure if you can subclass. You can implement interfaces though.
[16:07] <ELFrederich> agronholm, now I need to see if its just an interface. I think it might be
[16:07] <agronholm> for that you write a python class and place a special variable there
[16:07] <agronholm> the docs have this covered though
[16:11] <ELFrederich> agronholm, its an interface
[16:14] <ELFrederich> http://py4j.sourceforge.net/advanced_topics.html
[16:14] <ELFrederich> found it.. thanks
[16:56] <jimbaker> agronholm, ELFrederich - so my concern with something like Py4J compared to RPyC is the security model, namely http://py4j.sourceforge.net/faq.html#are-there-any-security-concerns-with-py4j
[18:58] <ELFrederich> jimbaker, thanks for the link. I think RPyC uses ssh or something (though not by default)
[18:58] <ELFrederich> jimbaker, I'd like to investigate getting RPyC's netref objects to get coerced into Java arrays in Jython. It would make using the two together much easier.
[18:59] <ELFrederich> Any idea where to look in the code for this? Maybe its a simple fix with a couple of lines if Jython is checking the type to be a python list I'd just add a netref to a python list as well
[19:14] <agronholm> jimbaker: I suggested a subprocess and pipes as an alternative but py4j has been coded in a way that makes it impossible without significant refactoring
[19:15] <ELFrederich> agronholm, I'm leaning towards the RPyC/Jython solution because it would solve another problem for us...
[19:16] <agronholm> what problem
[19:16] <ELFrederich> Our application runs on Linux and needs to use a SmartCard to log in... this is a problem when users are working remotely on a Windows laptop.
[19:16] <agronholm> I'm not following
[19:16] <ELFrederich> they'll have their SmartCard with them so it won't be on the Linux machine
[19:17] <ELFrederich> with RPyC they could use VNC to run a Linux application that talks to this 3rd party Java app on Windows
[19:18] <agronholm> I'm thoroughly confused :)
[19:18] <ELFrederich> agronholm, let me try again... hang on
[19:19] <ELFrederich> agronholm, This 3rd party application is Teamcenter. We need to access it from CPython. To do this we think it'll be easier going through Java than trying to create bindings to their C++ API...
[19:19] <ELFrederich> agronholm, normal usage would be that the user is in the office in front of a Linux machine
[19:20] <ELFrederich> agronholm, this application written in CPython would be talking to the Teamcenter APIs through Java running in another process (Jython/RPyC)
[19:20] <ELFrederich> agronholm, this solution also has the benefit of being able to work in a different way...
[19:21] <ELFrederich> agronholm, when a user is working from home on a Windows laptop they can use VNC to connect to their Linux workstation...
[19:21] <ELFrederich> agronholm, the problem there is that without their SmartCard being in the Linux workstation they cannot log into Teamcenter
[19:22] <ELFrederich> agronholm, but... since this is an RPC setup, they could just run Jython/RPyC on Windows (where they can insert their SmartCard) and have the Linux application use that connection
[19:22] <ELFrederich> ... done ;)
[19:22] <ELFrederich> not sure if that is more or less clear now
[19:22] <agronholm> it sounds like it'd be easier to do the whole thing in Java
[19:23] <agronholm> or even Jython
[19:23] <ELFrederich> agronholm, too many CPython things we need to use. We have an in-house C++ framework that is exposed to CPython via boost. Also the GUI will be written in PyQt
[19:24] <agronholm> couldn't that framework be exposed as a remote service somehow?
[19:25] <ELFrederich> don't get me started... I had a whole prototype of using services. My colleagues couldn't grasp it. Keep mumbling about performance
[19:25] <agronholm> I can't comment on that without knowing what the framework actually does
[19:26] <agronholm> I take it that a windows build is out of the question then?
[19:26] <ELFrederich> agronholm, this is a system that can be programmed in C++, Python, and Tcl. Originally they wanted Perl support too. I suggested just using Redis as the datastore since Redis has like 50 different language bindings
[19:27] <ELFrederich> agronholm, the framework is just a glorified data store
[19:27] <agronholm> I see
[19:27] <agronholm> if redis could've done the job then redis or redislabs (better) would've been a better choice
[19:27] <ELFrederich> agronholm, we aren't targeting Windows now but it won't be out of the question. We're using platform independent technologies (Python, PyQt, Boost, etc)
[19:28] <agronholm> I personally prefer writing as little code as I possibly can
[19:28] <agronholm> the less code I have to test and support the better
[19:29] <ELFrederich> agronholm, I agree... I really wanted to use Redis, but my colleagues got hung up on serializing numbers into strings back into numbers again
[19:29] <agronholm> oh, Decimals?
[19:30] <agronholm> they were concerned about the loss of accuracy...?
[19:30] <ELFrederich> Huge arrays of floating point numbers.
[19:30] <ELFrederich> I told them they wouldn't lose accuracy. At least from Python it is guaranteed that serializing a float to string back to float gives you the same float
[19:31] <ELFrederich> I don't know how the other Redis APIs work.
[19:32] <ELFrederich> but there were legit concerns about performance of storing huge 3d,4d, or 5d arrays in Redis.
[19:32] <agronholm> and this in-house framework performs better there?
[19:32] <ELFrederich> I just think those concerns are going to have to be overcome either way.... probably better in-memory ways to store them than in Redis.
[19:32] <ELFrederich> oh well... now talking about this I'm getting upset about this project
[19:33] <ELFrederich> agronholm, that'll be something I'll do over a weekend. Show that I can store a ton more data in Redis running in a different process, serializing to strings faster than I can go through this in-house C++ to boost stuff
[19:34] <agronholm> heh
[19:34] <ELFrederich> agronholm, are you familiar with Jython's source?
[19:34] <agronholm> somewhat
[19:34] <ELFrederich> git log | grep -i agronholm | wc -l
[19:34] <ELFrederich> agronholm, that shows me 4 ;-)
[19:35] <agronholm> I haven't committed a lot of code
[19:35] <ELFrederich> so I guess that equates to "somewhat"
[19:35] <ELFrederich> yeah... those 4 were basically thank yous
[19:35] <agronholm> my biggest contributions are the issues I've found and created
[19:35] <agronholm> created, as in created bug reports
[19:35] <ELFrederich> anyway... any idea where the coercion from Python lists to Java arrays is located?
[19:36] <agronholm> I don't know. but if you want to create java arrays from python, you can do that with array.array
[19:36] <ELFrederich> jarray.array you mean?
[19:36] <agronholm> no, array.array
[19:37] <ELFrederich> agronholm, when using Jython directly I can do a call like someService.getProperties([some_object, another_object], ['prop1', 'prop2'])
[19:37] <ELFrederich> when using it over RPyC I have to use jarray.array to create those 2 arrays
[19:38] <agronholm> most likely rpyc's type coercion is not as sophisticated as that of jython
[19:39] <ELFrederich> agronholm, I suspect that the fault is Jython's only whitelisting certain container types
[19:42] * Trundle (~andy@python/site-packages/trundle) Quit (Remote host closed the connection)
[20:12] * enebo (~enebo@c-75-73-8-169.hsd1.mn.comcast.net) Quit (Quit: enebo)
[20:31] <jimbaker> ELFrederich, if i understand your problem, you would want to implement __tojava__, presumably for the netref objects. this can be done in python in most cases.
[20:32] <jimbaker> you might find this example useful - http://www.jython.org/jythonbook/en/1.0/Concurrency.html#interruption
[20:33] <jimbaker> it shows how to monkeypatch a __tojava__ method onto a class
[20:34] <jimbaker> it came up because i wanted to demonstrate using interruption, and realized jython's implementation of threading.Thread lacked __tojava__. so i fixed this aspect in jython, and wrote up the workaround. fun stuff :)
[20:36] <jimbaker> there are other variants that might apply. it came up yesterday in fact with respect to datetime support - see for example https://hg.python.org/jython/rev/67210f355576#l1.99
[20:49] * enebo (~enebo@166.175.186.74) has joined #jython
[20:57] * enebo (~enebo@166.175.186.74) Quit (Ping timeout: 256 seconds)
[20:58] * enebo (~enebo@166.175.60.45) has joined #jython
[21:20] * mbooth (~mbooth@redhat/mbooth) Quit (Ping timeout: 264 seconds)
[21:34] * mbooth (~mbooth@cpc11-shef10-2-0-cust659.barn.cable.virginm.net) has joined #jython
[21:34] * mbooth (~mbooth@cpc11-shef10-2-0-cust659.barn.cable.virginm.net) Quit (Changing host)
[21:34] * mbooth (~mbooth@redhat/mbooth) has joined #jython
[21:43] <ELFrederich> jimbaker, thanks for the reply. I'll take a look there
[21:44] <ELFrederich> jimbaker, any idea when 2.7 will be officially released? The other problem is that I had to patch both RPyC and its dependency plumbum because they were using a "with" statement as well as catching exceptions with the word "as"
[21:44] <jimbaker> ELFrederich, well, i have been pretty wrong about dates so far
[21:45] <jimbaker> so major grains of salt to be applied here...
[21:45] <ELFrederich> grains of salt taken
[21:45] <jimbaker> but right now we have only one outstanding urgent bug, which is a blocker for our first release candidate
[21:45] <jimbaker> (and perhaps only RC)
[21:46] <jimbaker> there are a few high bugs that would be nice to get in. but we are certainly very close. i might even say next week
[21:46] <ELFrederich> jimbaker, cool... thanks
[21:47] <ELFrederich> what is the bug related to?
[21:47] <ELFrederich> ... maybe I won't care about it and then I can just use 2.7
[21:47] <jimbaker> ELFrederich, np. i think this might be closer to truth this time around, since the end of the beta 4 contained most of what we wanted to defer to RC1 :)
[21:47] <jimbaker> ELFrederich, it's related to __slots__, specifically support for __dict__ as a slot name
[21:48] <jimbaker> werkzeug uses this in its object proxy
[21:48] * ELFrederich read about slots in some book 5 years ago but has never used them
[21:48] <jimbaker> http://bugs.jython.org/issue2272
[21:48] <jimbaker> ELFrederich, right, you're most likely to see it in someone else's package
[21:49] <jimbaker> so i have a workaround, which i'm mulling as being the fix
[21:50] <ELFrederich> jimbaker, I'm already forking RPyC because it doesn't work on 2.5 so I don't need to monkeypatch, I can do a real patch
[21:50] <ELFrederich> jimbaker, the question is... what do I need to put in there?
[21:50] <jimbaker> right now, we are too strict in restricting usage of how we implement checking on what's permissible on __slots__. my workaround is weaker than CPython, but it's convoluted to get to what CPython actually checks for
[21:51] <jimbaker> ELFrederich, well, what i would do is figure out what the java conversion should be. then use that monkeypatch decorator in your code. then upstream your patch to RPyC if that makes sesne
[21:51] <jimbaker> sense
[21:52] <ELFrederich> jimbaker, its just a netref to __builtin__.list
[21:52] <ELFrederich> jimbaker, so it should be the same conversion that a normal Python list is
[21:53] <ELFrederich> haha... looking at RPyC and seeing that it uses __slots__ ;-)
[21:54] <ELFrederich> https://github.com/tomerfiliba/rpyc/blob/v3.3/rpyc/core/netref.py
[21:55] * enebo (~enebo@166.175.60.45) Quit (Ping timeout: 244 seconds)
[21:56] * enebo (~enebo@166.175.186.171) has joined #jython
[22:16] * enebo_ (~enebo@166.175.185.141) has joined #jython
[22:17] * enebo (~enebo@166.175.186.171) Quit (Ping timeout: 272 seconds)
[22:21] * enebo (~enebo@166.175.59.60) has joined #jython
[22:21] * enebo_ (~enebo@166.175.185.141) Quit (Ping timeout: 250 seconds)
[22:39] <ELFrederich> jimbaker, here is a small example. It has a jython_example.py which works, and an rpyc_example.broke.py which I'd like to get working. I have an ugly working version called rpyc_example.working.py https://bitbucket.org/ericfrederich/jythonrpyc/src
[22:39] <ELFrederich> its ugly because I have to use jarray.array to create the java arrays
[22:39] <ELFrederich> just a Bar class with a method that takes an array of Foo objects
[22:40] <jimbaker> ELFrederich - re __slots__ - this should work, unless it hits a TypeError because of the overly strict policy that we apply (i don't see this being the case in this code however)
[22:41] <ELFrederich> I'll try to play around with monkey patching netref with __tojava__. I'm not sure if I need to do it client side (meaning in the rpyc_example.broke.py) or serverside (meaning the ./rpyc/bin/rpyc_classic.py)
[22:44] <jimbaker> ELFrederich, i would presume both ends, to be uniform
[22:44] <jimbaker> or at least the ends running java
[22:58] * ELFrederich (~ELFrederi@63.87.61.77) Quit (Remote host closed the connection)
[22:58] * ELFrederich (~ELFrederi@12.54.94.28) has joined #jython
[23:04] <ELFrederich> jimbaker, I switched that bitbucket repo to point to the upstream versions of rpyc and plumbum because it was pointing to my private github repo before. I'll try to work on this tonight and see if I can get __tojava__ working serverside
[23:05] <jimbaker> ELFrederich, thanks, keep us posted on your progress!
[23:17] * enebo (~enebo@166.175.59.60) Quit (Ping timeout: 256 seconds)
[23:18] * xemdetia (xemdetia@nat/ibm/x-hhgpusqrmfjnzwjz) Quit (Ping timeout: 244 seconds)
[23:21] <jimbaker> peke, looks like we already have a fix for the backslash bug, just need to update the jline2 jar
[23:21] <jimbaker> same problem seen with other jline2 users like scala or leiningen
[23:23] <jimbaker> so after our turkish bug fix, we get a finnish (and other altgr using keyboard layouts) bug fix. all things considered, i do think we are getting close to a RC!
[23:25] * mbooth (~mbooth@redhat/mbooth) Quit (Ping timeout: 244 seconds)
These logs were automatically created by JythonLogBot_ on irc.freenode.net using a slightly modified version of the Java IRC LogBot (github).