Timestamps are in GMT/BST.
[0:22] <agronholm> I'm seeing something very strange stuff related to sockets when the remote disconnects
[0:22] <agronholm> WARNING: An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
[0:26] * fwierzbicki (~Adium@99-106-169-5.lightspeed.sntcca.sbcglobal.net) Quit (Quit: Leaving.)
[0:40] <agronholm> I need to investigate this thoroughly
[0:56] * fwierzbicki (~Adium@99-106-169-5.lightspeed.sntcca.sbcglobal.net) has joined #jython
[1:06] <apollonovich> I'm sure I'm missing something obvious, but is there a straightforward way of packaging additional python modules into the jython jar that comes from maven? E.g., I'd like to grab a few things from pip, and shove them into jython.jar/Libs at build time so that I need no hard-coded python paths in my java code.
[1:23] * jimbaker (~jbaker@97.sub-70-196-194.myvzw.com) has joined #jython
[1:26] * jimbaker (~jbaker@97.sub-70-196-194.myvzw.com) Quit (Client Quit)
[1:41] <xemdetia> apollonovich, did you already explore this: http://stackoverflow.com/a/7078178
[1:41] <xemdetia> I have no idea if it works or not I am just curious
[1:42] <apollonovich> I have looked at it. I"m not using maven directly (I'm using apache buildr, which uses maven2 for dependency management, but that's about it), but I should perhaps take a closer look/see if I can invoke a maven plugin from buildr without too much agony.
[1:43] <xemdetia> I see
[1:43] <apollonovich> I have a dependency on pkg_resources because I have a large blob of static data that exceeds the jvm's 64kB class size limitation. pkg_resources is a part of relatively recent setuptools versions, and that module is not packaged with jython-standalone.
[1:44] <apollonovich> it may be that simply requiring a system installation of python2.7, and then finding pkg_resources.py in the standard PYTHONPATH location is easier.
[1:45] <xemdetia> Could you do something like this with buildr? http://docs.oracle.com/javase/tutorial/deployment/jar/update.html Just call jar to patch those files into the jython.jar as the next step?
[1:47] <apollonovich> oh, for sure.
[1:47] <apollonovich> that's the plan. I wish there was an easy way to just use pip to pull in the dependencies.
[1:48] <apollonovich> as it is, I fear I'm going to be wget'ing specific versions of them, untarring them, and then copying the files I want into Lib, before updating the jar.
[1:48] <apollonovich> which all feels kinda brittle, but at least will work.
[1:48] <xemdetia> yeah at least there is a centralized place to get some of these libs through pypi though
[1:48] <xemdetia> as opposed to how it was some years aback
[1:48] <apollonovich> yeah. so I think if I pin version #, I can wget from pypi with reasonable confidence.
[1:48] <apollonovich> yeah.
[1:49] <xemdetia> I haven't tried the jar update thing I just assume it will work
[1:49] <xemdetia> Trying to be a little creative late at night :)
[1:50] <apollonovich> I'd tried using autojar at one point, but it was screwing up the manifest stuff, and wasn't worth the trouble, but I trust if it's in oracle's docs, it should work reasonably well.
[1:50] <xemdetia> That's my feeling- hopefully the actual jar tool will not mangle jars
[2:16] <apollonovich> I think I'm going to end up writing a buildr module for this in ruby...if it does anything beyond the perfunctory, perhaps I'll post it up as a gist and paste a link into this channel when I'm done.
[2:23] <xemdetia> That is one thing I don't like about some of the convention over configuration-class build tools
[2:23] <xemdetia> when you have to deviate it can be really a huge pain
[2:33] <apollonovich> yup
[2:51] * craigez1 (~craign@2001:8003:601a:f401:d30:5485:5816:6043) has left #jython
[3:04] * apollonovich (~rdowne@c-50-131-40-240.hsd1.ca.comcast.net) Quit (Quit: Leaving)
[3:50] * lopex_ (uid4272@gateway/web/irccloud.com/x-kzetwmsqlcpqhbwo) has joined #jython
[3:50] * lopex_ is now known as Guest97469
[3:51] * pjenvey_ (pjenvey@underboss.org) has joined #jython
[3:53] * lopex (uid4272@gateway/web/irccloud.com/x-acoccffpmpjgkhif) Quit (Ping timeout: 250 seconds)
[3:53] * pjenvey (pjenvey@underboss.org) Quit (Ping timeout: 250 seconds)
[3:53] * Guest97469 is now known as lopex
[3:58] * xemdetia (xemdetia@nat/ibm/x-zoyaefualvdtafef) Quit (Ping timeout: 272 seconds)
[4:54] * Oti (~ohumbel@adsl-178-39-145-123.adslplus.ch) Quit (Quit: Oti)
[5:47] * lheuer (~Adium@unaffiliated/lheuer) has joined #jython
[6:27] * lheuer (~Adium@unaffiliated/lheuer) Quit (Quit: Leaving.)
[6:41] * lheuer (~Adium@unaffiliated/lheuer) has joined #jython
[7:02] * ohumbel (5390f784@gateway/web/freenode/ip.83.144.247.132) Quit (Quit: Page closed)
[8:09] * ohumbel (5390f784@gateway/web/freenode/ip.83.144.247.132) has joined #jython
[8:18] * yecril71pl (51d2144a@gateway/web/freenode/ip.81.210.20.74) has joined #jython
[8:19] * yecril71pl (51d2144a@gateway/web/freenode/ip.81.210.20.74) Quit (Changing host)
[8:19] * yecril71pl (51d2144a@opensuse/member/yecril71pl) has joined #jython
[8:21] <yecril71pl> Hello, I have got a bug in registering a socket for polling in Windows
[8:21] <yecril71pl> The message is: TypeError: ("Object '<socket.ssl instance at 0x202>' is not watchable", 10038)
[8:22] <yecril71pl> Could you give me a hint?
[8:22] <yecril71pl> As you can see, the socket has been wrapped (because it talks https)
[8:25] <yecril71pl> I changed except: to except None: in socket.py
[8:26] <yecril71pl> (I really hate to change bundled libraries but it seems I have to)
[8:26] <yecril71pl> and now it says: AttributeError: ssl instance has no attribute 'fileno'
[8:27] <yecril71pl> So what is the point of transforming an attribute error into a type error?
[8:27] <yecril71pl> It gives less info to the victim :-(
[8:34] <yecril71pl> It seens that the SSL wrapper fails to forward the fileno to the underlying TCP socket
[8:36] <yecril71pl> Search for fileno in bugs returned nothing relevant
[8:41] <yecril71pl> def fileno (self): return self .jython_socket_wrapper .fileno()
[8:43] <yecril71pl> I did that and now I get: error: (20000, 'socket must be in non-blocking mode')
[8:43] <yecril71pl> so it makes no sense to poll a blocking socket
[8:53] <yecril71pl> How do I catch such an exception?
[8:58] <yecril71pl> I pasted java.nio.channels.IllegalBlockingModeException and Eclipse got OOM from Git
[8:58] <yecril71pl> What a mess
[8:58] * lheuer (~Adium@unaffiliated/lheuer) Quit (Quit: Leaving.)
[9:14] <agronholm> yecril71pl: you're having socket trouble too?
[9:14] <agronholm> yecril71pl: which version of jython?
[9:14] <yecril71pl> How do I tell that?
[9:14] <agronholm> what did you install it from?
[9:14] <yecril71pl> I am on PyDev
[9:15] <agronholm> yeah but to run jython you have to register a jython interpreter
[9:15] <yecril71pl> I have.
[9:15] <agronholm> which one then?
[9:16] <agronholm> if you registered one, surely you installed it
[9:16] <yecril71pl> 3.6.0.201406232321
[9:16] <agronholm> um, no
[9:17] <agronholm> that is not a jython version
[9:17] <agronholm> import sys; print(sys.version)
[9:18] <yecril71pl> 2.7b2
[9:18] <agronholm> 2.7b3 enabled polling on blocking sockets
[9:18] <agronholm> but I've been seeing other problems
[9:19] <yecril71pl> I believe I can live with this non-polling
[9:19] <yecril71pl> The non-blocking socket probably will not have anything to say anyway
[9:19] <yecril71pl> so I may just assume it is exhausted
[9:20] <yecril71pl> However, I would like to know how to tell what exception type is thrown
[9:20] <yecril71pl> I tried SocketError but it did not get handled
[9:20] <agronholm> from what operation
[9:20] <yecril71pl> from register
[9:21] <yecril71pl> (SocketError being an alias for socket.error)
[9:21] <agronholm> sorry, register on what
[9:21] <yecril71pl> on poll
[9:22] <yecril71pl> that is, tell poll to register a socket for input (or something like this)
[9:22] <agronholm> 2.7b3 got a lot of socket related changes
[9:22] <agronholm> which improved compatibility with CPython
[9:23] <agronholm> in particular SSL was problematic before that
[9:23] <yecril71pl> yes, this particular socket is https
[9:24] <yecril71pl> the exception is select.error
[9:25] <yecril71pl> not socket.error
[10:07] * yecril71pl (51d2144a@opensuse/member/yecril71pl) Quit (Ping timeout: 246 seconds)
[10:09] <agronholm> pjenvey_, fwierzbicki: do you know what jim is doing? he's not been around here lately
[10:13] * yecril71pl (51d2144a@gateway/web/freenode/ip.81.210.20.74) has joined #jython
[10:13] * yecril71pl (51d2144a@gateway/web/freenode/ip.81.210.20.74) Quit (Changing host)
[10:13] * yecril71pl (51d2144a@opensuse/member/yecril71pl) has joined #jython
[10:13] <yecril71pl> Error: distutils.dist.user is not set
[10:17] <yecril71pl> Error: distutils.dist.command_obj.user is not set
[10:27] <yecril71pl> I understand that "user" is supposed to mean that the package should be installed for the user only
[10:28] <yecril71pl> but there is no such option in dist/command/install.py
[10:28] <yecril71pl> I wonder if it is documented anywhere
[10:30] <yecril71pl> <URL: https://docs.python.org/2/install/index.html#alternate-installation-the-user-scheme >
[10:44] <yecril71pl> setup install --help does not give me --user
[10:44] <yecril71pl> even though the specification requires it
[10:48] * RMBM (~RMBM@unaffiliated/xori) has joined #jython
[10:49] <RMBM> Hey there, I need some help with setting virtualenv for django, I googled a bit but nothing seems to be working.
[10:51] <yecril71pl> The thing is, virtualenv includes pip, and pip is bro-ken.
[10:52] <yecril71pl> It is broken partly because it is broken and partly beacause Jython is substandard.
[10:53] <RMBM> hmm that means there's no way to run virtualenv using jython ? can't we make virtualenv use easy_install instead ?
[10:54] * jimbaker (~jbaker@24.9.252.60) has joined #jython
[10:56] <RMBM> Also unsetting JYTHON_HOME doesn't seems to affect. I still get the not compatible message just like this issue: https://github.com/pypa/virtualenv/issues/185
[11:00] <yecril71pl> I have been working on it for three days now.
[11:01] <yecril71pl> You need to write your own Jython startup script to satisfy this requirement.
[11:01] <yecril71pl> And to say -Dphython.executable=???
[11:01] <yecril71pl> And to say -Dpython.executable=???
[11:02] <yecril71pl> But this is only the beginning.
[11:11] <yecril71pl> issue 2189 created
[11:11] <RMBM> thanks
[11:22] <RMBM> What if I already have an existing virtualenv, can I activate into it with jython ?
[11:25] <agronholm> RMBM: no, virtualenvs don't work that way
[12:06] * jimbaker (~jbaker@24.9.252.60) Quit (Quit: Computer has gone to sleep.)
[12:24] * enebo (~enebo@c-75-73-8-169.hsd1.mn.comcast.net) has joined #jython
[12:36] * xemdetia (xemdetia@nat/ibm/x-xeuvgdzjfbbyzszy) has joined #jython
[12:54] <yecril71pl> unichr (0xD800) fails in displayhook with "string index out of bounds"
[12:57] * smaudet (~smaudet@206-51-157-254.nktelco.net) has joined #jython
[12:58] * xemdetia_ (xemdetia@nat/ibm/x-xspzhwjlrdxgayde) has joined #jython
[13:00] * xemdetia (xemdetia@nat/ibm/x-xeuvgdzjfbbyzszy) Quit (Ping timeout: 240 seconds)
[13:15] <agronholm> yecril71pl: create an issue
[13:15] <agronholm> I found another socket bug while investigating that previous one
[13:15] <agronholm> recv() from a dead socket should return b'' but returns None instead
[13:18] <agronholm> yecril71pl: it's actually the __repr__() that dies
[13:18] <agronholm> not unichr() itself
[13:21] * lheuer (~Adium@unaffiliated/lheuer) has joined #jython
[13:22] <agronholm> I may be able to fix some of these bugs myself
[13:27] <agronholm> yecril71pl: oh...D800 is a unicode surrogate character
[13:27] <agronholm> yecril71pl: java does not allow single surrogates
[13:27] <agronholm> it must be a pair always
[13:28] <yecril71pl> Of course, but "index out of bounds"?
[13:28] <agronholm> not sure what it should do
[13:28] <yecril71pl> UnicodeDecodeException?
[13:28] <agronholm> but it's not being decoded?
[13:28] <yecril71pl> Internally it is.
[13:29] <agronholm> the user doesn't need to know that
[13:29] <agronholm> still
[13:29] <agronholm> maybe a check is in order
[13:29] <agronholm> check if this is the last character in the string
[13:30] <yecril71pl> Maybe replace it with the <?> character?
[13:30] <agronholm> no
[13:30] <agronholm> heh this is odd
[13:30] <agronholm> line 243: /* Fall through: isolated surrogates are copied as-is */
[13:30] <agronholm> but it never gets there since it never checks for isolated surrogates!
[13:31] <yecril71pl> OK, you know better then :-)
[13:31] <agronholm> okay the fix should be trivial
[13:31] <yecril71pl> Do you still want me to create the issue?
[13:31] <agronholm> hrm
[13:32] <agronholm> not sure if there is a point
[13:33] <yecril71pl> can you comment on issue 2189?
[13:33] <agronholm> is this a distutils feature?
[13:34] <yecril71pl> it is a failure to implement the command syntax
[13:34] <agronholm> I don't understand
[13:34] <yecril71pl> <URL: https://docs.python.org/2/install/index.html#alternate-installation-the-user-scheme >
[13:34] <yecril71pl> you do not support a mandatory option
[13:35] <agronholm> setup is a distutils thing and it's pure python so I don't quite understand why it would not be supported since we just copy cpython's stdlib
[13:36] <yecril71pl> Interesting. I do not have any explanation for that.
[13:37] <yecril71pl> Or maybe again my installation is botched?
[13:38] <agronholm> I'll probably look at it later
[13:38] <agronholm> let's see if my unicode fix worked
[13:39] <yecril71pl> It is not botched, --user is missing from stand-alone Jython too
[13:41] <agronholm> yecril71pl: http://bpaste.net/show/pIN3EphnvrxUDTt14r1Y/
[13:42] <yecril71pl> perfect.
[13:42] <agronholm> this was a one line fix :D
[13:44] <yecril71pl> install.py in CPython is different from install.py in Jython
[13:45] <yecril71pl> The difference is that --user is missing in Jython.
[13:46] <agronholm> distutils/command/install.py?
[13:46] <yecril71pl> Yep, the one from CPython says "$Id$", so it does not introduce itself correctly.
[13:48] <yecril71pl> Meaning it is not possible to compare their releases.
[13:48] <agronholm> the thing with jython is that we have module overrides
[13:48] <agronholm> it's possible that install.py was not properly updated when switching to 2.7
[13:48] * xemdetia_ (xemdetia@nat/ibm/x-xspzhwjlrdxgayde) Quit (Ping timeout: 240 seconds)
[13:48] <yecril71pl> <@agronholm> we just copy cpython's stdlib
[13:49] <yecril71pl> the question is which one :-)
[13:49] <agronholm> yes, but we also override certain modules
[13:49] <yecril71pl> I understand, and necessarily so
[13:49] <agronholm> and if the override is not updated when the CPython library is updated...
[13:49] <yecril71pl> Then you were spreading misinformation in your original statement :-)
[13:49] <agronholm> I feel I should discuss this with the other devs before doing anything drastic
[13:50] <agronholm> it was incomplete, yeah
[13:50] <agronholm> I wasn't aware that install.py was being overridden
[13:51] <agronholm> huh.
[13:51] <yecril71pl> My situation is that I need to patch pip because pip expects --user to work and it breaks without --user.
[13:51] <agronholm> the difference is minimal
[13:51] <agronholm> I wonder what's going on
[13:51] <yecril71pl> But the patch is technically invalid because --user is documented and required.
[13:52] <yecril71pl> Of course, I can always code defensively and account for the situation without --user
[13:52] <agronholm> I'll figure it out
[13:52] <yecril71pl> but such defensive code may be dismissed
[13:53] <agronholm> I will commit the unicode fix later
[13:53] <yecril71pl> as an unnecessary complication
[13:53] <agronholm> I need to do my real job now
[13:53] <yecril71pl> I use Jython in my real job :-)
[13:53] <agronholm> as do I
[13:54] <agronholm> though in only one project
[14:03] <agronholm> yecril71pl: it might be good to actually have an issue about the unichr thing
[14:03] <agronholm> so I could reference it in the test
[14:03] <yecril71pl> Do you want me to create the issue?
[14:04] <agronholm> please do
[14:04] * xemdetia (xemdetia@nat/ibm/x-rhmamiaytnmouxqa) has joined #jython
[14:05] * siel (~siel@unaffiliated/motley) Quit (Ping timeout: 260 seconds)
[14:06] * jimbaker (~jbaker@63-253-67-243.ip.mcleodusa.net) has joined #jython
[14:07] * siel (~siel@unaffiliated/motley) has joined #jython
[14:08] <yecril71pl> issue 2190 created
[14:11] <agronholm> thanks
[14:35] <yecril71pl> re.compile(u"%c-\U0001FFFE" % 0xD800) : string index out of range
[14:36] <agronholm> breaks even with my new build :(
[14:37] <yecril71pl> bug or comment?
[14:37] <agronholm> it's a similar bug but in a totally different part of the codebase
[14:37] <agronholm> new issue please
[14:41] <agronholm> jimbaker: just the person I wanted to see
[14:41] <jimbaker> agronholm, yes, i'm back
[14:41] <agronholm> jimbaker: were you aware that socket.recv() returns None when no data is available?
[14:41] <agronholm> it should be returning b''
[14:42] <agronholm> this is with the latest incarnation of b3
[14:42] <yecril71pl> issue 2191 created
[14:42] <jimbaker> agronholm, we can readily fix that
[14:42] <agronholm> yep
[14:42] <agronholm> I ran into a more complex issue earlier in which a thread crashed with java.lang.IOError
[14:42] <agronholm> I haven't been able to reproduce it trivially
[14:43] <agronholm> but it involves send()ing with a socket that is being disconnected by the remote side
[14:43] <jimbaker> agronholm, it certainly sounds a lot easier than managing the lifecycle of sockets, which i think is more or less done at this point
[14:43] <jimbaker> agronholm, or perhaps not if you are getting this IOError
[14:44] <jimbaker> anyway, if we can reproduce, we can fix
[14:44] <agronholm> I'm working on it
[14:44] <agronholm> I just recently moved to jython 2.7 in my big app
[14:44] <agronholm> 2.5 was a deal breaker since I wanted to use websockets
[14:44] <agronholm> no websocket library works with 2.5 syntax
[14:45] <jimbaker> yecril71pl, sorry about the lack of support for lone surrogates. this just breaks the underlying utf-16 encoding
[14:45] <yecril71pl> html5lib requires it
[14:45] <jimbaker> i have thought about supporting alternative encodings, just like what is done in coython3
[14:45] <agronholm> jimbaker: I just fixed the first problem, I just need to add a test for it
[14:45] <jimbaker> yecril71pl, not any more, once this pull request goes through
[14:45] <jimbaker> https://github.com/html5lib/html5lib-python/pull/150
[14:46] <agronholm> should I add a new test in tests/python/ for it?
[14:46] <agronholm> or where should I put tests that test the core classes?
[14:46] <jimbaker> oh, looks like i had trailing whitespace. well i will fix that
[14:47] <jimbaker> agronholm, test_unicode_jy.py would be the natural place to put such a test
[14:48] <agronholm> ah okay
[14:48] <yecril71pl> jimbaker: your pull request fails
[14:49] <jimbaker> yecril71pl, yeah, i saw it failed a flakes test for trailing whitespace. darn how such things sneak in
[14:50] <yecril71pl> Did you manage to convince gsnedders?
[14:50] <jimbaker> yecril71pl, yes, gsnedders had a great set of discussions at europython
[14:50] <jimbaker> with me
[14:51] <yecril71pl> Should I just pull your html5lib then?
[14:52] <jimbaker> one thing that we did discuss is reopening this for jython 3.x
[14:52] <jimbaker> yecril71pl, please, it would be useful for at least additional testing
[14:53] <jimbaker> the scope of the change is extremely restricted, but always best to have it used
[14:58] <yecril71pl> I do not like this utils.supports thing. It looks like User-Agent testing for Python.
[14:59] <yecril71pl> Why can???t you just say if sys.name == 'java'?
[15:02] <yecril71pl> Why do you need unichr? I use (u'%c' % ???) instead.
[15:03] <yecril71pl> Why do you import unichr from six?
[15:03] <agronholm> jimbaker: is test_unicode_jy.py better than adding a test in bugtests/ ?
[15:03] <jimbaker> agronholm, yes, very much so
[15:04] <agronholm> alright
[15:04] <jimbaker> best to mention the specific bug being fixed, but it helps keep things together better
[15:04] <agronholm> I did
[15:04] <agronholm> pushed the commit now
[15:06] <yecril71pl> I think I shall test for issue 2919 and remove the surrogates if it fails
[15:06] <yecril71pl> I believe it has very little chance of surviving a pull request though
[15:07] <agronholm> jimbaker: I hadn't read your comment on the issue
[15:08] <agronholm> I was about to close the issue >_<
[15:08] <jimbaker> yecril71pl, really? what are your concerns in addition to what was discussed on the critic review/
[15:08] <jimbaker> ?
[15:08] <agronholm> I guess it was wrong then? made jython's behavior match cpython's
[15:09] <agronholm> this is why I'm reluctant to commit *anything* to jython...
[15:10] <jimbaker> agronholm, well this is a sort of fundamental question here for jython
[15:10] <jimbaker> we are on java. so jython should use java strings (wrapping as necessary), java gc, java threading model, etc
[15:10] <agronholm> jimbaker: I did this because the existing comments hinted at lone surrogates being fine
[15:11] <agronholm> see line 243 of PyString.java
[15:11] <agronholm> at any rate, this fix is better than throwing a java exception
[15:12] <jimbaker> agronholm, hmmm, that comment is not a good one
[15:12] <agronholm> I couldn't have known that
[15:15] <jimbaker> agronholm, i understand. sorry about that. i don't know the implication of this piece of code. in general, isolated surrogates are not going to work, although i suppose it's possible to construct say single character strings where it would appear to be fine
[15:15] <jimbaker> and maybe other examples, because of the way the escaping works
[15:19] <paolo> guys, could you have a look at this traceback? https://gist.github.com/paolodina/611faca32e36bb73fb26
[15:19] <paolo> in jython i'm not able to do from flask_oauthlib.client import OAuth
[15:19] <paolo> in python it works
[15:20] <yecril71pl> jimbaker: do we agree that testing User-Agent strings is wrong?
[15:20] <yecril71pl> Because if we do, this is exactly what your implementation does.
[15:20] <jimbaker> yecril71pl, i'm sorry, i don't understand your point
[15:21] <yecril71pl> When programming JavaScript, we are told never to ask "Who are you?", but rather "Can you do that?"
[15:21] <yecril71pl> Your implementation of supports, etc., asks who the executor is.
[15:21] <jimbaker> yecril71pl, right. so unichr(0xD800) should raise a ValueError. cannot do that
[15:22] <yecril71pl> Why can???t you?
[15:23] <jimbaker> yecril71pl, let's say hypothetically we had unicode string support that provides an alternative encoding. now, let's try to call a java method with this unicode string. what behavior should we expect?
[15:23] <jimbaker> so we can push this boundary around, but isolated surrogates are in fact illegal unicode
[15:23] <yecril71pl> But we do not and we shall not.
[15:24] <yecril71pl> have an alternative encoding.
[15:24] <agronholm> yecril71pl: what is your use case for lone surrogates?
[15:24] <yecril71pl> I fully support jimbaker that it should throw a ValueError.
[15:24] <yecril71pl> The problem is, jimbaker says it cannot.
[15:25] <agronholm> wut
[15:25] <jimbaker> yecril71pl, i'm sorry, i think there may be a misunderstanding then. i certainly want unichr(0xD800) to throw a ValueError
[15:25] <yecril71pl> [17:21] <jimbaker> yecril71pl, right. so unichr(0xD800) should raise a ValueError. cannot do that
[15:25] <agronholm> the "cannot do that" didn't refer to raising the ValueError
[15:26] * lheuer (~Adium@unaffiliated/lheuer) Quit (Quit: Leaving.)
[15:26] <yecril71pl> What did it refer to then?
[15:26] <jimbaker> exactly, it was in reference to this capability model "can you do that"
[15:26] <jimbaker> sorry about the confusion here
[15:27] <yecril71pl> Yes, this should be fixed ASAP
[15:27] <jimbaker> ok, very good
[15:30] <yecril71pl> paolo: have you got oauthlib/oauth2/base.py ?
[15:30] <jimbaker> paolo, looks like a problem in starred relative imports
[15:31] <paolo> i think base should be shipped with oauthlib library, let me check
[15:32] <paolo> yecril71pl: ^--
[15:33] <yecril71pl> paolo: excuse me, first check and then write
[15:33] <yecril71pl> I have to do a context switch to learn that you have to check :-(
[15:34] * enebo_ (~enebo@c-75-73-8-169.hsd1.mn.comcast.net) has joined #jython
[15:35] * jimbaker_ (~jbaker@63-253-67-243.ip.mcleodusa.net) has joined #jython
[15:35] <jimbaker_> (i really need to get my bouncer fixed)
[15:40] <paolo> yecril71pl: it is, https://github.com/idan/oauthlib/tree/master/oauthlib/oauth2/rfc6749/clients
[15:40] <jimbaker> there are still some outstanding failures we need to fix in test_imports, which might account for this specific case
[15:40] <jimbaker> i believe it's a bit more than relative starred imports, one other aspect of how the import is being done. (import support is complex with too many corner cases)
[15:40] <jimbaker> so this is a must fix for 2.7, if that's any consolation
[15:40] * jimbaker (~jbaker@63-253-67-243.ip.mcleodusa.net) Quit (Read error: Connection reset by peer)
[15:40] * enebo (~enebo@c-75-73-8-169.hsd1.mn.comcast.net) Quit (Ping timeout: 245 seconds)
[15:40] * enebo_ is now known as enebo
[15:41] <paolo> jimbaker_: any workaround?
[15:42] <jimbaker_> paolo, well you could try explicit imports. painful, because relative starred imports is the one good use of such starred imports in library code
[15:42] <yecril71pl> The workaround is to use an absolute path, or to use absolute references in the code.
[15:42] <paolo> i'll go for that
[15:43] <paolo> thanks.
[15:43] <jimbaker_> i have done one line-by-line comparison of the c code and the java code for imports to figure out why we are missing those test cases. that was really painful. but i will do it again :)
[15:43] <yecril71pl> using absolute references in the code means the reader will know what you refer to and where it is without extensive IDE support.
[15:44] <yecril71pl> But, of course, you cannot do much if it is not your library.
[15:44] <jimbaker_> one thing i'm looking forward to jython 3.x is that importlib is both pure python and substantial
[15:44] <jimbaker_> yecril71pl, but as a temporary workaround, we can isolate
[15:45] <jimbaker_> *it will be fixed*
[15:45] <agronholm> yecril71pl: you never told me what you wanted lone surrogates for?
[15:45] <jimbaker_> we are seeing the same issues in twisted support
[15:46] <yecril71pl> I never wanted lone surrogates but Ian Hickson wanted to guard against them.
[15:46] <agronholm> who is Ian Hickson?
[15:46] <jimbaker_> agronholm, yecril71pl - one thing we could do is change that pull request against html5lib-python to be capability-based. but that's a one-line change in the PR
[15:46] <jimbaker_> or close
[15:46] <yecril71pl> But we cannot guard against them if we do not acknowledge their existence.
[15:47] <yecril71pl> Ian Hickson is the editor of HTML5.
[15:47] <yecril71pl> jimbaker_: or close what?
[15:47] <agronholm> close to one-line
[15:48] <jimbaker_> close to 1 one line. given that we are testing an exception and we want to be pythonic and not use semicolons
[15:48] <jimbaker_> not certain if we can in this case in any event ;)
[15:49] <agronholm> I think my last commit to PyString.java can stay. The problem there was a genuine programming error. But I need to change the test to test for ValueError in unichr(0xd800)
[15:50] * lheuer (~Adium@unaffiliated/lheuer) has joined #jython
[15:50] <yecril71pl> Will it be possible to sanitise text by converting isolated surrogates to <?>? Text editors to that.
[15:51] <agronholm> sounds like a bad idea
[15:52] <jimbaker_> yecril71pl, hmmm, there is support in codec decoding in general to do this sort of thing
[15:53] <yecril71pl> So it should be enough to decode from UTF-16LE, taken as a sequence of bytes?
[15:56] <jimbaker_> yecril71pl, yes, that should work. i'm thinking of the parallel case, which is to decode using the string_escape codec. i don't see why not, there's a standard mechanism to say use replace, ignore, etc for a given codec, although this requires more work on the dev than say u"\uD800" or whatever
[15:57] <yecril71pl> I see. Not nice but can live with that, I suppose.
[16:02] <jimbaker_> yecril71pl, the codec apis are overly complex. but the simple case is reasonable enough:
[16:02] <jimbaker_> >>> "\uD800".decode("unicode_escape", "replace")
[16:02] <jimbaker_> u'\ufffd'
[16:03] <yecril71pl> Good, thanks.
[16:06] <agronholm> it doesn't make sense to me to allow u'\ufffd' but not unichr(0xfffd) which would result in the same thing
[16:06] <agronholm> besides the interactive shell won't let you enter that -- it expects more input when you press return
[16:06] <agronholm> unlike cpython which is fine
[16:08] <yecril71pl> But who said unichr(0xfffd) should be in error?
[16:09] <agronholm> oh sorry, I assumed it was still in the surrogate section
[16:09] <agronholm> it's not, now that I look
[16:09] <jimbaker_> yeah, just to be clear 0xfffd is a standard unicode character - http://www.fileformat.info/info/unicode/char/0fffd/index.htm
[16:09] <agronholm> I meant this: <jimbaker_> >>> "\uD800".decode("unicode_escape", "replace")
[16:10] <jimbaker_> specifically for replacement usage; python codecs provide ready support for this when the decode would be invalid
[16:10] <agronholm> so... u"\uD800" should raise a ValueError but "\uD800" should pass?
[16:11] <jimbaker_> fwiw, when one writes the following in python: u"some text" it means "some text".decode("unicode_escape", "strict")
[16:11] <agronholm> I'm clearly not understanding what \u means in a bytestring
[16:11] <jimbaker_> actually, i got that wrong
[16:12] <jimbaker_> r"some text".decode("unicode_escape", "strict")
[16:12] <jimbaker_> otherwise the backslash would be interpreted
[16:12] <agronholm> ok that makes more sense
[16:12] <jimbaker_> agronholm, we are just going down some interesting blackholes of complexity
[16:12] <yecril71pl> It is not interpreted in '\uD800'
[16:13] * siel (~siel@unaffiliated/motley) Quit (Ping timeout: 240 seconds)
[16:13] <yecril71pl> It is equivalent to '\\uD8000'
[16:13] <jimbaker_> but if anyone really wants to go down the blackhole, let me suggest reading https://docs.python.org/2/library/codecs.html
[16:13] <yecril71pl> It is equivalent to '\\uD800'
[16:13] <jimbaker_> it's good bedside reading ;)
[16:15] <yecril71pl> So what would be the result of re.compile('\\D8000')?
[16:15] <yecril71pl> So what would be the result of re.compile('[\\D8000]')?
[16:16] <yecril71pl> re.sub('[\\D800]', 'A', '0') == 'A'
[16:17] * jimbaker_ is now known as jimbaker
[16:17] <yecril71pl> Does that means the whole regexp stuff in html5lib is wrong?
[16:17] <yecril71pl> (note that html5lib does not use u'')
[16:21] <jimbaker> yecril71pl, well, i did try to fix that in my PR against html5lib
[16:23] <yecril71pl> re.sub('[\u400]', 'A', 'D0')
[16:23] <yecril71pl> re.sub('[\u400]', 'A', 'D0') == 'DA'
[16:24] <yecril71pl> The whole regexp stuff is broken, not just the surrogates!
[16:25] <yecril71pl> I am shocked. Whizz. Bang. Tadaa.
[16:25] <yecril71pl> The title of your PR is inappropriately narrow.
[16:25] <agronholm> yecril71pl: err...
[16:26] <agronholm> yecril71pl: the first argument is the pattern, right?
[16:26] <yecril71pl> Right.
[16:26] <agronholm> so as we discussed, \u is just \u in a bytestring
[16:27] <agronholm> so essentially you're saying that you want to replace any of \, u, 4 and 0 with A in "D0"
[16:27] <agronholm> which it did
[16:27] * siel (~siel@unaffiliated/motley) has joined #jython
[16:27] <jimbaker> yecril71pl, comments welcome on that PR!
[16:27] <jimbaker> should make it as awesome as possible
[16:28] <yecril71pl> agronholm: if you look at the PR, the original code does exactly what I did.
[16:29] <agronholm> what pr? link plz
[16:29] * xemdetia (xemdetia@nat/ibm/x-rhmamiaytnmouxqa) Quit (Ping timeout: 260 seconds)
[16:29] <yecril71pl> [16:45]
[16:30] <agronholm> in what timezone?
[16:31] <yecril71pl> sorry, [14:45] <jimbaker> https://github.com/html5lib/html5lib-python/pull/150
[16:33] <agronholm> yecril71pl: I must be dumb for not finding that in the PR changes
[16:33] <agronholm> but I can't
[16:33] <jimbaker> i think the only thing we may want to do is change https://github.com/jimbaker/html5lib-python/blob/master/html5lib/utils.py#L24 such that it tests for a ValueError when using unichr(0xD800) - then we can make it good for any platform, not just jython, with this issue
[16:34] <yecril71pl> -invalid_unicode_re = re.compile("[\u0001-\u0008\u000B\u000E-\u001F\u007F-\u009F ???
[16:35] * siel (~siel@unaffiliated/motley) Quit (Ping timeout: 272 seconds)
[16:36] <yecril71pl> If I understand it right, the regexp is just an ASCII string, and so it cannot work.
[16:37] <yecril71pl> But then why does jython complain about invalid Unicode when it loads this file?
[16:37] * yecril71pl is confused again :-(
[16:38] <agronholm> yecril71pl: from __future__ import unicode_literals
[16:38] * yecril71pl is even more confused
[16:39] <yecril71pl> How can Python import from future? Congratulations on that.
[16:39] <agronholm> ?
[16:39] <agronholm> is there confusion about what that import does?
[16:40] <yecril71pl> Sorry, I just do not know what __future__ is.
[16:40] <agronholm> unbelievable
[16:40] <agronholm> that specific import turns on implicit unicode literals
[16:40] <jimbaker> yecril71pl, you raise an interesting point. i think it's due to how strings and unicode are comingled together in 2.x, and it works fine on 3.x
[16:40] <agronholm> so every string literal is a unicode literal unless prefixed with b
[16:40] <jimbaker> specifically for re
[16:41] <jimbaker> and how it compiles to sre bytecodes (sre is a specific regex engine for python which jython ported from cpython)
[16:41] <yecril71pl> Now this looks like a very bad idea, unless your only point is to confuse everyone.
[16:41] <agronholm> what does
[16:41] <yecril71pl> this __future__ thing
[16:41] <agronholm> it lets you enable features from a future python version
[16:42] <agronholm> unicode literals are always on in python 3
[16:42] <agronholm> and cannot be turned off
[16:42] <yecril71pl> And it is a bad idea, because now you do not know what a particular piece of code means without reading it all.
[16:42] <agronholm> huh?!?
[16:42] <agronholm> how would you know what it does anyway without reading it all?
[16:42] <yecril71pl> If you cut a line from such a file, you do not know what the literal means.
[16:43] <agronholm> better not cut it then :)
[16:43] <yecril71pl> But that is how git works: it shows only the nearest neighbourhood.
[16:44] <yecril71pl> And the whole merit of it is to save several u???s. Unbelievable.
[16:45] <yecril71pl> Also, you often copy single expressions to test them on the command line, and the result is different.
[16:46] * jimbaker (~jbaker@63-253-67-243.ip.mcleodusa.net) Quit (Quit: Computer has gone to sleep.)
[16:51] * siel (~siel@unaffiliated/motley) has joined #jython
[16:51] <agronholm> yecril71pl: that's not the whole merit
[16:51] <agronholm> makes it easy to make py3k compatible code
[16:51] <yecril71pl> In that u'' is incompatible?
[16:52] <agronholm> in python 3.2 it is
[16:52] <agronholm> 3.3+ support u'' again
[16:52] <agronholm> but you shouldn't bother with u'' in py3k only code anyway
[16:52] <yecril71pl> (expletives suppressed)
[16:53] <agronholm> I basically write code for python 3 and then backport it to 2.6
[16:53] <yecril71pl> I basically write code for 2.7 and I do not care for 3
[16:53] <agronholm> that's your choice
[16:53] <agronholm> I like having new language features :)
[16:54] <yecril71pl> My employer does not like to deal with unexpected obstacles :-)
[16:54] <agronholm> what is unexpect about python 3?
[16:54] <agronholm> *unexpected
[16:54] <yecril71pl> this breakage of u'' for example
[16:54] <agronholm> use 3.3 or higher and they work fine
[16:54] <agronholm> but if you only write for python 3, you need not bother anyway
[16:55] <yecril71pl> I only write for 2.7, and I do not want that __future__ stuff if I can help that because it is confusing.
[16:55] <agronholm> you're the first person I've heard say that
[16:56] <agronholm> everybody else I've spoken with love them
[16:57] <yecril71pl> Maybe every other Python programmer loves them
[16:57] <yecril71pl> but we have N languages times M employees here
[16:57] <yecril71pl> so it is not that easy
[16:58] <agronholm> you're going to have to switch eventually
[16:58] <yecril71pl> AIUI, we are waiting for Ubuntu LTS.
[16:58] <agronholm> what ubuntu lts?
[16:58] <agronholm> there are plenty of them already
[16:58] <agronholm> trusty ships with python 3.4
[16:59] <yecril71pl> That is, when 12.04 goes out of support.
[16:59] <agronholm> okay then, so it'll be a few years then
[17:01] * jimbaker (~jbaker@129.sub-70-196-194.myvzw.com) has joined #jython
[17:02] * xemdetia (xemdetia@nat/ibm/x-pftgixklnsoqjeby) has joined #jython
[17:02] * Taylor (~Taylor@unaffiliated/taylor) Quit (Excess Flood)
[17:04] * Taylor (~Taylor@unaffiliated/taylor) has joined #jython
[17:21] * jimbaker (~jbaker@129.sub-70-196-194.myvzw.com) Quit (Quit: Computer has gone to sleep.)
[17:23] * lheuer (~Adium@unaffiliated/lheuer) Quit (Quit: Leaving.)
[17:28] * yecril71pl (51d2144a@opensuse/member/yecril71pl) Quit (Ping timeout: 246 seconds)
[17:37] * Oti (~ohumbel@adsl-178-39-145-123.adslplus.ch) has joined #jython
[18:03] * lheuer (~Adium@unaffiliated/lheuer) has joined #jython
[18:19] * jimbaker (~jbaker@ucb-np2-225.colorado.edu) has joined #jython
[18:20] * siel (~siel@unaffiliated/motley) Quit (Ping timeout: 260 seconds)
[18:26] * siel (~siel@unaffiliated/motley) has joined #jython
[18:32] * jimbaker (~jbaker@ucb-np2-225.colorado.edu) Quit (Quit: Computer has gone to sleep.)
[18:34] * siel (~siel@unaffiliated/motley) Quit (Ping timeout: 260 seconds)
[18:37] * siel (~siel@unaffiliated/motley) has joined #jython
[18:43] * siel (~siel@unaffiliated/motley) Quit (Ping timeout: 240 seconds)
[18:45] * lheuer (~Adium@unaffiliated/lheuer) Quit (Quit: Leaving.)
[19:03] * siel (~siel@unaffiliated/motley) has joined #jython
[19:11] * siel (~siel@unaffiliated/motley) Quit (Ping timeout: 260 seconds)
[19:11] * lheuer (~Adium@unaffiliated/lheuer) has joined #jython
[19:31] * siel (~siel@unaffiliated/motley) has joined #jython
[19:37] * siel (~siel@unaffiliated/motley) Quit (Ping timeout: 272 seconds)
[19:53] * siel (~siel@unaffiliated/motley) has joined #jython
[19:53] * siel (~siel@unaffiliated/motley) Quit (Read error: Connection reset by peer)
[20:05] * newbie|2 (~smaudet@206-51-157-254.nktelco.net) has joined #jython
[20:08] * smaudet (~smaudet@206-51-157-254.nktelco.net) Quit (Ping timeout: 256 seconds)
[20:36] * jimbaker (~jbaker@ucb-np2-225.colorado.edu) has joined #jython
[20:41] * jimbaker (~jbaker@ucb-np2-225.colorado.edu) Quit (Quit: Computer has gone to sleep.)
[20:59] * xemdetia (xemdetia@nat/ibm/x-pftgixklnsoqjeby) Quit (Ping timeout: 240 seconds)
[21:00] * jimbaker (~jbaker@ucb-np2-201.colorado.edu) has joined #jython
[21:10] * newbie|2 (~smaudet@206-51-157-254.nktelco.net) Quit (Ping timeout: 256 seconds)
[21:20] * jimbaker (~jbaker@ucb-np2-201.colorado.edu) Quit (Changing host)
[21:20] * jimbaker (~jbaker@python/psf/jimbaker) has joined #jython
[22:42] * lheuer (~Adium@unaffiliated/lheuer) Quit (Quit: Leaving.)
[22:57] * siel (~siel@unaffiliated/motley) has joined #jython
[23:32] * jimbaker (~jbaker@python/psf/jimbaker) Quit (Quit: Computer has gone to sleep.)
These logs were automatically created by JythonLogBot_ on irc.freenode.net using a slightly modified version of the Java IRC LogBot (github).