ndg package not being installed

[variable]

So I had a very long day debugging why I was getting
EOF occurred in violation of protocol (_ssl.c:590)

It turns out that init.py tried to monkey patch the ssl_wrap_socket

    from .packages.urllib3.contrib import pyopenssl
    pyopenssl.inject_into_urllib3()

But it couldn't, since it couldn't import from .packages.urllib3.contrib import pyopenssl

Which due to packages/urllib3/contrib/pyopenssl.py tried to import:

from ndg.httpsclient.ssl_peer_verification import SUBJ_ALT_NAME_SUPPORT

but ndg didn't exist.

Once I installed ndg-httpsclient then the error went away.

What I am wondering is why package ndg-httpsclient was not installed during installing requests? Or is it supposed to be installed individually?

[Lukasa]

The reason we don't install it when you install requests is that it's optional. Requests can, on many systems, successfully handle TLS without needing any third-party libraries. However, some users will want to use a newer version of OpenSSL or enable Requests to use more advanced features. Those require some additional third-party dependencies that are strictly optional. They can be requested when installing Requests by running pip install requests[security].

[IntelBob]

pip install requests[security] works great on Mac OS X Sierra but on Ubuntu 14.04, it ran fine but didn't seem to have any effect, any idea why?

    >>> ssl.OPENSSL_VERSION
    'OpenSSL 1.0.1f 6 Jan 2014'
    $ openssl version
    OpenSSL 1.0.1f 6 Jan 2014

BTW I am using pyenv and pyenv-virtualenvwrapperif that makes a difference.

[Lukasa]

@IntelBob The security packages don't install a newer OpenSSL on Linux and, even if they did, they don't monkeypatch the standard library. Are you having a specific problem on Ubuntu?

[IntelBob]

@Lukasa I am getting intermittent EOF occurred in violation of protocol (_ssl.c:590) error on Ubuntu, should I just upgrade Ubuntu openssl package? Looks like the latest openssl version on Ubuntu is 1.0.1g, would installing that version help with my issue? I am not explicitly using ssl in my code, just using requests to access https URLs.

[Lukasa]

You can try, but intermittent errors rather suggest that the problem isn't with OpenSSL. Are the errors intermittent in the sense that they only occur for some hosts, or intermittent in the sense that sometimes a request works for a host and sometimes it fails?

[IntelBob]

It's the latter, intermittent in the sense that it works almost all the time and occasionally it will generate that error, it's not reliably reproducible so I don't know how to debug it, I am on Python 2.7.12. Also I am using requests with threads and unsure if such an error might have caused a thread to hang (I am using try...exception block), that's the core of my issue is that I get these random hanging threads and no discernible cause.

Also sometimes i also get this error [('Connection aborted.', BadStatusLine("''",))]. I am making these requests through a proxy server and that has also thrown intermittent errors. I read from another issue that using requests.session might help, right now I am using requests.get, requests.delete in all the threads. Any insights will be much appreciated.

[Lukasa]

So both of these intermittent issues smell like issues that have nothing to do with your OpenSSL version and everything to do with various kinds of network weirdness. Both of these errors basically boil down to "the remote peer shut the connection down when we didn't expect them to". Working out a more specific understanding of why the remote peer is doing that is much harder if you don't have access to server logs.

[IntelBob]

Yeah, I kinda had the same feeling since the errors are intermittent and random and unfortunately I don't have access to the backend logs. As a side note, do you know if using requests.get(), requests.delete() in multiple threads is safe? I assume so since requests is thread-safe and i have a try...except block that the thread should end normally even in the case of such an exception.

[Lukasa]

Yes, requests.* methods share no state so you can safely use them across threads.

[szyl111]

Hi @Lukasa

I the same error , and it stuck me for couple of days 👍
urllib2.URLError: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)>
my python version is 2.7.10, and I use urllib2
below is my related code:
cookieprocessor = urllib2.HTTPCookieProcessor() opener = urllib2.build_opener(cookieprocessor) urllib2.install_opener(opener) request = urllib2.Request(url,postParams) if sys.version_info < (2, 7, 9): file = urllib2.urlopen(request) else: ctx = ssl.create_default_context() ctx.check_hostname = False ctx.verify_mode = ssl.CERT_NONE print ("---") file = urllib2.urlopen(request, context=ctx, timeout=30) fileInfo = file.read()

Please help me...

[Lukasa]

@szyl111 Your code is using urllib2. This issue tracker is for support with the Python Requests library only. You'll need to go discuss your problem upstream with the Python Development team.