Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I have an little app I wrote in Python and it used to work... until yesterday, when it suddenly started giving me an error in a HTTPS connection. I don't remember if there was an update, but both Python 2.7.3rc2 and Python 3.2 are failing just the same.

I googled it and found out that this happens when people are behind a proxy, but I'm not (and nothing have changed in my network since the last time it worked). My syster's computer running windows and Python 2.7.2 has no problems (in the same network).

>>> url = 'https://www.mediafire.com/api/user/get_session_token.php'
>>> response = urllib2.urlopen(url).read()
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1215, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1177, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 8] _ssl.c:504: EOF occurred in violation of protocol>

What's wrong? Any help is appreciated.

PS.: Older python versions don't work either, not in my system and not in a live session from USB, but DO work in a Ubuntu 11.10 live session.

share|improve this question
1  
Does it happen for every SSL site you try to contact, or just the one? If it doesn't occur for every site, then could you tell us what site is causing the problem? –  James Henstridge Mar 26 '12 at 4:03
    
Well, I'm not an experienced programmer myself, and I'm trying to read a page from a site's API, and that's the only call that requires SSL, so I don't know if I was doing it right in the first place. I've been using it like a normal urllib.urlopen(url).read() call and it was working. Could you please give me another site's address or a python script that would answer this question? –  Pablo Mar 26 '12 at 5:54
    
Oh, I forgot to mention: the site is Mediafire. It's its get_session_token call that is causing the problem. –  Pablo Mar 26 '12 at 6:12
    
I was able to reproduce this with that site. I've updated your question to include the site in question. I suspect that this is a problem with OpenSSL, since wget fails too. –  James Henstridge Mar 26 '12 at 6:38
    
This happens with stream.twitter.com for me at the time of writing. –  MarkR Mar 20 '13 at 11:30

7 Answers 7

up vote 12 down vote accepted

This appears to be related to the addition of TLS 1.1 and 1.2 support to the version of OpenSSL found in 12.04. The connection failure can be reproduced with the OpenSSL command line tool:

$ openssl s_client -connect www.mediafire.com:443
CONNECTED(00000003)
140491065808544:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 320 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

The connection succeeds if I force the connection to use TLS 1.0 with the -tls1 command line argument.

I would suggest you file a bug report about this problem here:

https://bugs.launchpad.net/ubuntu/+filebug

share|improve this answer
1  
Thank you! I reported a bug. Please, see if you can add any relevant info to it: bugs.launchpad.net/ubuntu/+source/openssl/+bug/965371 –  Pablo Mar 26 '12 at 15:06
    
How does this help him work around the problem in Python? –  Cerin Sep 23 '13 at 20:46
1  
@Cerin: it isolated the problem as an OpenSSL bug rather than something in Python, and directed him to use the bug tracker. That problem has since been fixed. –  James Henstridge Sep 24 '13 at 0:47

For python novices like me, here is the way to override httplib the easiest way. At the top of your python script, include these lines:


import httplib
from httplib import HTTPConnection, HTTPS_PORT
import ssl

class HTTPSConnection(HTTPConnection):
    "This class allows communication via SSL."
    default_port = HTTPS_PORT

    def __init__(self, host, port=None, key_file=None, cert_file=None,
            strict=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
            source_address=None):
        HTTPConnection.__init__(self, host, port, strict, timeout,
                source_address)
        self.key_file = key_file
        self.cert_file = cert_file

    def connect(self):
        "Connect to a host on a given (SSL) port."
        sock = socket.create_connection((self.host, self.port),
                self.timeout, self.source_address)
        if self._tunnel_host:
            self.sock = sock
            self._tunnel()
        # this is the only line we modified from the httplib.py file
        # we added the ssl_version variable
        self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, ssl_version=ssl.PROTOCOL_TLSv1)

#now we override the one in httplib
httplib.HTTPSConnection = HTTPSConnection
# ssl_version corrections are done

From here on, you can use urllib or whatever you use just like you normally would.

Note: This is for python 2.7. For a python 3.x solution, you need to override the HTTPSConnection class found in http.client. I leave that as an exercise for the reader. :-)

share|improve this answer
2  
I really like this solution, it avoids modifying any system libraries or other hackery. –  MarkR Mar 20 '13 at 11:28
1  
Fails using Python 2.7.4 on Ubuntu 12.04: NameError: name 'socket' is not defined. --- You'll need to add "import socket" as well. –  Ben Walther Jul 24 '13 at 20:34
    
Works great on Ubuntu 13.04. Thanks! –  dharmatech Aug 31 '13 at 22:04
    
There is no reason to only patch httplib. People may use other SSL sockets. One could patch ssl instead as in my answer below. –  temoto Sep 16 '13 at 8:36
    
This gives me the error BadStatusLine: '' –  Cerin Sep 23 '13 at 21:08

You can avoid modifying the httplib.py file by modifying your HTTPSConnection object:

import httplib, ssl, socket

conn = httplib.HTTPSConnection(URL.hostname)
sock = socket.create_connection((conn.host, conn.port), conn.timeout, conn.source_address)
conn.sock = ssl.wrap_socket(sock, conn.key_file, conn.cert_file, ssl_version=ssl.PROTOCOL_TLSv1)
conn.request('POST', URL.path + URL.query)

The request method creates a new socket only if connection.sock is not defined. Creating your own one adding the ssl_version parameter will make the request method use it. Then everything else works as usual.

I was having the same issue and this works for me.

Regards

share|improve this answer
    
I'll try that. Thanks! –  Pablo Jan 13 '13 at 19:33

EDIT httplib.py (/usr/lib/pythonX.X/httplib.py on Linux)

FIND HTTPSConnection class declaration

  class HTTPSConnection(HTTPConnection):
....

Inside class code CHANGE line

self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)

TO

self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, ssl_version=ssl.PROTOCOL_TLSv1)

Then httplib HTTPS request should work

import httplib
from urlparse import urlparse
url = XXX
URL = urlparse(url)
connection = httplib.HTTPSConnection(URL.hostname)
connection.request('POST', URL.path + URL.query)
response = connection.getresponse()
share|improve this answer
1  
It really isn;t right to edit a system file like that. Instead, redefine any definitions that need to be changed, by redefining them in your code. –  hexafraction Jan 13 '13 at 0:47

The problem is in ssl, it has nothing to do with HTTP, so why patching httplib if you can patch ssl. The following code should fix all SSL sockets including, but not limited to HTTPS, for Python 2.6+ (built in ssl, did not try with pyopenssl).

import functools
import ssl

old_init = ssl.SSLSocket.__init__

@functools.wraps(old_init)
def ubuntu_openssl_bug_965371(self, *args, **kwargs):
  kwargs['ssl_version'] = ssl.PROTOCOL_TLSv1
  old_init(self, *args, **kwargs)

ssl.SSLSocket.__init__ = ubuntu_openssl_bug_965371
share|improve this answer

functools.partial actually will help us override any keyword argument to a function. We can get a much shorter answer as shown below:

from functools import partial
import ssl

ssl.SSLSocket.__init__ = partial(ssl.SSLSocket.__init__, ssl_version=ssl.PROTOCOL_TLSv1)
share|improve this answer

A simple fix that worked for me was to override SSL's default protocol:

import ssl
ssl.PROTOCOL_SSLv23 = ssl.PROTOCOL_TLSv1
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.