153

As of today, a bug in OpenSSL has been found affecting versions 1.0.1 through 1.0.1f (inclusive) and 1.0.2-beta.

Since Ubuntu 12.04, we are all vulnerable to this bug. In order to patch this vulnerability, affected users should update to OpenSSL 1.0.1g.

How can every affected user apply this update now?

5
  • Do you have an affected version of openssl?
    – Braiam
    Apr 7, 2014 at 22:56
  • I've got the patched version 1.0.1-4ubuntu5.12 and I've restarted apache service but filippo.io/Heartbleed testing on my site still says I'm vulnerable Any idea why ?
    – user266576
    Apr 8, 2014 at 8:32
  • 1
    @Mat I don't know what that site tests, but maybe it detects that you're using an old key. Since the key may have leaked, you need to regenerate it. Apr 8, 2014 at 10:09
  • 1
    You really don't want to update OpenSSL to wholesale new versions, that's an unbelievable pain. Far easier is to just install the updated package that patches the issue: ubuntu.com/usn/usn-2165-1
    – sarnold
    Apr 8, 2014 at 17:09
  • have you restarted your services after upgrading?
    – Axel
    Apr 14, 2014 at 6:59

6 Answers 6

142

Security updates are available for 12.04, 12.10, 13.10 and 14.04 see Ubuntu Security Notice USN-2165-1.

So first you need to apply the available security updates, for example by running

sudo apt-get update
sudo apt-get upgrade

from the command line.

Do not forget to restart the services (HTTP, SMTP, etc.) that use the affected OpenSSL version, otherwise you are still vulnerable. See also Heartbleed: What is it and what are options to mitigate it? on Serverfault.com.

The following command shows (after an upgrade) all services that need to be restarted:

sudo find /proc -maxdepth 2 -name maps -exec grep -HE '/libssl\.so.* \(deleted\)' {} \; | cut -d/ -f3 | sort -u | xargs --no-run-if-empty ps uwwp

After that, you need to regenerate all server SSL keys, then evaluate whether your keys may have leaked, in which case attackers may have retrieved confidential information from your servers.

19
  • 23
    Not sure this works on Ubuntu 12.04.4 LTS. After full update, openssl version gives OpenSSL 1.0.1 14 Mar 2012. That’s not the patched version, right? Or am I misreading it? Apr 8, 2014 at 6:05
  • 7
    What to do with Ubuntu 13.04 ? No upgraded openssl available :-(
    – Frodik
    Apr 8, 2014 at 6:14
  • 20
    On Ubuntu 12.04 even the fixed OpenSSL displays version 1.0.1 14 Mar 2012. Read crimi's answer to find out whether your installation is including the fix.
    – dan
    Apr 8, 2014 at 7:34
  • 7
    Thanks, @dan! Resummarizing @crimi's answer here: if you run dpkg -l | grep ' openssl ' and you get 1.0.1-4ubuntu5.12 then you’re good to go. Apr 8, 2014 at 8:21
  • 20
    Patching and restarting is not enough. You need to regenerate keys and assess whether your keys have been leaked as well as other confidential material. See e.g. Does Heartbleed mean new certificates for every SSL server? Apr 8, 2014 at 9:34
71

The bug is known as Heartbleed.

Am I vulnerable?

Generally, you're affected if you run some server that you generated an SSL key for at some point. Most end-users are not (directly) affected; at least Firefox and Chrome don't use OpenSSL. SSH is not affected. The distribution of Ubuntu packages isn't affected (it relies on GPG signatures).

You are vulnerable if you run any kind of server that uses OpenSSL versions 1.0–1.0.1f (except of course versions that were patched since the bug was discovered). The affected Ubuntu versions are 11.10 oneiric through 14.04 trusty pre-releases. It's an implementation bug, not a flaw in the protocol, so only programs that use the OpenSSL library are affected. If you have a program linked against the old 0.9.x version of OpenSSL, it isn't affected. Only programs that use the OpenSSL library to implement the SSL protocol are affected; programs that use OpenSSL for other things are not affected.

If you ran a vulnerable server exposed to the Internet, consider it compromised unless your logs show no connection since the announcement on 2014-04-07. (This assumes that the vulnerability wasn't exploited before its announcement.) If your server was only exposed internally, whether you need to change the keys will depend on what other security measures are in place.

What is the impact?

The bug allows any client who can connect to your SSL server to retrieve about 64kB of memory from the server. The client doesn't need to be authenticated in any way. By repeating the attack, the client can dump different parts of the memory in successive attempts.

One of the critical pieces of data that the attacker may be able to retrieve is the server's SSL private key. With this data, the attacker can impersonate your server.

How do I recover on a server?

  1. Take all affected servers offline. As long as they're running, they're potentially leaking critical data.

  2. Upgrade the libssl1.0.0 package, and make sure that all affected servers are restarted.
    You can check if affected processes are still running with ``grep 'libssl.(deleted)' /proc//maps`

  3. Generate new keys. This is necessary because the bug might have allowed an attacker to obtain the old private key. Follow the same procedure you used initially.

    • If you use certificates signed by a certification authority, submit your new public keys to your CA. When you get the new certificate, install it on your server.
    • If you use self-signed certificates, install it on your server.
    • Either way, move the old keys and certificates out of the way (but don't delete them, just ensure they aren't getting used any more).
  4. Now that you have new uncompromised keys, you can bring your server back online.

  5. Revoke the old certificates.

  6. Damage assessment: any data that has been in the memory of a process serving SSL connections may potentially have been leaked. This can include user passwords and other confidential data. You need to evaluate what this data may have been.

    • If you're running a service that allows password authentication, then the passwords of users who connected since a little before the vulnerability was announced should be considered compromised. (A little before, because the password may have remained unused in memory for a while.) Check your logs and change the passwords of any affected user.
    • Also invalidate all session cookies, as they may have been compromised.
    • Client certificates are not compromised.
    • Any data that was exchanged since a little before the vulnerability may have remained in the memory of the server and so may have been leaked to an attacker.
    • If someone has recorded an old SSL connection and retrieved your server's keys, they can now decrypt their transcript. (Unless PFS was ensured — if you don't know, it wasn't.)

How do I recover on a client?

There are only few situations in which client applications are affected. The problem on the server side is that anyone can connect to a server and exploit the bug. In order to exploit a client, three conditions must be met:

  • The client program used a buggy version the OpenSSL library to implement the SSL protocol.
  • The client connected to a malicious server. (So for example, if you connected to an email provider, this isn't a concern.) This had to happen after the server owner became aware of the vulnerability, so presumably after 2014-04-07.
  • The client process had confidential data in memory that wasn't shared with the server. (So if you just ran wget to download a file, there was no data to leak.)

If you did that between 2014-04-07 evening UTC and upgrading your OpenSSL library, consider any data that was in the client process's memory to be compromised.

References

12
  • 4
    I don't believe "Only the server side of SSL/TLS connections is affected" is true. openssl.org/news/secadv_20140407.txt says it can reveal secrets from either client or server. ubuntu.com/usn/usn-2165-1 agrees. The chances that you are using client certificates while connecting to a malicious server are small, but the possibility exists.
    – armb
    Apr 8, 2014 at 12:14
  • @armb You make a good point. It doesn't even matter whether client certificates are used, the data leakage is unrelated to the use of certificates. I've enlisted the help of professionals. Apr 8, 2014 at 12:58
  • Client certificates are the case where you would leak private keys, but yes, passwords, authorization cookies etc. could leak anyway. However, with an OpenSSL based client like curl or wget in typical usage, you wouldn't have secrets for other sites in memory while connecting to a malicious server, so in that case I think the only leakage would be if you gave the client secrets anticipating giving them to a legitimate site, and Heartbleed leaked them during handshake before certificate verification reveals you aren't connected to the right site.
    – armb
    Apr 9, 2014 at 8:41
  • 1
    @Gilles You might be interested in the answers to What clients are proven to be vulnerable to Heartbleed?. I managed to gain "interesting" memory on nginx (proxy mode), wget, links and others.
    – Lekensteyn
    Apr 10, 2014 at 7:50
  • 1
    @MuhamedHuseinbašić The package openssl contains command line tools. It is not used by applications that use the OpenSSL library to implement the SSL protocol (such as Apache). But you should just apply the distribution's security updates. Sep 7, 2017 at 18:01
40

To see which OpenSSL version is installed on Ubuntu run:

dpkg -l | grep openssl

If you see the following version output, patch for CVE-2014-0160 should be included.

ii  openssl      1.0.1-4ubuntu5.12      Secure Socket Layer (SSL)...

Looking at https://launchpad.net/ubuntu/+source/openssl/1.0.1-4ubuntu5.12, it shows which kind of bugs are fixed:

...
 SECURITY UPDATE: memory disclosure in TLS heartbeat extension
    - debian/patches/CVE-2014-0160.patch: use correct lengths in
      ssl/d1_both.c, ssl/t1_lib.c.
    - CVE-2014-0160
 -- Marc Deslauriers <email address hidden>   Mon, 07 Apr 2014 15:45:14 -0400
...
9
  • 2
    I've upgraded and get version 5.12 but this tool still tells me I'm vulnerable filippo.io/Heartbleed Thoughts?
    – toxaq
    Apr 8, 2014 at 11:23
  • 3
    I have tested our updated servers over this side and it told me that I'm not affected. Did you reboot your system, or at least are you sure that all necessary processes have been restarted?
    – crimi
    Apr 8, 2014 at 11:33
  • 3
    After updating the OPENSSL, all I had to do was to restart the apache service, but graceful did not helped. I had to go and restart by using sudo service apache2 restart
    – Tom Hert
    Apr 8, 2014 at 17:14
  • 1
    I just found the cause of my vulnerability: i had mod-spdy-beta installed. After removing it and restarting apache all tests are green now. Apr 9, 2014 at 5:30
  • 3
    Updating openssl doesn't fix applications such as Apache, Nginx or postfix. You have to update libssl1.0.0 and restart them as explained in other posts.
    – tnj
    Apr 10, 2014 at 10:28
17

If your apt-get repositories don't contains any precompiled 1.0.1g OpenSSL version, so just download sources from official website and compile it.

Below the single command line to compiling and install the last openssl version.

curl https://www.openssl.org/source/openssl-1.0.1g.tar.gz | tar xz && cd openssl-1.0.1g && sudo ./config && sudo make && sudo make install

Replace old openssl binary file by the new one via a symlink.

sudo ln -sf /usr/local/ssl/bin/openssl `which openssl`

You are all good !

# openssl version should return
openssl version
OpenSSL 1.0.1g 7 Apr 2014

Cf this blog post.

NB: As stated in the blog post, this workaround will not fix "Nginx and Apache server who have to be recompile with 1.0.1g openSSL sources."

9
  • 2
    As usually Ubuntu doesn't provide the new upstream version but patched the versions for all the supported releases to keep the changes at a minimum. Apr 8, 2014 at 2:55
  • 1
    Note: Make sure you restart your server after updating OpenSSL. Apache and Nginx picked up the new lib and the vulnerability was closed.
    – dAngelov
    Apr 8, 2014 at 17:27
  • 6
    Heh, now that I take the time to read the details of this posting, I'm even more aghast -- downloading a tarball from some random place off the Internet, unpacking, and executing parts of it as root is just reckless behaviour. It'd be slightly better if the tarball signatures were downloaded and checked, but making sure you validate the signatures were signed by the right key is itself a difficult question. Distributions have already gone to the effort to ensuring safe provenance of tarballs and patches. Thanks.
    – sarnold
    Apr 9, 2014 at 0:05
  • 2
    it might be a good idea to compile from source NOW, and install a newer one later on from apt, that way your more secure than without expectly on older versions of ubuntu anyhow just my two cents
    – nwgat
    Apr 10, 2014 at 16:02
  • 2
    @sarnold openssl.org doesn't seem like a random place to download the source for openssl. Canonical should make this unnecessary, but openssl.org should be the authoritative upstream to work from.
    – Rustavore
    Sep 13, 2016 at 15:53
12

For those who do not want to do a serverwide package upgrade. I read a bunch of these guides today and apt-get upgrade openssl === apt-get upgrade this will apply all security fixes required by your machine. Wonderful, unless you are explicitly leaning on an old package version somewhere.

This is the minimal action required on Ubuntu 12.04 LTS running Apache 2:

  • Go to this address and prove you have the vulnerability. You should use the DIRECT EXTERNAL ADDRESS OF YOUR WEB SERVER. If you use a loadbalancer (for example ELB) you might not be contacting your web server directly.

  • Run the following 1 liner to upgrade packages and restart. Yes I saw all the guides saying that you should have a timestamp later than April 4th 2014, this doesn't seem to be the case to me.

    apt-get update && apt-get install openssl libssl1.0.0 && /etc/init.d/apache2 restart

  • Ensure you have appropriate package versions installed and check your webserver for the vulnerability once more.

The key packages are as follows, I determined this information using the command below then edited away the cruft (you don't need to know that much about the state of my machines).

$ dpkg -l | grep ssl

ii  libssl-dev                       1.0.1-4ubuntu5.12          SSL development libraries, header files and documentation
ii  libssl1.0.0                      1.0.1-4ubuntu5.12          SSL shared libraries
ii  openssl                          1.0.1-4ubuntu5.12          Secure Socket Layer (SSL)* binary and related cryptographic tools

1.0.1-4ubuntu5.12 should NOT contain the vulnerability. Ensure this is the case by again going to the website below, and testing your web server.

http://filippo.io/Heartbleed/

5
  • 2
    Using an external site to prove a vulnerability in a server seems to be the wrong approach to me.
    – Rinzwind
    Apr 8, 2014 at 22:00
  • External vulnerability testing scripts are becoming more and more commonplace these days. It does exactly what an internal script does, the connection is just initiated from an external webserver. You can look to sites like WhiteHatSecurity.com for an example of a program that initiates all connections remotely. There are instances where this wouldn't fly, for example network vulnerability testing but for testing a forward facing webserver (which in general an SSL server will be) this is almost ideal.
    – Adrian
    Apr 8, 2014 at 23:03
  • Why install the package if it's being upgraded?
    – Braiam
    Apr 9, 2014 at 0:37
  • 1
    apt-get install openssl libssl1.0.0 did it for me. Running openssl version -a now shows: built on: Mon Apr 7 20:33:29 UTC 2014
    – topher
    Apr 9, 2014 at 7:26
  • "External vulnerability testing scripts are becoming more and more commonplace these days. " that opens the possibility of that external site abusing my system: all they need to know it fails and hack my system before I patch it. No this is not the correct way. (and yes I host my own sites with apache and openssl).
    – Rinzwind
    Apr 10, 2014 at 12:13
11

I noticed many commenters here that need help urgently. They are following the instructions, and upgrading, and rebooting, and still vulnerable when using some of the test websites.

You must check to make sure you don't have packages on hold such as libssl.

:~$ sudo apt-get upgrade -V
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages have been kept back:
  libssl-dev (1.0.1-4ubuntu5.10 => 1.0.1-4ubuntu5.12)
  libssl1.0.0 (1.0.1-4ubuntu5.10 => 1.0.1-4ubuntu5.12)
  linux-image-virtual (3.2.0.31.34 => 3.2.0.60.71)
  linux-virtual (3.2.0.31.34 => 3.2.0.60.71)
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.

To upgrade those apt-mark unhold libssl1.0.0 (for example). Then upgrade: apt-get upgrade -V. Then, restart affected services.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .