7

I have multiple cloud stacks that are running Ubuntu 14.04.2 and I need to remediate CVEs that I'm exposed to in the zlib library (specifically zlib1g and zlib1g-dev. Eventually I need to migrate these systems to a more recent version of Ubuntu, however until I have resolved blockers to upgrading I need to mitigate existing CVEs.

  • What is the best practices are for upgrading system packages?
  • What should I be worried about breaking / how to test for functional regression?

What I'm currently testing is to add sources from more recent versions of Ubuntu (e.g. artful):

sudo cp /etc/apt/sources.list /etc/apt/sources.list.d/artful.list
sudo vim /etc/apt/sources.list.d/artful.list  # replace "trusty" with "zesty"
sudo apt-get update

Pin all packages to trusty:

$ cat /etc/apt/preferences

Package: *
Pin: release n=trusty
Pin-Priority: 900

Package: *
Pin: release o=Ubuntu
Pin-Priority: -10

Then upgrade specific packages with:

apt-get install --only-upgrade <package> -t zesty

The package that I need to upgrade: zlib1g / zlib1g-dev

Upgrading system packages does not get me a version of zlib1g with the CVE's resolved. I need version >= 1:1.2.8.dfsg-4 closest is probably 1:1.2.11.dfsg-0ubuntu1 from zesty. See:

$ dpkg -s zlib1g | grep Version:
Version: 1:1.2.8.dfsg-1ubuntu1

$ sudo apt-get update && apt-get upgrade

$ dpkg -s zlib1g | grep Version:
Version: 1:1.2.8.dfsg-1ubuntu1

Content of /etc/apt/sources.list:

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

deb http://archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted

## Uncomment the following two lines to add software from the 'universe'
## repository.
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ trusty universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty universe
deb http://archive.ubuntu.com/ubuntu/ trusty-updates universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates universe

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
# deb http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted

deb http://archive.ubuntu.com/ubuntu/ trusty-security main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-security universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security universe
# deb http://archive.ubuntu.com/ubuntu/ trusty-security multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ trusty-security multiverse
Improve this question
9
  • 1
    Just upgrade your system. Having a .2 in 14.04.2 indicates you haven't upgraded your system in a long while! (I don't mean upgrading to a newer version of ubuntu, stay on 14.04 but upgrade the packages; you may also need to look at hwe to pull the next supported LTS kernel)
    – Stéphane Chazelas
    Oct 31, 2017 at 19:02
  • @StéphaneChazelas unfortunately, upgrading system packages doesn't resolve the CVE. The version of zlib1g with the fix isn't availble in trusty or trusty-updates. I need version >= 1:1.2.8.dfsg-4 but probably better to get 1:1.2.11.dfsg-0ubuntu2. I updated the question.
    – Ryan
    Oct 31, 2017 at 19:34
  • 3
    As others have mentioned, 14.04 is still supported, so security issues do get addressed. The zlib CVEs are rather low in severity, which is why they haven’t been fixed; in fact at least two of them are theoretical (relying on undefined behaviours which may cause issues with future compilers). Just because an issue is issued a CVE doesn’t mean you need to go to huge lengths to fix it.
    – Stephen Kitt
    Oct 31, 2017 at 19:57
  • @terdon I'm fine with the question being migrated to Ask Ubuntu, wasn't sure how specific the issue actually was. I also added the content of my sources.list file, I'm using a docker container to test this, but comparing it to my production environment on AWS they seem to be using the same sources.
    – Ryan
    Oct 31, 2017 at 20:34
  • @StephenKitt thanks for the input, if taking the zlib package from a more recent Ubuntu version is going to be high-risk then I will try pushing back on this. However, I don't know how well that will be received. The CVE is being flagged as high severity by AWS Inspector and a security audit by a 3rd party is driving this. The CVEs have been fixed in more recent versions of zlib and upstream changes brought into more recent Ubuntu versions, but they haven't been ported back to trusty it seems.
    – Ryan
    Oct 31, 2017 at 20:37

3 Answers 3

Reset to default
3

Without more specifics of what you're wanting patched, and which pocket of the Ubuntu repositories that software is in for a given release, it's impossible to give you a full answer that is narrow enough.

But, I'll try and give you an overview of how to 'patch' the security issues in Ubuntu packages.

Software packages in Main, and community-contributed patches in Universe, via $RELEASE-security repositories

For security patches submitted in packages by users for Security Team consideration in Universe packages, and security patches by the Security Team itself in Ubuntu Main packages, once released they are available from the $RELEASE-security repositories (for example, xenial-security) and the $RELEASE-updates repositories. This way, you can just simply do a sudo apt-get update && sudo apt-get dist-upgrade and get all the patches.

Tracking the individual CVEs in the CVE Tracker will let you know whether a CVE's had a fix released in Ubuntu yet, as well as the SEcurity team's determination of the priority level of the CVE and how rapidly it needs addressed (default is "Medium" regardless of the CVE severity).

Packages not updated in the repositories

We have two problems here. Firstly, Universe packages only get patches when the community provides them for the Security team to look at and review. Secondly, a large number of packages as a result are not updated.

You have two solutions for these issues: either rebuild the packages yourself with the patches, or wait for an update to land in the Repositories (or for someone to submit the patches for the Security team).

For the first solution, you'll have to follow the Packaging Guide from step 1 through step 3.9, and then steps detailed in section 6 if you want to submit them for the repositories, and build the patched packages locally on your system and install them.

The actual process for this is very very complex depending on the package, so it's impossible to answer it here.

Custom compiled software

Your only hope here is to apply patches to the software yourself, and recompile and install. This is for anything where the software is not included in the repositories, or in the cases you have things installed which you had compiled in the first place. The process for that is infinitely varied, so it's impossible to answer here.

Improve this answer
1

If you have a CVE that you must fix, and it isn't fixed in the official repositories for the release you're using, what you should not do is download and install packages from an arbitrary future release. Such packages may install fine, but there's no guarantee that existing other existing software may be able to interoperate with them. The ABIs or APIs may have changed, maybe significantly, maybe not. Subtle changes maybe enough to throw up hard-to-debug bugs. (if a library doesn't load as expected, a command-line app might throw a file-not-found error, even though the command's file clearly exists!)

What I would suggest is:

  1. Check if a patch is available for the version you're using, either in CVE reports elsewhere, or from upstream.
  2. If it is, then download and modify the source package of the concerned package: How do I get and modify the source code of packages installed through apt-get?
    • Use quilt to apply the patch (see Debian wiki, or this Debian hwoto).
    • I'd suggest that you bump only the least significant part of the version number (the parts after the last -) - certainly do not bump the part before the first :, the epoch number.
  3. Install the package so built.

This is much more likely to maintain compatibility with other components of the OS (in as much as the fix itself doesn't break something), while still allowing you to upgrade if an update hits the repositories of your release. This way, you can also guarantee that the particular CVE you want to fix is fixed as far as you can tell, which may not be the case with the package from some arbitrary future release.

Improve this answer
1
  • 1
    Thank you, I'll have to look into the effort involved but this sounds like the most comfortable solution thus far. Once I patch the version available via apt-get for trusty, I'm assuming I could create my own PPA and host it for distribution to my systems. I'm going to work on this and see how it goes.
    – Ryan
    Nov 2, 2017 at 18:05
1

Here's how I did it for my first startup.

  1. I packaged my application into micro-services with a custom base Docker image. The micro-services were all based on Node.js, so that was the foundation
  2. I deployed those services using Kubernetes / Docker Compose into stage/prod environments
  3. I stored those Docker images on cloud.docker.com which has a nice Docker image scanner that finds relevant CVE's by looking inside the images.

That's how I got to the relevant CVE's. Then

  1. I read the CVE's to see which one's applied. These 4 CVE's look like:
    • inftrees.c in zlib 1.2.8 might allow context-dependent attackers to have unspecified impact by leveraging improper pointer arithmetic.
    • inffast.c in zlib 1.2.8 might allow context-dependent attackers to have unspecified impact by leveraging improper pointer arithmetic.
    • The inflateMark function in inflate.c in zlib 1.2.8 might allow context-dependent attackers to have unspecified impact via vectors involving left shifts of negative integers.
    • The crc32_big function in crc32.c in zlib 1.2.8 might allow context-dependent attackers to have unspecified impact via vectors involving big-endian CRC calculation.

To me "unspecified" and "context-dependent" mean this is a pretty theoretical attack. This would mean bad-guys with alot of money wanted to break into you - as opposed to regular bad-guys looking to break someone. Only you know what's the best place to put your resources.

For my case, there were Chrome CVE's (Chrome is the base for Node.js) that didn't apply to my use cases, so I ignored them waiting for upstream fixes. Sometimes there were things that needed a fix right away, so:

  1. I updated the Docker images. Because all the micro-services started with a custom base Docker image, I could push out Node and OS updates relatively quickly.
  2. Deploy to stage, smoke test on stage, and deploy to production.

I upvoted some other answers here - hopefully you'll get what you're looking for.

Improve this answer
2
  • 1
    Thank you, it's helpful to get some context around triaging CVEs. It seems quite subjective which makes it difficult.
    – Ryan
    Nov 2, 2017 at 18:28
  • @RyanFisher glad I could help. Yeah, it's definitely subjective - and very much a business cost/risk calculation.
    – Michael Cole
    Nov 3, 2017 at 22:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .