102

I just installed Ubuntu Gnome 16.04. I told it to save my documents - which worked. Some software had to be reinstalled. Now when I run sudo apt-get update I see this:

W: Target Packages (main/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (main/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (main/binary-all/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (main/i18n/Translation-en_GB) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (main/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11 (main/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11-icons (main/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (restricted/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (restricted/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (restricted/binary-all/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (restricted/i18n/Translation-en_GB) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (restricted/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11 (restricted/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11-icons (restricted/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (multiverse/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (multiverse/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (multiverse/binary-all/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (multiverse/i18n/Translation-en_GB) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (multiverse/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11 (multiverse/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11-icons (multiverse/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: The repository 'http://extras.ubuntu.com/ubuntu xenial Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: http://dl.google.com/linux/chrome/deb/dists/stable/Release.gpg: Signature by key 4CCA1EAF950CEE4AB83976DCA040830F7FAC5991 uses weak digest algorithm (SHA1)
W: http://repo.sinew.in/dists/stable/InRelease: Signature by key B6DA722E2E65721AF54B93966F7565879798C2FC uses weak digest algorithm (SHA1)
E: Failed to fetch http://extras.ubuntu.com/ubuntu/dists/xenial/main/source/Sources  404  Not Found [IP: 91.189.92.152 80]
E: Failed to fetch http://extras.ubuntu.com/ubuntu/dists/xenial/main/binary-amd64/Packages  404  Not Found [IP: 91.189.92.152 80]
E: Failed to fetch http://extras.ubuntu.com/ubuntu/dists/xenial/main/binary-i386/Packages  404  Not Found [IP: 91.189.92.152 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
W: Target Packages (main/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (main/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (main/binary-all/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (main/i18n/Translation-en_GB) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (main/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11 (main/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11-icons (main/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (restricted/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (restricted/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (restricted/binary-all/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (restricted/i18n/Translation-en_GB) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (restricted/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11 (restricted/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11-icons (restricted/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (multiverse/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (multiverse/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Packages (multiverse/binary-all/Packages) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (multiverse/i18n/Translation-en_GB) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target Translations (multiverse/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11 (multiverse/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87
W: Target DEP-11-icons (multiverse/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list:33 and /etc/apt/sources.list:87

Is there an automated way to fix this?

142

Recent, up-to-date solution

I wrote a Python script to automate this task. You can find the most recent version here.

Installation:

  1. Install the prerequisites:

    sudo apt install python3-apt
    
  2. Download the ZIP bundle (aptsources-cleanup.zip) from the latest release.

Usage:

  1. sudo python3 -OEs aptsources-cleanup.zip

  2. Follow the instructions appearing on the screen.


Historical, outdated solution

This is an older, shorter and simpler version more suitable to inline quotation:

#!/usr/bin/env python3
"""
Detects and interactively deactivates duplicate Apt source entries.

Usage: sudo python3 apt-remove-duplicate-source-entries.py
"""

from __future__ import print_function
import aptsources.sourceslist

EMPTY_COMPONENT_LIST = (None,)

def get_duplicates(sourceslist):
    """
    Detects and returns duplicate Apt source entries.
    """

    sentry_map = dict()
    duplicates = list()
    for se in sourceslist.list:
        if not se.invalid and not se.disabled:
            for c in (se.comps or EMPTY_COMPONENT_LIST):
                key = (se.type, se.uri, se.dist, c)
                previous_se = sentry_map.setdefault(key, se)
                if previous_se is not se:
                    duplicates.append((se, previous_se))
                    break

    return duplicates


if __name__ == '__main__':
    try:
        input = raw_input
    except NameError:
        pass

    sourceslist = aptsources.sourceslist.SourcesList(False)
    duplicates = get_duplicates(sourceslist)

    if duplicates:
        for dupe, orig in duplicates:
            print(
                'Overlapping source entries:\n'
                '  1. {0}: {1}\n'
                '  2. {2}: {3}\n'
                'I disabled the latter entry.'.format(
                    orig.file, orig, dupe.file, dupe),
                end='\n\n')
            dupe.disabled = True

        print('\n{0} source entries were disabled:'.format(len(duplicates)),
            *[dupe for dupe, orig in duplicates], sep='\n  ', end='\n\n')
        if input('Do you want to save these changes? (y/N) ').upper() == 'Y':
            sourceslist.save()

    else:
        print('No duplicated entries were found.')

It takes no parameter and works on all /etc/apt/sources.list* with the help of the python3-apt package. Lines don't have to be exact duplicates (even after normalization) but its enough if type, URI, suite and at least one component overlap.

I'm not sure the equality test is correct in all corner cases (e. g. various “option” strings). Please leave your comments and forks on GitHub.

61

For me (16.04 LTS) going to

System settings >> Software and Updates >> Other Software and removing each duplicate entry from the list worked fine.

  • 4
    I was requesting an automated method. – Tim Nov 6 '16 at 10:58
  • 11
    this is a better solution than read foreign python3 script before executing it... worked perfectly – Max Aug 28 '17 at 12:06
  • This is a good answer! Worked perfectly for me too – haksayng Nov 6 '17 at 17:02
  • 1
    Can you delete "duplicates" which have a different type? For example I have http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial main and http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial main (source code). Is it save to remove the (source code) one (this one isn't checked either, as opposed to the other one)? – Deruijter Mar 1 '18 at 10:02
  • @Deruijter: Could you please open a new question if you have a new or follow-up question? The comment section is not suitable or meant for new questions or extended discussion. You’re welcome to send me a comment with a notification to draw my attention to it. Thanks. – David Foerster Jun 13 '18 at 18:07
27

Reset your repositories like this

  1. sudo rm /etc/apt/sources.list
  2. sudo software-properties-gtk
  3. Pick your options
  4. Save

(P.S. I had the same problem today after updating Ubuntu 15.10 to Ubuntu 16.04.)

1

Since the errors tell you where the problem is, and that all the problems are due to a single duplicated line, you can easily fix this manually with a text editor like vim, nano, etc. One benefit of this is that you can see exactly what the problem is and what you're doing to fix it.

In your case, with sudo or as root, open /etc/apt/sources.list in a text editor. Delete either line 33 or line 87. You can visually check that they're duplicates first.

If you like vim...

$ sudo vim /etc/apt/sources.list
33 [Shift]+g    # to goto line 33
87 [Shift]+g    # to goto line 87, don't need a duplicate of line 33
dd              # to delete the line you're on
:wq             # to write the file and quit

Or, to get out if you make a mistake...

:q!             # to quit without saving any changes
1

For Ubuntu-based systems with Linux mint, go to Software Sources > Maintenance > Remove Duplicate packages

0

I encountered this for 18.10. I used Synaptic to check the repositories and then deleted the ones I thought were duplicates. Result: no more error messages.

protected by Thomas Ward Dec 1 '16 at 23:31

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.