Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I keep getting this warning whenever I try to run sudo apt-get update.

W: Duplicate sources.list entry http://archive.ubuntu.com/ubuntu/ precise-updates/main i386 Packages (/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_precise-updates_main_binary-i386_Packages)
W: You may want to run apt-get update to correct these problems

Below is the output from /etc/apt/sources.list file:

deb http://archive.ubuntu.com/ubuntu precise main restricted
deb-src http://archive.ubuntu.com/ubuntu precise main restricted

deb http://archive.ubuntu.com/ubuntu precise-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu precise-updates main restricted

deb http://archive.ubuntu.com/ubuntu precise universe
deb-src http://archive.ubuntu.com/ubuntu precise universe
deb http://archive.ubuntu.com/ubuntu precise-updates universe
deb-src http://archive.ubuntu.com/ubuntu precise-updates universe

deb http://archive.ubuntu.com/ubuntu precise multiverse
deb-src http://archive.ubuntu.com/ubuntu precise multiverse
deb http://archive.ubuntu.com/ubuntu precise-updates multiverse
deb-src http://archive.ubuntu.com/ubuntu precise-updates multiverse

deb http://archive.ubuntu.com/ubuntu precise-security main restricted
deb-src http://archive.ubuntu.com/ubuntu precise-security main restricted
deb http://archive.ubuntu.com/ubuntu precise-security universe
deb-src http://archive.ubuntu.com/ubuntu precise-security universe
deb http://archive.ubuntu.com/ubuntu precise-security multiverse
deb-src http://archive.ubuntu.com/ubuntu precise-security multiverse

How do I fix it?

share|improve this question

23 Answers 23

10.10 and Earlier

Applications > Ubuntu Software Center > Edit > Software Sources > Other Software**

OR

System > Administration > Synaptic Packet Manager > Settings > Repositories**

alt text

Select and Remove duplicate entries.

share|improve this answer

Install ubuntu-tweak and remove the duplicate entries from there,it will be easy

for you..

sudo add-apt-repository ppa:tualatrix/ppa
sudo apt-get update
sudo apt-get install ubuntu-tweak

alt text

share|improve this answer
    
This does not say how to do it. –  RolandiXor Nov 15 at 17:43

I try to complete the possibilities offered from other users using terminal, since that's what you asked, mainly:

login if you're using a text based install or

Press alt+f2 and digit gnome-terminal if you're using GUI

once there digit

sudo nano /etc/apt/sources.list

(To understand better: in the folder /etc you can find (nearly) all configuration files of your computer and in the apt folder you find the souces.list file that contains all the repositories that apt-get uses to update or install and upgrade software)

edit visually the file removing or commenting with # the duplicated rows.

Press control + x to exit and choose y to save the file n to quit without saving.

Done that check your work with an update:

sudo apt-get update

Have fun!

share|improve this answer
    
I also think that evolved text editor like vim can automatically find duplicates but I don't know how to do that any kind of info would be interesting –  Pitto Nov 18 '10 at 17:17

I googled for such a tool, but I didn't find any.. So, I ended up coding one myself with PHP.

Sorry for the dirty unorganized source code..

chkdup - Screenshot:

enter image description here

Author

Mubarak Alrashidi (DeaDSouL)

Requirements:

  1. PHP.
  2. Sudo.

Installation:

  1. Open the terminal.
  2. To install PHP if you don't have it, type: sudo apt-get install php5 php5-cli.
  3. Download the chkdup.
  4. Extract the file anywhere you like.
  5. Type: sudo sh install.sh.

Uninstall:

  1. Open the terminal.
  2. Go to the path you extracted the tool in.
  3. Type: sudo sh uninstall.sh.

How to use it:

Just open the terminal and type sudo chkdup.

FAQs:

What does the install.sh do ?

  • The installer will move the chkdup.php to /opt/ then will create a symbolic link in /usr/bin/.

What about the uninstall.sh ?

  • The uninstaller will remove the following files:

    1. install.sh
    2. /opt/chkdup.php
    3. /usr/bin/chkdup
    4. itself "uninstall.sh"

Is it safe to use chkdup ?

  • Yes it is, since it takes a backup of the current sources.list before attempting to do anything. But I'm not responsible if anything happened. The full responsibility will be yours.

License

GNU General Public License

And goodluck

share|improve this answer
    
Thank you very much for the code. –  Vassilis Mar 17 '11 at 8:27
    
just tried it out, nice and simple :-) nice :-) –  bmbaker Jan 28 '12 at 19:40

It sounds like you have two identical lines in your /etc/apt/sources.list file.

Go to terminal, and enter sudo editor /etc/apt/sources.list, then look for any identical lines in that file, and if you find them, delete them. After your done, save and exit, and then do sudo apt-get update and that should resolve your problem.

If that doesn't solve your problem then you're going to have to use the trail and error approach. First, create a backup of your sources.list file, then in the actual file go through and comment out or delete each line, saving and apt-get update-ing after each line has been commented out or deleted. Going through this file line by line commenting out different lines will eventually tell you which line is the superfluous one. Don't forget to uncomment any lines you commented if apt-get update still shows the same prompt, otherwise you'll be left with no sources in your sources.list file.

share|improve this answer

I had the same problem, open software sources. unchecked "Canonical Partners". The Conflict is between "Canonical Partners" and " Canonical partners Added by software Center"

share|improve this answer

If you couldn't figure it out in your source.list, then do this:

  1. Left-click on settings icon (at the extreme top corner of your screen) and select System Settings.
  2. Click on Software Sources and move to Other Software.
  3. Uncheck one of the 'Canonical Partners(Source Code) -software packaged by Canonical for their partners' files and one of the 'Canonical Partners -software packaged by Canonical for their partners' files also and click Close to leave.
  4. Simultaneously press CTRL+ALT+T keys to open terminal and type sudo apt-get update.
share|improve this answer
    
I followed your suggestions, but still I am getting this error. –  Harbhag Apr 11 '12 at 8:08
    
Please, post the image of your 'Software Sources' here to know the culprit. –  all4naija Apr 11 '12 at 18:19
    
Please, post the image of your 'Other Software' here. –  all4naija Apr 11 '12 at 18:22

Did you check ls /etc/apt/sources.list.d/? Maybe you have some "extra" repositories in there?

share|improve this answer

For Ubuntu 12.04

Graphically:

Press Alt + F2 and paste software-properties-gtk (or you can open "Software Center" then go to "Edit" > "Software Sources"). Go to tab "Other Software", choose the duplicate entry and press the "Remove" button.

enter image description here


If you need a command line option, here it is:

cat /etc/apt/sources.list | perl -ne '$H{$_}++ or print' > /tmp/sources.list && sudo mv /tmp/sources.list /etc/apt/sources.list  

The command do this:

cat reads the file and passes the content to perl which will remove the duplicate lines. The result will be saved > in a temporary file which will then be moved to replace the original /etc/apt/sources.list file.

share|improve this answer
    
Your script worked perfectly. –  Lucio Dec 20 '13 at 21:01
    
Just how to fix it, without confusion and all the too deep details. Thank you! –  Peter Masiar Jul 21 at 18:00
    
It actually worked! Could not, for some reason, install git on a 12.04 either. Instead of remove, I just checked the apparent duplicated "Canonical partners". –  davidkonrad Oct 15 at 12:10

For Ubuntu 9.10 - 12.04

There is an app called Y-PPA-Manager which can do that and much more.

You can install it like that from a Terminal:

sudo add-apt-repository ppa:webupd8team/y-ppa-manager
sudo apt-get upgrade
sudo apt-get install y-ppa-manager

After that, just start the application an go to the Advanced menu.

main window

Then select "Scan and remove duplicate PPAs" and hit OK.

advanced window

share|improve this answer
    
It's cool tool but it doesn't fix problem. I did this choice and after this have the same warnings. –  nazar_art Jul 25 '13 at 10:28

No, there isn't a single command line to find and delete duplicated entries in the /etc/apt/sources.list file.

This is because the suggested command lines, which merely duplicate the function of the much simpler built-in uniq command, will only work on a SORTED file and will only remove ADJACENT lines. Furthermore, those commands will only remove lines consisting of an identical string of characters.

Duplicate entries reported by apt-get update will consist of duplicate function entries, such as a repository being included in both its i386 and amd64 variants. The easy and effective way to remove these is to note which repositories are reported as duplicates by apt-get update and remove them via the Software Center. Open it and choose Edit -> Software Sources -> Other Software tab. Simply look for the duplicate entries and uncheck them. (This is also an opportunity to remove any source-code repositories if you're not compiling the packages).

However, it should be noted that apt-get update doesn't only find duplicate entries via sources.list, but includes repository files located in /var/lib/apt/lists/. Deleting duplicate entries from there will only temporarily remove the apt-get update error messages if Ubuntu thinks it needs them, as the files will be automatically re-installed. The fact that apt-get update will report some repositories as duplicates, and then suggest that that you run apt-get update itself to repair them, is a clue that you really don't need to worry about that error message.

share|improve this answer

In my case the duplicates were in the prerequists-sources.list file. I was able to continue the upgrade by running the following commands in the Terminal:

sudo mv /etc/apt/sources.list.d/prerequists-sources.list /tmp
sudo apt-get update

Then go to the Update Manager and install the updates.

share|improve this answer
    
Moving to /tmp is not necessary. Renaming to something which doesn't end with .list is enough –  Daniel Alder Oct 13 at 20:03
S="/etc/apt/sources.list" ;
S2="$S ${S}.d/*.list" ;
grep -b "^deb\`cat $S2 |
    grep -i "^deb[[:space:]]http" |
    sort |
    uniq -dc |
    sed -e 's;[[:space:]]\\+[[:digit:]]\\+[[:space:]]\\+deb\\(.\\+$\\);\\1;g'\`$" $S2

* Line breaks inserted for readability.

share|improve this answer
2  
Please explain the answer or risk having it removed. –  jokerdino Jul 4 '12 at 4:55
    
Welcome to Ask Ubuntu! As jokerdino as indicated, this answer would benefit substantially, if you were to edit it to add some explanation (or at least to tell the user exactly what to do with these lines). –  Eliah Kagan Jul 4 '12 at 10:34
    
This answer is wrong because it may does remove duplicate entries but it does not remove duplicate source entries. see here for an explanation –  Anwar Shah Sep 6 '12 at 5:26

Make a back up copy of your sources.list:

sudo cp /etc/apt/sources.list{,.backup}

Now remove the original and update:

sudo rm /etc/apt/sources.list && sudo apt-get update
share|improve this answer
    
How can he update without sources.list file? –  Anwar Shah Jul 4 '12 at 5:27
    
It will repopulate. =) –  wojox Jul 4 '12 at 5:28
    
@wojox It works, thanks. –  quantme Dec 16 '12 at 5:26
    
mine did not repopulate –  conman253 Feb 14 at 22:38
    
Revert to the one you backed up –  wojox Feb 15 at 13:42

The partner repository is duplicated inside /etc/apt/sources.list and /etc/apt/sources.list.d/precise-partner.list.

Just remove those "precise-partner" files as the partner repository is already present in sources.list.

sudo rm /etc/apt/sources.list.d/precise-partner.*
share|improve this answer
    
I'm getting rm: cannot remove '/etc/apt/sources.list.d/precise-partner.*': No such file or directory –  Dennis Mar 30 '13 at 23:35
1  
Try sudo rm /etc/apt/sources.list.d/*partner*. Also check your /etc/apt/sources.list, look for duplicate lines. –  Eric Carvalho Mar 31 '13 at 0:51
    
No luck, I don't have any files with "partner" in the name in that directory. And /etc/apt/sources.list doesn't contain any duplicates according to uniq. –  Dennis Jun 23 '13 at 23:03

Your sources.list really has duplicate entries.

First Let's see the correct form of a repository line or source line:

The correct format of repository source line is

<type of repository>  <location>  <dist-name> <components> 

For example, a repo line can be like this one

  deb http://archive.ubuntu.com/ubuntu precise main

Here, it means, the repository is for binary packages, which are hosted in http://archive.ubuntu.com/ubuntu and this repository is for Ubuntu precise (12.04) and this repository contains the main (software which are officially supported by Canonical) component.

  • Type: The type can be deb and deb-src. deb means a binary repository where deb-src means a source repository

  • Location: http://archive.ubuntu.com/ubuntu location of the repository.

  • Dist-name: The distribution name of Ubuntu release. For Ubuntu 12.04 it is precise, for 11.10 it is oneiric.

  • Component: It can be main, universe, multiverse and restricted. These words indicates the level of supports for the packages and the licensing status.

See this page for more information.

Please take note that, you can add one or more component in a line, so "main", "universe", "restricted" and "multiverse" can be in a single line. Also note, Though you add more than one component in a single line, APT system considers them as seperate line containing only one component.

So, If your sources.list have a line like this

deb http://archive.ubuntu.com/ubuntu precise universe

Then it can't have another line like the below (which your files have)

deb http://archive.ubuntu.com/ubuntu precise main universe

which is equivalent of these two lines

deb http://archive.ubuntu.com/ubuntu precise main
deb http://archive.ubuntu.com/ubuntu precise universe

Because, you are duplicating universe twice, so there will be an error for that duplicate. An error is given for each duplicate found and your file has 6 duplicates there.

Solution:

After analyzing your sources.list file, I found that, It is a basic one which is supplied by Ubuntu by default. You can build a default file following the procedure:

  1. Open a terminal and first rename the existing sources.list file to sources.list.bak file. (we can safely remove that, but caution is good).

     sudo mv /etc/apt/sources.list /etc/apt/sources.list.bak
    
  2. Then open software-properties-gtk to build a new one. Type the below command in a terminal or in dash command prompt with pressing Alt+F2

    software-properties-gtk
    

    Note, for Ubuntu version 11.10 or earlier you have to use gksu before the software-properties-gtk command

  3. After the window is open,

    • Select all four catagory in Ubuntu Software tab
    • Select precise-security, precise-updates and precise-backports catagory in Updates tab.
    • Select Canonical partner and Intependent catagory from Other software tab.

That's it. You have now a default sources.list file without error. Compare this file with the previous one, if you want.


Update to deal with sources.list.d dir's files

Sometimes a duplicate entry can be in a file in the /etc/apt/sources.list.d directory, which is also used byt APT. So, you should look at that directory and see if there is any duplicate in those files. It is not necessary for duplicats being in the same file.

Example case:

A user had this error message showing while doing sudo apt-get update.

W: Duplicate sources.list entry http://dl.google.com/linux/chrome/deb/ stable/main amd64 Packages (/var/lib/apt/lists/dl.google.com_linux_chrome_deb_dists_stable_main_binary-amd64_Packages)
W: Duplicate sources.list entry http://dl.google.com/linux/chrome/deb/ stable/main i386 Packages (/var/lib/apt/lists/dl.google.com_linux_chrome_deb_dists_stable_main_binary-i386_Packages)
W: You may want to run apt-get update to correct these problems

But there was no entry with http://dl.google.com in the main /etc/apt/sources.list file. Looking at the /etc/apt/sources.list.d/ directory we found these files:

output of ls /etc/apt/sources.list.d/:

aims-sagemath-precise.list
aims-sagemath-precise.list.save
dropbox.list
dropbox.list.save
fossfreedom-packagefixes-precise.list
fossfreedom-packagefixes-precise.list.save
google-chrome.list
google-chrome.list.save
google.list
google.list.save
sagemath-monolithic-precise.list
sagemath-monolithic-precise.list.save

The files google-chrome.list and google.list looked promising for containing duplicates. So, investing the content of both files were necessary

Output of cat /etc/apt/sources.list.d/google-chrome.list:

deb http://dl.google.com/linux/chrome/deb/ stable main  

and of cat /etc/apt/sources.list.d/google.list

deb http://dl.google.com/linux/chrome/deb/ stable main

So, there was a duplicate entries in the sources of Apt. Since google-chrome.list only contain a single apt line and it was also listed in google.list file we can safely remove this file with the command

sudo rm /etc/apt/sources.list.d/google-chrome.list

Then the problem was solved

share|improve this answer
5  
+1 This is a great answer. Fixed the problem and fully explained what why it had occurred. –  akmad Apr 27 '13 at 9:13
    
What did I do to cause myself to have duplicate entries in my sources.list.d? –  Seanny123 Nov 11 '13 at 9:19
    
@Seanny123 That is difficult to say. –  Anwar Shah Nov 11 '13 at 9:58
    
link to Solve Duplicate source.list entry mentioned here solved my confusion. Try, and if it helps you, praise O.A.K. –  Peter Masiar Jul 21 at 17:57

Just sudo gedit sources.list of the file where the Duplicate is and remove the line or put the # before the line!

share|improve this answer

Use the Ubuntu Sources List Generator.

  • Run sudo nautilus in terminal
  • /etc/apt/sources.list
  • Right click and open the file with a text editor (if you click on it, it will open the software sources GUI)
  • Simply copy and paste into the sources.list and save.
  • Exit terminal and you're done.

Google seems to have a problem, so I'd suggest leaving that out. I have done this on Ubuntu 12.10 (Studio version with Ubuntu desktop added and Unity for another desktop sign in).

share|improve this answer

Well, I solved the Duplicate Entry problem myself by renaming /etc/apt/sources.d (as root) to something else and running sudo apt-get update to re-establish the directory and it's contents.

share|improve this answer

How bad are duplicate entries in sources.list?

I don’t know how bad it is, but i don’t like sudo apt-get update showing me duplicate entries.

By the way its not that bad, its just showing you that you have duplicate entries.

SourcesList

The sources.list file is a key factor in adding or upgrading applications to your Ubuntu installation. This is also used by your system for system updates. The file is basically the roadmap for your system to know where it may download programs for installation or upgrade.

Its just like Windows update

You can remove duplicate entries in few easy steps with Y PPA Manager

 sudo add-apt-repository ppa:webupd8team/y-ppa-manager -y
 sudo apt-get update
 sudo apt-get install y-ppa-manager -y

Open y-ppa-manager form Dash

Enter You Admin Password

enter image description here

Double Click On Advanced

enter image description here

Scan & Remove Duplicate PPA's & Click Ok

enter image description here

It will take some time ( 1 or 2 Mints ) To Scan & Remove Duplicate PPA's

share|improve this answer
1  
In my case, Y PPA kept saying no duplicates found. I made a backup of the source.list removed it, and ran sudo apt-get update to recreate the source list (ubuntu 12.04). This worked for me. –  michel.iamit Jul 5 '13 at 8:27
1  
Worked for me on Mint 17, thanks man! –  spences10 Sep 28 at 11:42

In /etc/apt/sources.list you must comment (#) duplicate repository and save changes. After that repeat apt-get update.

share|improve this answer

Went through /etc/apt/sources.list manually sudo nano /etc/apt/sources.list and removed the duplicates. After apt-get update, the problem was solved.

share|improve this answer

Execute the following to remove the duplicates

cat /etc/apt/sources.list | perl -ne '$H{$_}++ or print' > /tmp/sources.list && sudo mv /tmp/sources.list /etc/apt/sources.list  
share|improve this answer
    
more info on this please –  cipricus Nov 5 '13 at 12:32

protected by Community Jan 7 at 12:19

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.