8

My issue is that for some reason I have duplicates of some music tracks. However they are not named identically. For instance:

Music/Prefuse 73/One Word Extinguisher/07. Detchibe.mp3

&

Music/Prefuse 73/One Word Extinguisher/07 - Detchibe.mp3

Notice they are duplicate songs but the 07*.* & the 07 - is tricking duplicate file finders that search based on file names.

3
  • 2
    Have you tried any of the suggestions here?
    – jpd
    Mar 28, 2012 at 18:44
  • Can fdupes single out one of the two duplicates and delete it? Otherwise the fdupes -r command only generates a list which doesn't get me anywhere. Apr 3, 2012 at 0:32
  • Does this answer your question? How can I find duplicate songs?
    – karel
    May 10, 2020 at 23:49

3 Answers 3

2

You can use fdupes like the answer for question »How to find and delete duplicate files« suggested. Let me give an example:

mkdir -p "Music/Prefuse 73/One Word Extinguisher/"
dd if=/dev/urandom of=Music/Prefuse\ 73/One\ Word\ Extinguisher/07.Detchibe.mp3 bs=1023 count=2048
  2048+0 records in
  2048+0 records out
  2095104 bytes (2.1 MB) copied, 0.379806 s, 5.5 MB/s
cp Music/Prefuse\ 73/One\ Word\ Extinguisher/07.Detchibe.mp3 Music/Prefuse\ 73/One\ Word\ Extinguisher/"07 - Detchibe.mp3"
fdupes -rd .
  [1] ./Music/Prefuse 73/One Word Extinguisher/07.Detchibe.mp3
  [2] ./Music/Prefuse 73/One Word Extinguisher/07 - Detchibe.mp3

  Set 1 of 1, preserve files [1 - 2, all]:

First I created the directory like in your example. The I made a file from random data and copied its contents to another files. When I run fdupes -rd the software finds the two exact files and asks which one to delete.

If you have lots of files, you can use the option -1. fdupes will print all duplicates on a single line. You can process them with xargs and other shell features.

4
  • Can the out put from the fdupes -r command be piped to the mv command? Apr 3, 2012 at 22:51
  • fdupes -rd is close to what I want but I'll have to press one about 2000 times for all the dupes to be gone. Apr 4, 2012 at 16:06
  • 1
    I added a paragraph to my answer above.
    – qbi
    Apr 5, 2012 at 9:25
  • The issue here is that fdupes does not seem to dive into sub folders. For instance ~/Desktop/Dupes2/ has a folder in it titled Blood Sugar Sex Magik with the Dupes, yet a simple fdupes -r command doesn't return any list. May 8, 2012 at 2:36
2

I found a somewhat simple command chain. Much thanks to @Oli.

fdupes -rf --quiet ~/Desktop/Dupes2/ | while read i; do mv "$i" ~/Desktop/Dupes/ ; done

This used fdupes to recursively (-r) find the dupes, omitting the first (-f). Bash reads this line by line through read amd hands each line to mv to move all duplicates to another directory. Note the use of quotes in the while loop to handle spaces and other dodgy punctuation that fdupes will not handle (even with -1/--sameline).

2
  • @Oli, this isn't doing the trick. No files are being moved from one folder to the other Apr 4, 2012 at 22:35
  • @Oli see my latest comment to "qbi" below. May 8, 2012 at 2:39
1

In the answers to Manually set track listen count in Banshee?, it describes how to get at the database that banshee uses to save all track information.

Once you're connected to the database, on the execute query table, paste

select tweaked_track, count(*) from 
  (select replace(replace(replace(title, ' ', ''), '-', ''), '.', '') as tweaked_track 
   from coretracks) 
group by tweaked_track 
order by 2, 1 desc;

into the SQL string box, then click 'execute query'. This will show you all the tracks you have with the same title ignoring spaces, dashes, and periods. If there are other characters you want to ignore, add them to the query in the same pattern. (IE add replace( before the first existing "replace" and after the last ")" on that line, add , '[character you want removed]', '').

(I don't know how much you know about sql - if you need more details, post a comment.)

This will give you a list of titles. You will have to actually do the delete yourself.

There may be a better way of doing this, but if there is, I don't know about it.

Once you have a big list of files to be deleted (either from my method or from fdupes like others have mentioned), put the list of files you want to delete into a text directory. Make sure one of the following is true:

Option #1: The filenames contain full path. For example the file might contain:

/home/doneill/music/weird_al/duped_file.mp3
/home/doneill/music/weird_al/another_dupe.mp3
/home/doneill/music/bach/baroque_dupe.mp3

Option #2: The filenames contain relative path, and the file with the list of filenames is saved in the parent folder. For example, if your file list was saved in /home/doneill/music/, it would contain:

weird_al/duped_file.mp3
weird_al/another_dupe.mp3
bach/baroque_dupe.mp3

In either case, open up a terminal window, and change to the folder that contains the file with the list cd /home/doneill/music/ for example.

Type in:

for a in `cat filelist.txt`; do echo $a; done

(Replacing filelist.txt with the name of the file with the list). This should spit out a list of all the files you want to delete. Take a moment to double check the list. If it is right, type:

for a in `cat filelist.txt`; do rm $a; done

This basically tells your computer: for each line in the file filelist.txt, remove a file with the name listed.

7
  • The question asks about duplicate files, no mention of Banshee.
    – jpd
    Mar 28, 2012 at 19:09
  • 2
    @jpd It's tagged banshee... Mar 28, 2012 at 19:12
  • Sorry, completely missed the tags.
    – jpd
    Mar 28, 2012 at 20:04
  • I can't get to the banshee data base because 11.10 doesn't have the option to open with a custom command and the workaround suggested there didn't work - I got thrown down the rabbit hole there for a second but I'm back with no solution. Apr 3, 2012 at 1:07
  • 1
    @JohnMcKeanPruitt 1) Run the sqlite database browser (the command is sqlitebrowser if it isn't put into your menu when you install it). 2) File -> Open. 3) Type /home/[user]/.config/banshee-1/ into the 'Look in' address bar (replace [user] with your username). 4) Double click on banshee.db. Apr 3, 2012 at 3:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .