I have up to now used the tool fdupes to find all the duplicates of a file but this doesn't seem to work. I tried to do :

fdupes -r ~/Large_directory file_to_find.txt

Firstly I have, just after launching this command, the message :

fdupes: could not chdir to file_to_find.txt.

So, as soon as launched, there is already a mistake in my syntax.

Secondly, the command takes a long time (must build an index which is really very slow).

So I am looking for an alternative tool.

Could anyone indicate me another tool which allows to find fastly all the duplicates of a given file in a given directory and of all its subdirectories?

Any suggestion/help is welcome.

  • 1
    fdupes doesn't take a file as an argument. It just finds duplicates, not from a key file. From man fdupes: fdupes [ options ] DIRECTORY .... – schrodigerscatcuriosity Jul 26 at 15:36
  • 1
    Is to file_to_find.txt in the same directory to search? – schrodigerscatcuriosity Jul 26 at 15:51
  • @schrodigerscatcuriosity : Sorry, I didn't know that fdupes doesn't take a file as argument. Is there a similar command that could find all the duplicates of a given file into a given recursive directory ? – youpilat13 Jul 26 at 16:29
  • 1
    I'm a little rusty on bash. This is an idea that comes to mind: paste a copy of the file under the directory you are going to search, then apply this command: fdupes -r -1 ~/Large_directory | grep "file_to_find" | tr -s ' ' '\n'. This would present you with a list of the files that are duplicates in a column. – schrodigerscatcuriosity Jul 26 at 16:57
  • If succesful I can write an answer explaining it. – schrodigerscatcuriosity Jul 26 at 16:57

A loop approach:

b="$(md5sum file_to_find.txt | cut -d' ' -f1)"

shopt -s globstar

for f in ~/Large_directory/**/*.txt; do 
  [[ "$(md5sum "$f" | cut -d' ' -f1)" = "$b" ]] && realpath "$f"
| improve this answer | |
  • 1
    This breaks on filenames with whitespace or globbing characters. $f in md5sum $f and realpath $f should be "$f". The outer quotes, around each enclosing $( ), suppress word splitting and globbing on the substituted output of the commands inside, but don't affect what commands are run. The result of parameter expansion of $f is still subject to splitting and globbing. I'd edit, but I'm unsure of the intent of echo "$(realpath $f)", which is better as echo "$(realpath "$f")", but presumably even better as just realpath "$f". You may want to use shellcheck.net. – Eliah Kagan Jul 27 at 0:17
  • @EliahKagan You are right! Thanks. I've been away from bash these last months, I'll edit with your advices. – schrodigerscatcuriosity Jul 27 at 1:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.