I'm using this bash script to check if the content of two paths install and install1 passed as arguments is the same:

du -a $1 | grep -Po "[0-9]+\t$1/\K.*" | sort -n > list.txt
du -a $2 | grep -Po "[0-9]+\t$2/\K.*" | sort -n > list1.txt
cmp list.txt list1.txt

I'm running the script this way:

./compare-folders install install1

And the comparison between the file lists is failing:

list.txt list1.txt differ: byte 3557704, line 37504

Which should mean that at least one file present in one folder is missing in the other folder.

I checked the differences between list.txt and list1.txt using diffuse, and both files are missing some lines compared to the other. The first line missing in list.txt but present in list1.txt (i.e. the first file apparently not present in install but present in install1) is:


But the file is actually present in both folders. Same for other two files which I bothered to check.

What's strange about this is that the choice of lines to drop is apparently random, while running the script multiple times always produce the exact same result.

I'm classically missing something idiotic. I'm trying to sort out what it is, but as of now no luck.

  • Can you create a reproducible example that we can try? Create 2 small directory hierarchies (i.e. less than 37K files) – glenn jackman Mar 24 '15 at 20:30
  • can you use diff instead to compare the files? – j0h Mar 30 '15 at 19:03
  • @j0h It's the same, the problem was that the directories lists weren't sorted properly – kos Mar 31 '15 at 12:59
du -a $1 | grep -Po "[0-9]+\t$1/\K.*" | sort -n > list.txt

You are removing the file size with your grep command, then you are sorting the result numerically. That doesn't sound right.

What about trying

du -a "$1" | sort -n | grep -Po "[0-9]+\t\Q$1\E/\K.*" > list.txt

I added \Q and \E around the variable to treat it as a literal string, even if it contains regex-special characters

  • If i understand you think that sorting the output numerically doesn't take into account every character, and that in case of particular characters it falls-back by sorting in the processing order, right? Let me check this and I'll report back, I'll try to enclose the string as well if that doesn't work – kos Mar 24 '15 at 20:48
  • Actually I'd better enclose the string as well, it can't hurt. Thank you, the fall-back is actually the problem. Got a word on how I could get rid of it without sorting multiple times? Files of the same size with a leading similiar name and containing the same unsortable character would be still sorted fuzzily – kos Mar 24 '15 at 21:14
  • Maybe you just want sort after grep, not sort -n – glenn jackman Mar 24 '15 at 21:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.