1

I'm trying to accomplish a simple task using the excellent post here: I would like to remove all files from a folder except those listed in a .txt file contained in the folder. The syntax works fine, but for some reason all files are removed - basically the script is saying that there are no correspondences between the filenames in the folder and those in the .txt file - but I can see that there are!

I thought maybe there might be some invisible white space after my file names that I cannot see (in the folder or the .txt file). But I don't think so...Is there another solution that I am not thinking of?

Improve this question
10
  • 2
    Try looking with the od -c command. You might find things like \r (return) are hidden just before the \n(newline). – ubfan1 Oct 25 '19 at 22:09
  • You could also try this frome within the directory ** replace exclude.txt with your list file ** ls -p | grep -v / | sed 's/\<exclude.txt\>//g' | sort | comm -3 - <(sort exclude.txt) | xargs this will only output without removing. To remove run ls -p | grep -v / | sed 's/\<exclude.txt\>//g' | sort | comm -3 - <(sort exclude.txt) | xargs rm – Raffa Oct 25 '19 at 23:41
  • @ubfan1 - there are indeed \n and \r at the end of each line. I removed the \r but don't I need to keep the \n to differentiate between each filename entry in the txt file? – MeC Oct 25 '19 at 23:44
  • is there a syntax error in that @raffa ? – MeC Oct 25 '19 at 23:51
  • ls -p | grep -v / | sed 's/\<a.txt\>//g' | sort | comm -3 - <(sort a.txt) | xargs No it is just a one line command that is easier and will do the job. I replaced exclude.txt with a.txt assuming this is the name of your list. If you run this command, it will only show you the output. If that is what you want, append rm after adding one space at the end to remove the files. – Raffa Oct 25 '19 at 23:55
2

It seems that your a.txt file contains hidden characters due to possibly being created or edited on other OS's like Windows.

You need to sanitize the file first in order to use it without syntax errors.

The easiest way is to use a tool called dos2unix. Please follow these steps to do so:

Firstly: Install dos2unix by opening a terminal and running the following command:

sudo apt install dos2unix

Secondly: Back up your a.txt file.

Thirdly: Use dos2unix on your file by running the following command in the terminal from within the directory containing your a.txt file:

dos2unix a.txt

Finally: Test your file. It should be fine now.

Improve this answer
0

The script you are referring to will give false positives. for will expand filenames literally, this means special characters will be preserved and using it as a pattern will break the matching.

Example:

#!/bin/bash

# the script is adapted and slightly shortened.
shopt -s nullglob
for i in a* b*; do
   grep -Fx -e "$i" exclude.txt
done

Output:

$ ls
a   'a'$'\n''b'   b   exclude.txt

$ cat exclude.txt
a
b

$ script.sh
a
a
b
b

A better approach would be to preserve the shell-escape. 

#!/bin/bash

shopt -s extglob

eval "list=( \
    $(ls --quoting-style=shell-escape !(exclude.txt) | \
        grep -vFx -f exclude.txt))"

for i in "${list[@]}"; do ls "$i"; done

Example:

$ cat exclude.txt  
'a'$'\n''b'

$ script.sh
a
b

$ cat exclude.txt  
a
b

$ script.sh
'a'$'\n''b'
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.