35

Something I feel I ought to know for sure: if I ls <something>, will rm <something> remove exactly the same files that ls displayed? Are there any circumstances where rm could remove files that ls did not show? (This is in the 18.04 bash)

Edit: thank you to everyone who answered. I think the full answer is a combination of all the answers, so I have accepted the most up-voted answer as "the answer".

Unexpected things I have learned along the way:

  • ls is not as straightforward as you might think in its handling of its arguments
  • In a simple un-fiddled-with installation of Ubuntu, .bashrc aliases ls
  • Don't name your files beginning with a dash as they can look like command arguments, and naming one -r is asking for it!
15
  • 8
    I am somewhat surprised that rm does not have a --dry-run flag...
    – fkraiem
    Sep 5, 2018 at 10:19
  • 20
    @Rinzwind Why would find -delete be better than rm? You say "That is why", but it's completely unclear to me what that refers to. Also note that your find invocation will delete all files recursively in the current directory, where rm will just delete the files in the immediate directory. Also -name * is a no-op. All in all, I'm quite puzzled by your advice...
    – marcelm
    Sep 5, 2018 at 10:53
  • 3
    @marcelm I think the advice for using find is because you can run it, see all the files, and then run the same command with -delete. Since you already saw the results from find, there should be no ambiguity to what will be removed (I'd actually like to hear more details about this in the form of an answer) Sep 5, 2018 at 12:24
  • 5
    @Scribblemacher "... you can run it, see all the files, and then run the same command with -delete" - But how is that better than running ls <filespec>, followed by rm <filespec> (which the OP already knows how to do)?
    – marcelm
    Sep 5, 2018 at 12:32
  • 12
    @Rinzwind "That solves choroba's answer for instant where a file is created after ls and before rm." - No, it doesn't. If you run find ... -print first to confirm what files would be deleted, and then find ... -delete, you'll still delete files created between the two commands. If you use both -print and -delete, you don't get confirmation, just an after-the-fact report of what has been deleted (and you might as well use rm -v).
    – marcelm
    Sep 5, 2018 at 12:34

10 Answers 10

41

Well, both ls and rm operate on the arguments which are passed to them.

These arguments can be a simple file, so ls file.ext and rm file.ext operate on the same file and the outcome is clear (list the file / delete the file).

If instead argument is a directory, ls directory lists the content of the directory while rm directory won't work as is (i.e. rm without flags cannot remove directories, while if you do rm -r directory, it recursively deletes all files under directory and the directory itself).

But keep in mind that command line arguments can be subjected to shell expansion, so it's not always guaranteed that the same arguments are passed to both commands if they contain wildcards, variables, output from other commands, etc.

As an extreme example think ls $(rand).txt and rm $(rand).txt, the arguments are "the same" but the results are quite different!

3
  • 3
    Also consider ls vs rm * where there are "hidden" (dot) files, though even that's not at all a fair comparison as I didn't write ls *. But rm is meaningless on its own so the entire thing is apples and oranges really. If I've understood correctly, that's the crux of your answer, so good job :) Sep 5, 2018 at 13:19
  • 5
    Another comment on ls vs rm -r: the command ls <directory> will not show hidden files inside the directory, but rm -r <directory> will delete even the hidden files. Sep 6, 2018 at 12:46
  • 1
    @LightnessRacesinOrbit ls won't list them (unless it's aliased to ls -a), and rm * won't delete them (unless you have dotglob set).
    – OrangeDog
    Sep 7, 2018 at 9:30
21

If you're thinking of something like ls foo*.txt vs. rm foo*.txt, then yes, they will show and remove the same files. The shell expands the glob, and passes it to the command in question, and the commands work on the listed files. One listing them, one removing them.

The obvious difference is that if any of those files happened to be a directory, then ls would list its contents, but rm would fail to remove it. That's usually not a problem, since rm would remove less than what was shown by ls.

The big issue here comes from running ls * or rm * in a directory containing filenames starting with a dash. They would expand to the command lines of the two programs as if you wrote them out yourself, and ls would take -r to mean "reverse sort order", while rm would take -r to mean a recursive removal. The difference matters if you have subdirectories at least two levels deep. (ls * will show the contents of the first level directories, but rm -r * will everything past the first sublevel, too.)

To avoid that, write permissive globs with a leading ./ to indicate the current directory, and/or put a -- to signal the end of option processing before the glob (i.e. rm ./* or rm -- *).

With a glob like *.txt, that's actually not an issue since the dot is an invalid option character, and will cause an error (until someone expands the utilities to invent a meaning for it), but it's still safer to put the ./ there anyway.


Of course you could also get different results for the two commands if you changed the shell's globbing options, or created/moved/removed files in between the commands, but I doubt you meant any of those cases. (Dealing with new/moved files would be extremely messy to do safely.)

3
  • 1
    Thank you. This seems to highlight an issue with the whole command line syntax: if you name files starting with a dash you are sailing in dangerous waters. Who knows what commands you might use in the future, long after you've forgotten about the - files.
    – B.Tanner
    Sep 6, 2018 at 6:01
  • 1
    @B.Tanner, yes. In a sense, the problem is that the command line arguments are just plain strings. If the system was designed today, there might be more structure in it so that the executed program could know if an argument was supposed to be an option flag or not. There are also other problems that come from the very lax structure of filenames. There's a very thorough essay on that by dwheeler, but I have to warn that reading it is going to hurt. (Either from the detail, or from the utter awfulness of what can go wrong.)
    – ilkkachu
    Sep 6, 2018 at 8:03
  • 3
    I can't resist, you've sold it to me, I'm off to read it now, with memories of the time I managed to get a carriage return character at the end of lots of filenames (trying to see if I could construct a Windows batch file that can also run as a bash script...I didn't see that one coming!)
    – B.Tanner
    Sep 6, 2018 at 18:47
18

Leaving aside shell behavior,let's focus on only what rm and ls can deal with themselves. At least one case where ls will show what rm can't remove involves directory permissions, and the other - special directories . and ...

Folder permissions

rm is an operation on a directory, because by removing a file, you're changing directory contents ( or in other words listing of directory entries,since directory is nothing more than a list of filenames and inodes). This means you need write permissions on a directory. Even if you are the owner of the file, without directory permissions you can't remove files. The reverse is also true: rm can remove files that may be owned by others, if you are directory owner.

So you may very well have read and execute permissions on a directory, which will allow you traverse the directory and view contents within just fine, for example ls /bin/echo, but you can't rm /bin/echo unless you are the owner of /bin or elevate your privileges with sudo.

And you'll see cases like this everywhere. Here's one such case: https://superuser.com/a/331124/418028


Special directories '.' and '..'

Another special case is . and .. directories. If you do ls . or ls .. , it will happily show you the contents, but rm'ing them is not allowed:

$ rm -rf .
rm: refusing to remove '.' or '..' directory: skipping '.'
15

If you type ls * and then rm *, it's possible you'll remove more files than ls showed - they might have been created in the tiny time interval between the end of ls and start of rm.

9
  • 1
    That's a probable case for /tmp, where many applications can create temporary files, so that's always a possibility in both commands with *. However, some applications also make files anonymous by unlink() ing them while keeping file handle open, so it may show in ls * but rm * may not catch it. Sep 5, 2018 at 9:52
  • 1
    @OrangeDog You're missing the point. We're talking about race condition Sep 7, 2018 at 15:51
  • 1
    @OrangeDog I'll need to verify this since I'm on the phone right now, but the point still stands even with * there's difference between what ls would show and what rm operates on, because listing of directory contents has changed in between. Sep 7, 2018 at 16:09
  • 1
    Even if you only do one command, there’s a race condition between expanding the arguments and then deleting them.
    – OrangeDog
    Sep 7, 2018 at 16:22
  • 1
    @OrangeDog I can agree with that. As wildcard expansion works on existing filenames, yes, there's a race condition between shell expanding the wildcard, and a command processing it, so ls * could in fact show filename that already is gone. Sep 7, 2018 at 16:59
9

ls * and rm * are not responsible for expanding the glob - that's done by the shell before passing it to the command.

This means that you can use any command with the expanded filelist - so I would use something that does as little as possible.

So a better way to do this (or at the least, another way) is to skip the middle-man.

echo * will show you exactly what would be passed to your rm command.

2
  • 2
    Thank you, I like the idea of using echo instead of ls in this scenario.
    – B.Tanner
    Sep 6, 2018 at 6:09
  • 3
    Or printf "%s\n" * to get an unambiguous view on filenames with spaces. (Or %q instead to deal with newlines and control characters too, at the expense of uglier output.)
    – ilkkachu
    Sep 6, 2018 at 8:10
4

How about:

$ mkdir what
$ cd what
$ mkdir -p huh/uhm ./-r
$ ls *
uhm
$ rm *
$ ls
-r
$ ls -R
.:
-r

./-r:

Basically wildcards expanding to stuff starting with - (or manually entered stuff starting with - but that looks a bit more like cheating) may be interpreted differently by ls and rm.

4

There are edge cases where what ls shows is not what rm removes. A rather extreme, but fortunately benign one is if the argument you pass is a symbolic link to a directory: ls will show you all the files in the symlinked directory, while rm will remove the symlink, leaving the original directory and its contents untouched:

% ln -s $HOME some_link
% ls some_link    # Will display directory contents  
bin    lib    Desktop ...
% rm some_link
% ls $HOME
bin    lib    Desktop ...
5
  • 1
    Huh. ln -s $HOME some_link; ls some_link outputs some_link@ for me, but I have ls aliased to ls -F. Apparently, -F changes the behaviour to showing the link instead of dereferencing it. Did not expect that.
    – marcelm
    Sep 6, 2018 at 21:55
  • Indeed! Also ls -l for example targets the link, not the destination... there have to be ways to inspect the link itself.
    – alexis
    Sep 6, 2018 at 22:03
  • 1
    Adding options to the question opens up too many possibilities-- my answer is about the unmodified behavior of ls and rm.
    – alexis
    Sep 6, 2018 at 22:03
  • If you say ls some_link/, ls -H some_link, or ls -L some_link, it will list the linked-to directory, even if you add -F or -l.  Conversely (sort-of), -d says to look at a directory rather than its contents; compare ls -l /tmp and ls -ld /tmp. Sep 8, 2018 at 22:52
  • Sure, you can add flags that change the behavior of ls. You're basically showing why enumerating behaviors with different flags is not worth the bother for this question...
    – alexis
    Sep 8, 2018 at 23:35
4

If you do only ls instead of ls -a, yes rm can remove hidden files you haven't seen with ls without -a.

Example :

According to :

dir_test
├── .test
└── test2

ls dir_test : will display only test2

ls -A dir_test : will display test2 + .test

rm -r dir_test : will remove all (.test + test2)

I hope that will help you.

4
  • Can you provide an example? Because normally, rm * won't remove dotfiles. If it does, ls * will also show them.
    – marcelm
    Sep 5, 2018 at 10:56
  • No, ls * don't display hidden files.
    – DevHugo
    Sep 5, 2018 at 13:31
  • But yes it's a little bit confused, I added some examples.
    – DevHugo
    Sep 5, 2018 at 13:38
  • 1
    ls -a will list ., .., .test and test2.  You might want to change your example to use ls -A, which list everything except . and .. (i.e., only .test and test2). Sep 8, 2018 at 22:22
3

There are already many good answers, but I want to add some more deep insight.

Ask yourself the question: How many parameters are passed to ls, if you write

ls *

...? Note that the ls command does not get the * as parameter if there are any files that * can be expanded to. Instead, the shell first performs globbing before invoking the command, so the ls command actually gets as many parameters as there are files matched by the globbing. To suppress globbing, quote the parameter.

This is true for any command: echo * vs echo '*'.

There is a script, call it countparams.sh to test the effect. It tells you how many parameters it got passed and lists them.

#!/bin/bash
echo "This script was given $# parameters."
arr=( "$@" )
for ((i=0;i<$#;i++)); do
        echo "Parameter $((i+1)): ${arr[$i]}"
done

Make it executable and run ./countparams.sh *. Learn from its output!

1

The glob will expand the same way both times, if the directory contents are the same at those two different times.


If you really want to check what will be removed, use rm -i *.txt. It will prompt you separately for each file before (trying to) remove it.

This is guaranteed to be safe against race conditions:
        ls *.txt / a new file is created / rm *.txt
because you're prompted for every file by the same program that's doing the removal.


This is too cumbersome for normal use, and if you alias rm to rm -i, you'll find yourself using \rm or rm -f fairly often. But it is worth at least mentioning that there is a solution to the race condition. (It's even portable to non-GNU systems: POSIX rm(1) specifies the -i option.)

Another option would be a bash array: to_remove=(*.txt), then ask the user to confirm (perhaps after doing ls -ld -- "${to_remove[@]}"), then rm -- "${to_remove[@]}". So glob expansion is only done once, and the list is passed verbatim to rm.

Another practically-usable option is GNU rm -I (man page), which prompts if removing more than 4 items. (But doesn't show you the list, just the total.) I use alias rm='rm -I' on my desktop.

It's a nice safeguard against fat-fingering return with a half-typed pattern that matches too much. But using ls first is generally good in a directory you own, or on a single-user system, and when there aren't background processes that could asynchronously create new files there. To guard against fat-fingering, don't type rm -rf /foo/bar/baz from left to right. rm -rf / is special-cased, but rm -rf /usr isn't!  Leave out the -rf part, or start with ls, and only add the rm -rf part after typing the path.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .