I wrote a script that deletes all except the last two files in a folder:

#!/bin/bash
ls -1 --quoting-style=shell-always /path/to/some/folder \
    | head -n -2 \
    | xargs printf -- "'/path/to/some/folder/%s'\n" \
    | xargs sudo rm -rf

This script will be executed as a cron job every day.

The reasoning is as follows:

  1. Obtain a list of all files using ls -1 (so that I get one file per line);

  2. Remove the last two from the list using head -n -2;

  3. Since ls prints relative paths, use the xargs printf thing to prepend the folder path and make it an absolute path;

  4. Send them to sudo rm -rf using xargs.

Everyone has access to this folder, so anyone can create and delete any files in this folder.

The problem is: sudo rm -rf is scary. xargs sudo rm -rf is incredibly scary.

I want to be sure that no one can damage other folders/systems by creating clever files to be deleted (either accidentally or on purpose). I don't know, something clever like:

file with / spaces.txt

which could result in a super scary sudo rm -rf /.

EDIT: My mistake, file names cannot contain /, so this specific problem wouldn't happen, but the question about whether or not there are other risks still stands.

This is why I am using --quoting-style=shell-always, this should prevent any tricks with files with spaces. But now I am wondering if someone could be extra clever with spaces and quotes in the filename, perhaps.

Is my script safe?


Note: I need sudo because I am acessing the folder remotely (from a mapped network drive using mount), and I couldn't get it to work without sudo.

  • 3
    Have you considered doing something like printf -- '%s\0' /path/to/some/folder/* | head -zn -2 | xargs -0 rm? – steeldriver Sep 4 at 17:45
  • Can a file with the character / in the name be created I am trying to achieve this back here – George Udosen Sep 4 at 17:52
  • 3
    @George No, a filename cannot contain a slash. – wjandrea Sep 4 at 17:54
  • So when OP says clever person I was wondering... – George Udosen Sep 4 at 17:55
  • 5
    Simply because you're parsing ls output this is already a poorly written command, even with quoting. ls also uses locale for sorting order, I think, so I don't see what's the purpose of head in removing last 2 ( unless you're trying to get rid of . and .. which iirc aren't allowed as arguments to rm anyway. Just use find /path/to/folder -type f delete. And no sudo if you run from cron - cron is already at root level – Sergiy Kolodyazhnyy Sep 4 at 19:35
up vote 10 down vote accepted

In Linux, any character is a valid filename constituting character except:

  • \0 (ASCII NUL): as used for string termination in C
  • / (forward slash): as used for path separation

So, your approach will definitely not work in many cases as you can imagine e.g. does it handle a newline (\n) in filename? (Hint: No).

Few notes:

  • Don't parse ls; use dedicated tools (there is at least one for most use cases)
  • When dealing with filenames, try to leverage the NUL separated output provided by almost all GNU tools that work with such data
  • Take care when piping, make sure both programs can understand NUL separations
  • Whenever you're invoking xargs, see if you can get away with find ... -exec; in most cases, you will be fine with just find alone

I think these will get you going for now. steeldriver already provided the NUL separated idea in the comment (printf -- '%s\0' /path/to/some/folder/* | head -zn -2 | xargs -0 rm), use this as a starting point.

  • Thanks for your answer :) I think you should quote steeldriver's comment instead of just mentioning (since comments are not permanent). I will take a look into find as well, thanks for the suggestion. – Pedro A Sep 5 at 13:39
  • I have one question though: I don't understand what you mean by "your approach will not definitely work in many cases as you can imagine" - "not work" as in "not safe" or "not unsafe"? Because "your approach" refers to me and not to the malicious user, and your prior statements are in my favor, so I am confused. – Pedro A Sep 5 at 13:40
  • @PedroA You are you :) Like i said, as all characters are valid except the mentioned two, you should be able to imagine the numerous cases your ls parsing approach would fail e.g. did you take into account a newline in filename? – heemayl Sep 5 at 13:46
  • Oh, a newline in the filename... I hadn't thought of that. If you don't mind adding that to your answer too :) Also, sorry to ask, but what does head -z do? It sounds ridiculous but I don't have man nor info in my CoreOS container linux... Couldn't find in the internet either. I get head: invalid option 'z' – Pedro A Sep 5 at 13:51
  • @PedroA Newline is just one case, there are many such as you might have guessed. You need GNU head (comes with GNU coreutils). Here's the online version: manpages.ubuntu.com/manpages/xenial/man1/head.1.html – heemayl Sep 5 at 13:54

xargs does support some quoting: with single quotes, double quotes or backslash which allows it to accept arbitrary arguments¹, but with a syntax that is different from the Bourne-like shells' quoting syntax.

The GNU implementation of ls as found on Ubuntu doesn't have any quoting mode that is compatible with the xargs input format.

Its ls --quoting-style=shell-always is compatible with ksh93, bash and zsh shells quoting syntax, but only when the output of ls is interpreted by the shell in the same locale as ls was when it output it. Also, some locales, like those using BIG5, BIG5-HKSCS, GBK or GB18030 should be avoided.

So with those shells, you can actually do:

typeset -a files
eval "files=($(ls --quoting-style=shell-always))"
xargs -r0a <(printf '%s\0' "${files[@]:0:3}") ...

But that has little advantage over:

files=(*(N))                 # zsh
files=(~(N)*)                # ksh93
shopt -s nullglob; files=(*) # bash

The only case where it becomes useful is when you want to use the -t option of ls to sort the files by mtime/atime/ctime or -S/-V. But even then, you might as well use zsh's:

files=(*(Nom))

for instance to sort the files by mtime (use oL for -S, and n for -V).

To remove all but the two most recently modified regular files:

rm -f -- *(D.om[3,-1])

¹ there are still some length limitations (by execve() and in some non-GNU xargs implementations much lower arbitrary ones), and some non-GNU xargs implementations will choke of input that contains sequences of bytes not forming valid characters.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.