8

I want to do something repeatedly on a list of files. The files in questions have spaces in their names:

david@david: ls -l
total 32
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha
-rw-rw-r-- 1 david david  0 Mai  8 11:55 haha~
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (3rd copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (4th copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (5th copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (6th copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (7th copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (another copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (copy)

Now I want to stat each of these files:

david@david: echo '
for file in $(ls)
do
stat $file
done' | bash

(I use echo and a pipe in order to write multi-line commands.)

When I do that, it works correctly on those files that do not have any spaces in their names. But the others...

stat: cannot stat ‘(another’: No such file or directory
stat: cannot stat ‘copy)’: No such file or directory

Changing $(ls) to "$(ls)" or $file to "$file" does not work. What can I do?

Edit:

echo '
for files in *
do
stat "$files"
done' | bash

does the trick! As I'm new to bash, I want to keep things as simple as possible - so nothing with trying to escape spaces, or using xargs or the solution with read -r, although they solve the problem.

As some have asked: Yes, using this instead of stat * is weird. But I just wanted to find a general way to apply the same command on a bunch of file names in bash, using a for loop. So stat could stand for gzip, gpg or rm.

  • 1
    what's wrong with stat *? (;-) – Rmano May 8 '15 at 10:54
  • Also see mywiki.wooledge.org/ParsingLs – Rmano May 8 '15 at 10:59
  • I just use stat as an example. :) I want to collect file names with ls, and then use the results of ls in a bash loop. – user258532 May 8 '15 at 11:03
12

The multiple quote from the echo ' is complicating the thing.

You can just use:

for f in *; do stat -- "$f"; done

But also

stat -- * 

...and if you want to collect the files and then apply the command (why?) you can go with (but be careful with file containing new lines...(1))

for f in *; do echo "$f"; done | xargs stat --

...and if you want hidden files too, just use * .* as a pattern, but then remember that . and .. will be in the set.

As an aside, you shouldn't parse ls output.


(1) but if you have file names with newlines, you somewhat deserve it... ;-)

  • "and then apply the command (why?)" --> stat just serves as an arbitrary command, as I'm trying out how to do bash loops with file names. It could be gpg, gzip or whatever else. – user258532 May 8 '15 at 11:21
  • @user258532 whatever the command, always use for f in *; do command "$f"; done. Never parse ls, certainly never do it in a for loop and why use echo? – terdon May 8 '15 at 11:21
  • echo: Because I like writing the commands on multiple lines... :) – user258532 May 8 '15 at 11:24
  • 2
    @user258532 Huh? Why would you need echo for that? Just hit enter and continue on a newline. If you end a line with an open quote or on one of do, |, && etc, you can continue on the new line. Either that or use heredocs. No reason to use echo and it can also cause problems. – terdon May 8 '15 at 11:28
  • "Just hit enter and continue on a newline." Ouch. :-D Now my IQ is officially below 0 - you know, I'm really new to bash and things like that. But I actually earn my money with R (the statistical suite and scripting language). – user258532 May 8 '15 at 11:33
6

On a side note: you can split long / complicated commands over multiple lines by adding a space followed by a backslash and hitting Enter everytime you want to start writing into a new line, instead of forking multiple processes by using echo [...] | bash; also you should enclose $file in double quotes, to prevent stat from breaking in case of filenames containing spaces:

for file in $(ls); \
do \
stat "$file"; \
done

The problem is that $(ls) expands to a list of filenames containing spaces, and the same will happen also with "$(ls)".

Even solving this problem, this method will still break on filenames containing backslashes and on filenames containing newlines (as pointed out by terdon).

A solution for both problems would be to pipe the output of find to a while loop running read -r so that, at each iteration, read -r will store one line of find's output into $file:

find . -maxdepth 1 -type f | while read -r file; do \
    stat "$file"; \
done
  • 1
    and hidden files? :) – A.B. May 8 '15 at 11:07
  • 5
    That will still fail on file names containing newlines. Just don't parse ls. Ever. – terdon May 8 '15 at 11:09
  • 4
    @user258532 no it doesn't. Seriously, just don't parse ls. There are better and more robust ways. You might also want to read this: Why *not* parse `ls`? for more details. – terdon May 8 '15 at 11:12
  • 1
    The problem here is not ls, it's for -- for iterates over (IFS separated) words given after the in keyword` – glenn jackman May 8 '15 at 11:13
  • 1
    @glennjackman well yes, it's the combination of for and ls. for f in * would be fine, for example. – terdon May 8 '15 at 11:17
3

Use the good old find, works with hidden files, newlines and spaces.

find . -print0 | xargs -I {} -0 stat {}

or any other instead of stat

find . -print0 | xargs -I {} -0 file {}
find . -print0 | xargs -I {} -0 cat {}
1

As a R guy, I already found a workaround in R:

filenames <- dir(); # reads file names into an array.
                    # It works also recursively
                    # with dir(recursive=TRUE)
for (i in 1:length(filenames)) {
system(     # calls a system function
 paste(     # join stat and the file name
  "stat",
  filenames[i]
 )
)
}

I know, it's mad. I wish the output of ls would be easier to parse... R can deal with spaces, because dir() returns a quoted character value. Anything between the quotes is then a valid file name with spaces.

  • 3
    Don't bother (but +1 for effort! :). Just use for f in *, hit enter and continue on a new line: do, hit enter again, stat "$f", enter again, done enter. That's a command split nicely on 4 lines and won't break on any kind of file name. – terdon May 8 '15 at 11:33
1

I have run into other instances of whitespace issues in for loops, so the following (imo more robust) command is what I generally use. It also fits nicely into pipes.

$ ls | while read line; do stat "$line"; done;

You could combine this with grep or use find instead:

$ find ./ -maxdepth 2 | grep '^\./[/a-z]+$' | while read line; do stat "$line"; done;
  • Your first answer fails on filenames that contain backslash, or leading or trailing spaces.  Your second answer fails totally unless you add the -E option to grep, without which it won't recognize + in a regular expression.  Even then, it's a swing and a miss on this question, since your grep removes filenames containing spaces.  It also removes filenames containing digits (numerals) and punctuation (e.g., parentheses), as the examples in the question do.  And that's not even mentioning filenames that contain newline or begin with - (dash). – Scott Aug 26 '15 at 18:06
-1

This answer will solve the problem of parsing ls and take care of the backspaces and new lines

Try this, it will solve your problem using Internal Field Separator IFS.

IFS="\n" for f in $(ls); do   stat "$f"; done

But Also you can eaisly solve it without need to parse ls output using

for f in *; do   stat "$f"; done
  • 1
    doesn't work for hidden files. – A.B. May 8 '15 at 11:07
  • 2
    OP didn't ask for hidden files – Maythux May 8 '15 at 11:08
  • 1
    I don't see why you need to modify IFS here: quoting the variable should be sufficient to prevent word splitting, surely? – steeldriver May 8 '15 at 11:15
  • for parsing ls . – Maythux May 8 '15 at 11:16
  • For what is the downvote!!! – Maythux May 8 '15 at 11:20
-1

Instead, you can rename your files replacing the space with some other character such as underscore, so you get rid of this problem:

To do that easily run the command:

for file in * ; do mv "$f" "${f// /_}" ; done

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.