8

I want to do something repeatedly on a list of files. The files in questions have spaces in their names:

david@david: ls -l
total 32
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha
-rw-rw-r-- 1 david david  0 Mai  8 11:55 haha~
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (3rd copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (4th copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (5th copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (6th copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (7th copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (another copy)
-rw-rw-r-- 1 david david 13 Mai  8 11:55 haha (copy)

Now I want to stat each of these files:

david@david: echo '
for file in $(ls)
do
stat $file
done' | bash

(I use echo and a pipe in order to write multi-line commands.)

When I do that, it works correctly on those files that do not have any spaces in their names. But the others...

stat: cannot stat ‘(another’: No such file or directory
stat: cannot stat ‘copy)’: No such file or directory

Changing $(ls) to "$(ls)" or $file to "$file" does not work. What can I do?

Edit:

echo '
for files in *
do
stat "$files"
done' | bash

does the trick! As I'm new to bash, I want to keep things as simple as possible - so nothing with trying to escape spaces, or using xargs or the solution with read -r, although they solve the problem.

As some have asked: Yes, using this instead of stat * is weird. But I just wanted to find a general way to apply the same command on a bunch of file names in bash, using a for loop. So stat could stand for gzip, gpg or rm.

3
  • 1
    what's wrong with stat *? (;-)
    – Rmano
    May 8, 2015 at 10:54
  • Also see mywiki.wooledge.org/ParsingLs
    – Rmano
    May 8, 2015 at 10:59
  • I just use stat as an example. :) I want to collect file names with ls, and then use the results of ls in a bash loop.
    – user258532
    May 8, 2015 at 11:03

7 Answers 7

12

The multiple quote from the echo ' is complicating the thing.

You can just use:

for f in *; do stat -- "$f"; done

But also

stat -- * 

...and if you want to collect the files and then apply the command (why?) you can go with (but be careful with file containing new lines...(1))

for f in *; do echo "$f"; done | xargs stat --

...and if you want hidden files too, just use * .* as a pattern, but then remember that . and .. will be in the set.

As an aside, you shouldn't parse ls output.


(1) but if you have file names with newlines, you somewhat deserve it... ;-)

7
  • "and then apply the command (why?)" --> stat just serves as an arbitrary command, as I'm trying out how to do bash loops with file names. It could be gpg, gzip or whatever else.
    – user258532
    May 8, 2015 at 11:21
  • @user258532 whatever the command, always use for f in *; do command "$f"; done. Never parse ls, certainly never do it in a for loop and why use echo?
    – terdon
    May 8, 2015 at 11:21
  • echo: Because I like writing the commands on multiple lines... :)
    – user258532
    May 8, 2015 at 11:24
  • 2
    @user258532 Huh? Why would you need echo for that? Just hit enter and continue on a newline. If you end a line with an open quote or on one of do, |, && etc, you can continue on the new line. Either that or use heredocs. No reason to use echo and it can also cause problems.
    – terdon
    May 8, 2015 at 11:28
  • "Just hit enter and continue on a newline." Ouch. :-D Now my IQ is officially below 0 - you know, I'm really new to bash and things like that. But I actually earn my money with R (the statistical suite and scripting language).
    – user258532
    May 8, 2015 at 11:33
6

On a side note: you can split long / complicated commands over multiple lines by adding a space followed by a backslash and hitting Enter everytime you want to start writing into a new line, instead of forking multiple processes by using echo [...] | bash; also you should enclose $file in double quotes, to prevent stat from breaking in case of filenames containing spaces:

for file in $(ls); \
do \
stat "$file"; \
done

The problem is that $(ls) expands to a list of filenames containing spaces, and the same will happen also with "$(ls)".

Even solving this problem, this method will still break on filenames containing backslashes and on filenames containing newlines (as pointed out by terdon).

A solution for both problems would be to pipe the output of find to a while loop running read -r so that, at each iteration, read -r will store one line of find's output into $file:

find . -maxdepth 1 -type f | while read -r file; do \
    stat "$file"; \
done
11
  • 1
    and hidden files? :)
    – A.B.
    May 8, 2015 at 11:07
  • 5
    That will still fail on file names containing newlines. Just don't parse ls. Ever.
    – terdon
    May 8, 2015 at 11:09
  • 4
    @user258532 no it doesn't. Seriously, just don't parse ls. There are better and more robust ways. You might also want to read this: Why *not* parse `ls`? for more details.
    – terdon
    May 8, 2015 at 11:12
  • 1
    The problem here is not ls, it's for -- for iterates over (IFS separated) words given after the in keyword` May 8, 2015 at 11:13
  • 1
    @glennjackman well yes, it's the combination of for and ls. for f in * would be fine, for example.
    – terdon
    May 8, 2015 at 11:17
3

Use the good old find, works with hidden files, newlines and spaces.

find . -print0 | xargs -I {} -0 stat {}

or any other instead of stat

find . -print0 | xargs -I {} -0 file {}
find . -print0 | xargs -I {} -0 cat {}
1

As a R guy, I already found a workaround in R:

filenames <- dir(); # reads file names into an array.
                    # It works also recursively
                    # with dir(recursive=TRUE)
for (i in 1:length(filenames)) {
system(     # calls a system function
 paste(     # join stat and the file name
  "stat",
  filenames[i]
 )
)
}

I know, it's mad. I wish the output of ls would be easier to parse... R can deal with spaces, because dir() returns a quoted character value. Anything between the quotes is then a valid file name with spaces.

1
  • 3
    Don't bother (but +1 for effort! :). Just use for f in *, hit enter and continue on a new line: do, hit enter again, stat "$f", enter again, done enter. That's a command split nicely on 4 lines and won't break on any kind of file name.
    – terdon
    May 8, 2015 at 11:33
1

I have run into other instances of whitespace issues in for loops, so the following (imo more robust) command is what I generally use. It also fits nicely into pipes.

$ ls | while read line; do stat "$line"; done;

You could combine this with grep or use find instead:

$ find ./ -maxdepth 2 | grep '^\./[/a-z]+$' | while read line; do stat "$line"; done;
1
  • Your first answer fails on filenames that contain backslash, or leading or trailing spaces.  Your second answer fails totally unless you add the -E option to grep, without which it won't recognize + in a regular expression.  Even then, it's a swing and a miss on this question, since your grep removes filenames containing spaces.  It also removes filenames containing digits (numerals) and punctuation (e.g., parentheses), as the examples in the question do.  And that's not even mentioning filenames that contain newline or begin with - (dash).
    – Scott
    Aug 26, 2015 at 18:06
-1

Instead, you can rename your files replacing the space with some other character such as underscore, so you get rid of this problem:

To do that easily run the command:

for file in * ; do mv "$f" "${f// /_}" ; done
-2

This answer will solve the problem of parsing ls and take care of the backspaces and new lines

Try this, it will solve your problem using Internal Field Separator IFS.

IFS="\n" for f in $(ls); do   stat "$f"; done

But Also you can eaisly solve it without need to parse ls output using

for f in *; do   stat "$f"; done
13
  • 1
    doesn't work for hidden files.
    – A.B.
    May 8, 2015 at 11:07
  • 2
    OP didn't ask for hidden files
    – Maythux
    May 8, 2015 at 11:08
  • 1
    I don't see why you need to modify IFS here: quoting the variable should be sufficient to prevent word splitting, surely? May 8, 2015 at 11:15
  • for parsing ls .
    – Maythux
    May 8, 2015 at 11:16
  • For what is the downvote!!!
    – Maythux
    May 8, 2015 at 11:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.