134

I am trying to run a for loop for file and I want to display whole line. But instead its displaying last word only. I want the complete line.

for j in `cat ./file_wget_med`

do
echo $j

done

result after run:

Found.

Here is my data:

$ cat file_wget_med
2013-09-11 14:27:03 ERROR 404: Not Found.
184

for loop splits when it sees any whitespace like space, tab, or newline. So, you should use IFS (Internal Field Separator):

IFS=$'\n'       # make newlines the only separator
for j in $(cat ./file_wget_med)    
do
    echo "$j"
done
# Note: IFS needs to be reset to default!
  • 5
    And put care about single quotes on IFS because IFS=$"\n" will split also "nn" strings. I found set IFS more reliable than use more complicated syntax or functions. – m3nda Aug 25 '15 at 4:08
  • 23
    apparently it is advised to unset IFS afterwards, so that other commands are not influenced in this same shell. – Boris Däppen Oct 8 '15 at 14:49
  • 2
    What does the $ mean in IFS=$'\n'? – Ren Dec 4 '16 at 8:02
  • 2
    $ makes line breaks work inside the string. This should also be done with local IFS=$'\n' inside a function. – Andy Ray Mar 3 '17 at 18:34
  • 2
    @Renn that's $'...' known "ANSI-C Quoting" – αғsнιη Jun 7 '18 at 16:43
85

for loops split on any whitespace (space, tab, newline) by default; the easiest way to work on one line at a time is to use a while read loop instead, which splits on newlines:

while read i; do echo "$i"; done < ./file_wget_med

I would expect your command to spit out one word per line (that's what happened when I tested it out with a file of my own). If something else is happening, I'm not sure what could be causing it.

  • 8
    This is also a good alternative to for i in `ls`; do echo $1; done, by piping the output to the while command: ls|while read i; do echo $i; done. This works on file/folder names with spaces, without needing to change environment variables. Very good answer. – jishi Jun 1 '15 at 21:23
  • the while didn't handle the first and last lines very, not as well as the for loop with IFS – grantbow Jul 28 '15 at 15:16
  • @jishi Since it's primarily designed for display, and is responsive to terminal-window size and such, ls is not considered safe for use with | (the pipe operator). find is more flexible in addition to being safer for this purpose. – SeldomNeedy Nov 20 '15 at 21:48
  • 3
    This is a GREAT answer, I used it to turn a list I had into PLIST entries for an IOS app I was developing on mac OSX. I replaced the file input by piping the clipboad using pbpaste -> pbpaste | while read t; do echo "<string>$t</string>"; done – Big Rich Mar 8 '16 at 15:43
  • 3
    ls -1 may be used with | to guarantee one-per-line unformatted output – AndreyS Scherbakov May 7 '16 at 0:08
19
#!/bin/bash
files=`find <subdir> -name '*'`
while read -r fname; do
    echo $fname
done <<< "$files"

Verified working, not the one liner you probably want but it's not possible to do so elegantly.

  • 1
    a here-string! most elegant of all other solutions IMO – Eliran Malka Jan 25 '17 at 22:20
5

Here is a slight extension to Mitchell Currie's answer, that I like because of the small scope of side effects, which avoids having to set a variable:

#!/bin/bash
while read -r fname; do
    echo $fname
done <<< "`find <subdir>`"
  • 1
    You could remove -name '*' and it would be the same, no? – wjandrea Dec 29 '18 at 20:24
1

I would write it like this:

cat ./file_wget_med | while read -r j
do
    echo $j
done

as it requires least changes in the original script (except for the solution using IFS, but it modifies bash behavior not only for this loop control statement).

  • 1
    Pipe and cat are unnecessary here. while loop accepts redirection just fine, which is why this is usually written as while IFS= read -r line; do...done < input.txt – Sergiy Kolodyazhnyy Dec 31 '18 at 5:14
0

Mapfile is a convenient way to read lines from a file into an indexed array, not as portable as read but slightly faster. By using for loop you avoid creating a subshell.

#!/bin/bash

mapfile -t < file.txt

for line in "${MAPFILE[@]}"; do
    echo $line
done

Keep in mind when using pipelines, it will put the while loop in a subshell. Changes inside the while loop like variables will not propagate to outer part of the script.

Example:

#!/bin/bash

a=0
printf %s\\n {0..5} | while read; do
  ((a++))
done
echo $a # 'a' will always be 0.

(Better solution):

#!/bin/bash

b=0
while read; do
  ((b++))
done < <(printf %s\\n {0..5})

echo $b # 'b' equal to 6 (works as expected).
-1

Dandalf got real close to a functional solution, but one should NOT EVER be trying to assign the result of unknown amounts of input (i.e. find ~/.gdfuse -name '*') to variables! Or, at least be trying to do such a thing via an array variable; if you insist on being so lazy! So, here's Dandalf's solution done without the dangerous maneuver; and in all in one line

while read -r fname; do
  echo $fname;
done <<< `find ~/.gdfuse -name '*'
  • Hey @odoncaoa , I tried to do the find command without the double quotes, but it makes all of the results appear as one line. I am confused about the "unknown amounts of input" with this command and the dangers involved. Can you provide an example of the dangers and how they could theoretically happen? I seriously do not desire to be lazy about it, I am just more comfortable in other programming languages. – Dandalf Jan 3 at 17:38

Not the answer you're looking for? Browse other questions tagged or ask your own question.