132

I am trying to run a for loop for file and I want to display whole line. But instead its displaying last word only. I want the complete line.

for j in `cat ./file_wget_med`

do
echo $j

done

result after run:

Found.

Here is my data:

$ cat file_wget_med
2013-09-11 14:27:03 ERROR 404: Not Found.
182

for loop splits when it sees any whitespace like space, tab, or newline. So, you should use IFS (Internal Field Separator):

IFS=$'\n'       # make newlines the only separator
for j in $(cat ./file_wget_med)    
do
    echo "$j"
done
# Note: IFS needs to be reset to default!
  • 5
    And put care about single quotes on IFS because IFS=$"\n" will split also "nn" strings. I found set IFS more reliable than use more complicated syntax or functions. – erm3nda Aug 25 '15 at 4:08
  • 23
    apparently it is advised to unset IFS afterwards, so that other commands are not influenced in this same shell. – Boris Däppen Oct 8 '15 at 14:49
  • 2
    What does the $ mean in IFS=$'\n'? – Ren Dec 4 '16 at 8:02
  • 2
    $ makes line breaks work inside the string. This should also be done with local IFS=$'\n' inside a function. – Andy Ray Mar 3 '17 at 18:34
  • 2
    @Renn that's $'...' known "ANSI-C Quoting" – αғsнιη Jun 7 '18 at 16:43
82

for loops split on any whitespace (space, tab, newline) by default; the easiest way to work on one line at a time is to use a while read loop instead, which splits on newlines:

while read i; do echo "$i"; done < ./file_wget_med

I would expect your command to spit out one word per line (that's what happened when I tested it out with a file of my own). If something else is happening, I'm not sure what could be causing it.

  • 8
    This is also a good alternative to for i in `ls`; do echo $1; done, by piping the output to the while command: ls|while read i; do echo $i; done. This works on file/folder names with spaces, without needing to change environment variables. Very good answer. – jishi Jun 1 '15 at 21:23
  • the while didn't handle the first and last lines very, not as well as the for loop with IFS – grantbow Jul 28 '15 at 15:16
  • @jishi Since it's primarily designed for display, and is responsive to terminal-window size and such, ls is not considered safe for use with | (the pipe operator). find is more flexible in addition to being safer for this purpose. – SeldomNeedy Nov 20 '15 at 21:48
  • 3
    This is a GREAT answer, I used it to turn a list I had into PLIST entries for an IOS app I was developing on mac OSX. I replaced the file input by piping the clipboad using pbpaste -> pbpaste | while read t; do echo "<string>$t</string>"; done – Big Rich Mar 8 '16 at 15:43
  • 3
    ls -1 may be used with | to guarantee one-per-line unformatted output – AndreyS Scherbakov May 7 '16 at 0:08
19
#!/bin/bash
files=`find <subdir> -name '*'`
while read -r fname; do
    echo $fname
done <<< "$files"

Verified working, not the one liner you probably want but it's not possible to do so elegantly.

  • 1
    a here-string! most elegant of all other solutions IMO – Eliran Malka Jan 25 '17 at 22:20
5

Here is a slight extension to Mitchell Currie's answer, that I like because of the small scope of side effects, which avoids having to set a variable:

#!/bin/bash
while read -r fname; do
    echo $fname
done <<< "`find <subdir>`"
  • 1
    You could remove -name '*' and it would be the same, no? – wjandrea Dec 29 '18 at 20:24
1

I would write it like this:

cat ./file_wget_med | while read -r j
do
    echo $j
done

as it requires least changes in the original script (except for the solution using IFS, but it modifies bash behavior not only for this loop control statement).

  • 1
    Pipe and cat are unnecessary here. while loop accepts redirection just fine, which is why this is usually written as while IFS= read -r line; do...done < input.txt – Sergiy Kolodyazhnyy Dec 31 '18 at 5:14
0

Mapfile is a convenient way to read lines from a file into an indexed array, not as portable as read but slightly faster. By using for loop you avoid creating a subshell.

#!/bin/bash

mapfile -t < file.txt

for line in "${MAPFILE[@]}"; do
    echo $line
done

Keep in mind when using pipelines, it will put the while loop in a subshell. Changes inside the while loop like variables will not propagate to outer part of the script.

Example:

#!/bin/bash

a=0
printf %s\\n {0..5} | while read; do
  ((a++))
done
echo $a # 'a' will always be 0.

(Better solution):

#!/bin/bash

b=0
while read; do
  ((b++))
done < <(printf %s\\n {0..5})

echo $b # 'b' equal to 6 (works as expected).
-1

Dandalf got real close to a functional solution, but one should NOT EVER be trying to assign the result of unknown amounts of input (i.e. find ~/.gdfuse -name '*') to variables! Or, at least be trying to do such a thing via an array variable; if you insist on being so lazy! So, here's Dandalf's solution done without the dangerous maneuver; and in all in one line

while read -r fname; do
  echo $fname;
done <<< `find ~/.gdfuse -name '*'
  • Hey @odoncaoa , I tried to do the find command without the double quotes, but it makes all of the results appear as one line. I am confused about the "unknown amounts of input" with this command and the dangers involved. Can you provide an example of the dangers and how they could theoretically happen? I seriously do not desire to be lazy about it, I am just more comfortable in other programming languages. – Dandalf Jan 3 at 17:38

protected by Sergiy Kolodyazhnyy Dec 31 '18 at 5:13

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.