210

I am trying to run a for loop for file and I want to display whole line. But instead its displaying last word only. I want the complete line.

for j in `cat ./file_wget_med`

do
echo $j

done

result after run:

Found.

Here is my data:

$ cat file_wget_med
2013-09-11 14:27:03 ERROR 404: Not Found.
1

7 Answers 7

301

for loop splits when it sees any whitespace like space, tab, or newline. So, you should use IFS (Internal Field Separator):

IFS=$'\n'       # make newlines the only separator
for j in $(cat ./file_wget_med)    
do
    echo "$j"
done
# Note: IFS needs to be reset to default!
9
  • 5
    And put care about single quotes on IFS because IFS=$"\n" will split also "nn" strings. I found set IFS more reliable than use more complicated syntax or functions.
    – m3nda
    Aug 25, 2015 at 4:08
  • 38
    apparently it is advised to unset IFS afterwards, so that other commands are not influenced in this same shell. Oct 8, 2015 at 14:49
  • 7
    What does the $ mean in IFS=$'\n'?
    – Ren
    Dec 4, 2016 at 8:02
  • 3
    $ makes line breaks work inside the string. This should also be done with local IFS=$'\n' inside a function.
    – Andy Ray
    Mar 3, 2017 at 18:34
  • 4
    @Renn that's $'...' known "ANSI-C Quoting" Jun 7, 2018 at 16:43
127

for loops split on any whitespace (space, tab, newline) by default; the easiest way to work on one line at a time is to use a while read loop instead, which splits on newlines:

while read i; do echo "$i"; done < ./file_wget_med

I would expect your command to spit out one word per line (that's what happened when I tested it out with a file of my own). If something else is happening, I'm not sure what could be causing it.

9
  • 17
    This is also a good alternative to for i in `ls`; do echo $1; done, by piping the output to the while command: ls|while read i; do echo $i; done. This works on file/folder names with spaces, without needing to change environment variables. Very good answer.
    – jishi
    Jun 1, 2015 at 21:23
  • the while didn't handle the first and last lines very, not as well as the for loop with IFS
    – grantbow
    Jul 28, 2015 at 15:16
  • 1
    @GrantBowman Depending on how the file is delimited, it might work better if you try while read -r i instead of just while read i. Nov 20, 2015 at 21:56
  • 3
    This is a GREAT answer, I used it to turn a list I had into PLIST entries for an IOS app I was developing on mac OSX. I replaced the file input by piping the clipboad using pbpaste -> pbpaste | while read t; do echo "<string>$t</string>"; done
    – Big Rich
    Mar 8, 2016 at 15:43
  • 4
    ls -1 may be used with | to guarantee one-per-line unformatted output May 7, 2016 at 0:08
23
#!/bin/bash
files=`find <subdir> -name '*'`
while read -r fname; do
    echo $fname
done <<< "$files"

Verified working, not the one liner you probably want but it's not possible to do so elegantly.

2
  • 2
    a here-string! most elegant of all other solutions IMO Jan 25, 2017 at 22:20
  • Notice that it works with bash but not with other shells like dash for example.
    – desgua
    May 31, 2020 at 13:35
8

Here is a slight extension to Mitchell Currie's answer, that I like because of the small scope of side effects, which avoids having to set a variable:

#!/bin/bash
while read -r fname; do
    echo $fname
done <<< "`find <subdir>`"
1
  • 1
    You could remove -name '*' and it would be the same, no?
    – wjandrea
    Dec 29, 2018 at 20:24
6

I would write it like this:

cat ./file_wget_med | while read -r j
do
    echo $j
done

as it requires least changes in the original script (except for the solution using IFS, but it modifies bash behavior not only for this loop control statement).

1
  • 1
    Pipe and cat are unnecessary here. while loop accepts redirection just fine, which is why this is usually written as while IFS= read -r line; do...done < input.txt Dec 31, 2018 at 5:14
3

Mapfile is a convenient way to read lines from a file into an indexed array, not as portable as read but slightly faster. By using for loop you avoid creating a subshell.

#!/bin/bash

mapfile -t < file.txt

for line in "${MAPFILE[@]}"; do
    echo $line
done

Keep in mind when using pipelines, it will put the while loop in a subshell. Changes inside the while loop like variables will not propagate to outer part of the script.

Example:

#!/bin/bash

a=0
printf %s\\n {0..5} | while read; do
  ((a++))
done
echo $a # 'a' will always be 0.

(Better solution):

#!/bin/bash

b=0
while read; do
  ((b++))
done < <(printf %s\\n {0..5})

echo $b # 'b' equal to 6 (works as expected).
1
  • OMG yes! I was launching tasks in the background, and using jobs/wait to control how many at a time. Until I found your "better solution" jobs/wait didn't see the background processes.
    – Michael
    Jul 12, 2021 at 22:43
-1

Dandalf got real close to a functional solution, but one should NOT EVER be trying to assign the result of unknown amounts of input (i.e. find ~/.gdfuse -name '*') to variables! Or, at least be trying to do such a thing via an array variable; if you insist on being so lazy! So, here's Dandalf's solution done without the dangerous maneuver; and in all in one line

while read -r fname; do
  echo $fname;
done <<< `find ~/.gdfuse -name '*'
2
  • Hey @odoncaoa , I tried to do the find command without the double quotes, but it makes all of the results appear as one line. I am confused about the "unknown amounts of input" with this command and the dangers involved. Can you provide an example of the dangers and how they could theoretically happen? I seriously do not desire to be lazy about it, I am just more comfortable in other programming languages.
    – Dandalf
    Jan 3, 2019 at 17:38
  • @Dandalf they are likely referring to memory leaks; storing potentially gigabytes of data in a variable is not a good idea unless you have a lot of RAM Nov 23, 2022 at 22:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .