5

I had a problem in a bash backup script caused by the following behavior that I don't understand.

First, we create a parameter holding a string with 2 consecutive spaces (right before '<'):

me@mysystem:~$ TEST="a string with 2 spaces right  <--HERE!"
me@mysystem:~$ echo $TEST
a string with 2 spaces right <--HERE!

I can grep a space:

me@mysystem:~$ echo ${TEST} | grep " "
a string with 2 spaces right <--HERE!

I can grep a space followed by another letter:

me@mysystem:~$ echo ${TEST} | grep " s"
a string with 2 spaces right <--HERE!

I can grep the part after the 2 spaces:

me@mysystem:~$ echo ${TEST} | grep "HERE"
a string with 2 spaces right <--HERE!

I can do a variable expansion replacing both spaces with an underscore:

me@mysystem:~$ echo ${TEST// /_}
a_string_with_2_spaces_right__<--HERE!

But grepping the double space requires quoting the variable:

me@mysystem:~$ echo $TEST   | grep "  "; echo "Done."
Done.
me@mysystem:~$ echo ${TEST} | grep "  "; echo "Done."
Done.
me@mysystem:~$ echo "$TEST" | grep "  "; echo "Done."
a string with 2 spaces right  <--HERE!
Done.
me@mysystem:~$ echo "${TEST}" | grep "  "; echo "Done."
a string with 2 spaces right  <--HERE!
Done.

I don't understand why this is the case.

Long story short: I had an if statement in the script checking whether ${FILE} represents an existing file, and that apparently also stops interpreting the variable when encountering a space unless quoted. So I'm not sure anymore what ${} does or doesn't actually take care off.

It's best to always quote the variable I guess?

1 Answer 1

6

This is a classic case of word splitting, done by shell, on unquoted variable expansion.

Basically when you do, $TEST (without quotes ("$TEST")), the shell:

  • Performs word splitting, based on the value of IFS environment variable (default: space, tab, newline), and splits the whole expansion into separate words

  • so as you have two consecutive spaces, the word splitting (and also pathname (glob) expansion if any of *, ?, [] is present) is done, and when the output string is rebuilt the words are joined by space, as a result you have only one space between right and <--HERE!. This would be true for any combination of IFS characters.

In a nutshell, quote your variable expansions:

"${variable}"

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .