4

I had a problem in a bash backup script caused by the following behavior that I don't understand.

First, we create a parameter holding a string with 2 consecutive spaces (right before '<'):

me@mysystem:~$ TEST="a string with 2 spaces right  <--HERE!"
me@mysystem:~$ echo $TEST
a string with 2 spaces right <--HERE!

I can grep a space:

me@mysystem:~$ echo ${TEST} | grep " "
a string with 2 spaces right <--HERE!

I can grep a space followed by another letter:

me@mysystem:~$ echo ${TEST} | grep " s"
a string with 2 spaces right <--HERE!

I can grep the part after the 2 spaces:

me@mysystem:~$ echo ${TEST} | grep "HERE"
a string with 2 spaces right <--HERE!

I can do a variable expansion replacing both spaces with an underscore:

me@mysystem:~$ echo ${TEST// /_}
a_string_with_2_spaces_right__<--HERE!

But grepping the double space requires quoting the variable:

me@mysystem:~$ echo $TEST   | grep "  "; echo "Done."
Done.
me@mysystem:~$ echo ${TEST} | grep "  "; echo "Done."
Done.
me@mysystem:~$ echo "$TEST" | grep "  "; echo "Done."
a string with 2 spaces right  <--HERE!
Done.
me@mysystem:~$ echo "${TEST}" | grep "  "; echo "Done."
a string with 2 spaces right  <--HERE!
Done.

I don't understand why this is the case.

Long story short: I had an if statement in the script checking whether ${FILE} represents an existing file, and that apparently also stops interpreting the variable when encountering a space unless quoted. So I'm not sure anymore what ${} does or doesn't actually take care off.

It's best to always quote the variable I guess?

5

This is a classic case of word splitting, done by shell, on unquoted variable expansion.

Basically when you do, $TEST (without quotes ("$TEST")), the shell:

  • Performs word splitting, based on the value of IFS environment variable (default: space, tab, newline), and splits the whole expansion into separate words

  • so as you have two consecutive spaces, the word splitting (and also pathname (glob) expansion if any of *, ?, [] is present) is done, and when the output string is rebuilt the words are joined by space, as a result you have only one space between right and <--HERE!. This would be true for any combination of IFS characters.

In a nutshell, quote your variable expansions:

"${variable}"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.