3

This question is related to my older one here.

After I finally found out how to bypass the output of git into a function I have now another issue. Using

git clone --progress XYZ &> git_clone.file

writes as expected

Cloning to 'someRepository' ...
remote: Counting objects: 2618, done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 2618 (delta 2), reused 12 (delta 1), pack-reused 2603
Received objects: 100% (2618/2618), 258.95 MiB | 4.39 MiB/s, Done.
Resolving Differences auf: 100% (1058/1058), Done.
Check Connectivity ... Done.

into git_clone.file.

Now I don't want to redirect the output to a file but to a function so I use

function PrintInfo(){
    tput el
    echo $1
    <Print ProgressBar> 
    #For further details about this see 
    # https://askubuntu.com/questions/988403/bash-pass-command-output-to-function-in-realtime
}

git clone --progress XYZ |& {
    while read -r line
    do
        PrintInfo $line
    done
}

Now I would expect to get

Cloning to 'someRepository' ...
remote: Counting objects: 2618, done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 2618 (delta 2), reused 12 (delta 1), pack-reused 2603
Received objects: 100% (2618/2618), 258.95 MiB | 4.39 MiB/s, Done.
Resolving Differences auf: 100% (1058/1058), Done.
Check Connectivity ... Done.

printed line by line and at the bottom my progressbar as described in my other question. But instead I only get

Cloning
remote:
remote:
Received
...

and so on. I already tried all forms of IFS like

... while IFS= read -r ...

... while IFS='' read -r ...

... while IFS="\n" read -r ...

but none of them solves this issue.

How can I read the complete line of the output?

6

The problem is word splitting. To solve it, replace:

PrintInfo $line

With:

PrintInfo "$line"

Explanation

Consider:

PrintInfo $line

Before executing, PrintInfo, the shell will expand $line and perform both word splitting and pathname expansion.

Let's take a simple example, starting by defining a function:

$ PrintInfo() { echo "1=$1 2=$2 3=$3"; }

Now, let's execute the function:

$ line="one two three"
$ PrintInfo $line
1=one 2=two 3=three

In the above, word splitting caused PrintInfo to see three arguments.

If you want PrintInfo to see just one argument, use:

$ PrintInfo "$line"
1=one two three 2= 3=

Pathname expansion is also a potential problem. Consider a directory with these files:

$ ls
file1  file2  file3

Now, let's run our version of PrintInfo again:

$ line='file?'
$ PrintInfo $line
1=file1 2=file2 3=file3

Because ? is a valid glob character, the shell looks to replace file? with filenames. To prevent that surprise, use double-quotes:

$ PrintInfo "$line"
1=file? 2= 3=

Summary

Except in cases where you explicitly want word splitting or pathname expansion, shell variables should always be inside double-quotes.

  • 1
    @derHugo Be careful. That's not a full solution. Without the double-quotes, PrintInfo $line is still subject to pathname expansion. – John1024 Dec 23 '17 at 23:20
  • yes you are right – derHugo Dec 24 '17 at 8:08
  • note that you also need to quote the $1 inside the function, otherwise you'll get the same splitting and globbing issues there. (it's not that obvious with echo, but it will still trash whitespace inside the variable) – ilkkachu Dec 25 '17 at 19:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.