17

I have a bash script that builds a command-line in a string based on some parameters before executing it in one go. The parts that are concatenated to the command string are supposed to be separated by pipes to facilitate a "streaming" of data through each component.

A very simplified example:

#!/bin/bash
part1=gzip -c
part2=some_other_command
cmd="cat infile"

if [ ! "$part1" = "" ]
then
    cmd+=" | $part1"
fi


if [ ! "$part2" = "" ]
then
    cmd+=" | $part2"
fi


cmd+="> outfile"
#show command. It looks ok
echo $cmd
#run the command. fails with pipes
$cmd

For some reason the pipes don't seem to work. When I run this script i get different error messages relating usually to the first part of the command (before the first pipe).

So my question is whether or not it is possible to build a command in this way, and what is the best way to do it?

4
  • What are the error messages? Jun 4, 2014 at 3:23
  • In my script (which is somewhat more complex than this simplification) I get "file not found" Jun 4, 2014 at 3:26
  • Is it safe to assume that infile exists in the current directory? Jun 4, 2014 at 3:33
  • yes. in my code it is wget -O - instead of a file. Actually, if i just copy the concatenated string and pasete it in the terminal it runs fine Jun 4, 2014 at 3:58

4 Answers 4

22

It all depends on when things get evaluated. When you type $cmd, the whole rest of the line is passed as arguments to the first word in $cmd.

walt@spong:~(0)$ a="cat /etc/passwd"
walt@spong:~(0)$ b="| wc -l"
walt@spong:~(0)$ c="$a $b"
walt@spong:~(0)$ echo $c
cat /etc/passwd | wc -l
walt@spong:~(0)$ $c
cat: invalid option -- 'l'
Try 'cat --help' for more information.
walt@spong:~(1)$ eval $c
62
walt@spong:~(0)$ a="echo /etc/passwd"
walt@spong:~(0)$ c="$a $b"
walt@spong:~(0)$ echo $c
echo /etc/passwd | wc -l
walt@spong:~(0)$ $c
/etc/passwd | wc -l
walt@spong:~(0)$ $c |od -bc
0000000 057 145 164 143 057 160 141 163 163 167 144 040 174 040 167 143  
          /   e   t   c   /   p   a   s   s   w   d       |       w   c  
0000020 040 055 154 012  
              -   l  \n  
0000024
walt@spong:~(0)$ eval $c
1  

This shows that the arguments passed to the echo command are: "/etc/passwd", "|" (the vertical bar character), "wc" and "-l".

From man bash:

eval [arg ...]  
    The  args  are read and concatenated together into   
    a single command.  This command is then read and  
    executed by the shell, and its exit status is returned  
    as the value of eval.  If there are no args, or only null  
    arguments, eval returns 0.
10

One solution to this, for future reference, is to use "eval". This ensures that whatever way the string is interpreted by bash is forgotten and the whole thing is read as if it was typed directly in a shell (which is exactly what we want).

So in the example above, replacing

$cmd

with

eval $cmd

solved it.

2
  • 1
    Be careful though with quoted parameters. eval foo "a b" would be the same as eval foo "a" "b".
    – udondan
    Aug 25, 2016 at 9:10
  • myCMD="foo \"a b\""; eval "$myCMD" keeps the parameters quoted; it's the same as eval foo "a b".
    – baltakatei
    Jul 4, 2020 at 0:10
2

@waltinator already explained why this does not work as you expected. Another way around it is to use bash -c to execute your command:

$ comm="cat /etc/passwd"
$ comm+="| wc -l"
$ $comm
cat: invalid option -- 'l'
Try 'cat --help' for more information.
$ bash -c "$comm"
51
2
  • 2
    Parsimony tells me not to start another process with bash -c, but use eval to do the command in the current process.
    – waltinator
    Jun 4, 2014 at 14:20
  • @waltinator sure, I'd probably use eval for this also (which is why I upvoted you and Lennart). I'm just providing an alternative.
    – terdon
    Jun 4, 2014 at 14:24
0

Possibly a better way to do this is to avoid using eval and just using a Bash array and it's inline expansion to build up all the arguments and then execute them against the command.

runcmd=() # This is slightly messier than declare -a but works
for cmd in $part1 $part2 $part3; do runcmd+="| $cmd "; done
cat infile ${runcmd[@]} # You might be able to do $basecmd ${runcmd[@]}
# but that sometimes requires an `eval` which isn't great

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .