10

I'm writing on a huge script-factory script generating a lot of maintenance scripts for my servers.

Until now I write some lines which need to be written in one line with echo -ne e.g.

echo -n "if (( " | sudo tee -a /usr/local/bin/upgradeAllServers &> /dev/null

# Generate exitCode check for each Server
IFS=" "
COUNT=0
while read -r name ipAddr
do
    if(($COUNT != 0))
    then
        echo -n " || " | sudo tee -a /usr/local/bin/upgradeAllServers &> /dev/null
    fi

    echo -n "(\$"$name"_E != 0 && \$"$name"_E != 1)" | sudo tee -a /usr/local/bin/upgradeAllServers &> /dev/null

    COUNT=$((COUNT+1))

    JOBSDONE=$((JOBSDONE+1))
    updateProgress $JOBCOUNT $JOBSDONE
done <<< "$(sudo cat /root/.virtualMachines)"

echo " ))" | sudo tee -a /usr/local/bin/upgradeAllServers &> /dev/null

this generates me (if there are e.g. 2 servers in my config file) code like

if (( ($server1_E != 0 && $server1_E != 1) || ($server2_E != 0 && $server2_E != 1) ))

All other code blocks which don't need this inline writing of code I produce with heredocs since I find them way better to write and maintain. E.g. after the upper code I have the rest generated like

cat << EOF | sudo tee -a /usr/local/bin/upgradeAllServers &> /dev/null
then
    # Print out ExitCode legend
    echo " ExitCode 42 - upgrade failed"
    echo " ExitCode 43 - Upgrade failed"
    echo " ExitCode 44 - Dist-Upgrade failed"
    echo " ExitCode 45 - Autoremove failed"
    echo ""
    echo ""
fi
EOF

So the final code block looks like

if (( ($server1_E != 0 && $server1_E != 1) || ($server2_E != 0 && $server2_E != 1) ))
then
    # Print out ExitCode legend
    echo " ExitCode 42 - upgrade failed"
    echo " ExitCode 43 - Upgrade failed"
    echo " ExitCode 44 - Dist-Upgrade failed"
    echo " ExitCode 45 - Autoremove failed"
    echo ""
    echo ""
fi

My Question
Is there a way having a heredoc behave similar to echo -ne without the end-of-line-symbol?

  • 1
    If you need sudo in a script you're doing it wrong: Run the whole script as root instead! – dessert Nov 30 '17 at 9:35
  • No because it is also doing other stuff I don't want to run as root – derHugo Nov 30 '17 at 11:15
  • The use sudo -u USERNAME on that instead, see How do I run a 'sudo' command inside a script?. – dessert Nov 30 '17 at 11:20
  • what is the problem doing it as I do? – derHugo Nov 30 '17 at 12:05
  • 4
    Well of course there's a problem: A script with sudo without username asks for the password, unless you typed it in there's a delay. It's even worse if you don't run the script in a terminal, then you can't even see the password query and it will just no work. The sudo -u approach doesn't have any of these problems. – dessert Nov 30 '17 at 13:13
10

No, heredoc always ends in a newline. But you can still remove the last newline, for example with Perl:

cat << 'EOF' | perl -pe 'chomp if eof'
First line
Second line
EOF
  • -p reads the input line by line, runs the code specified in -e, and prints the (possibly modified) line
  • chomp removes the final newline if it exists, eof returns true at the end of file.
  • 1
    @Yaron if per encountered end-of-file ( in this case closed pipe) it will chomp newline char from last line ( which is what heredoc and herestring add automatically) – Sergiy Kolodyazhnyy Nov 30 '17 at 9:22
7

Is there a way having a heredoc behave similar to echo -ne without the end-of-line-symbol?

Short answer is that heredoc and herestrings are just built that way, so no - here-doc and herestring will be adding a trailing newline always and there's no option or native way to disable it ( perhaps there will be in future bash releases?).

However, one way to approach it via bash-only ways would be to read what heredoc gives via the standard while IFS= read -r method, but add a delay and print last line via printf without the trailing newline. Here's what one could do with bash-only:

#!/usr/bin/env bash

while true
do
    IFS= read -r line || { printf "%s" "$delayed";break;}
    if [ -n "$delayed" ]; 
    then
        printf "%s\n" "$delayed"
    fi
    delayed=$line
done <<EOF
one
two
EOF

What you see here is the usual while loop with IFS= read -r line approach, however each line is stored into a variable and thus delayed ( so we skip printing first line, store it, and start printing only when we've read second line). That way we can capture last line, and when on next iteration read returns exit status 1, that's when we print last line without newline via printf. Verbose ? yes. But it works:

$ ./strip_heredoc_newline.sh                                      
one
two$ 

Notice the $ prompt character being pushed forward, since there's no newline. As a bonus, this solution is portable and POSIX-ish, so it should work in ksh and dash as well.

$ dash ./strip_heredoc_newline.sh                                 
one
two$
6

I recommend you try a different approach. Rather than piecing together your generated script from multiple echo statements and heredocs. I suggest you try to use a single heredoc.

You can use variable expansion and command substitution inside a heredoc. Like in this example:

#!/bin/bash
cat <<EOF
$USER $(uname)
EOF

I find that this often leads to much more readable end results than producing the output piece by piece.

Also it looks like you have been forgetting the #! line in the beginning of your scripts. The #! line is mandatory in any correctly formatted scripts. Attempting to run a script without the #! line will only work if you call it from a shell that works around badly formatted scripts, and even in that case it might end up getting interpreted by a different shell than you intended.

  • havent forgotten the #! but my code block is just one snippet out of a 2000 lines script ;) I don't see right now how your suggestion solves my problem generating one line of code in a while loop... – derHugo Nov 30 '17 at 13:59
  • @derHugo Command substitution for the part of the line where you need a loop is one way to do that. Another way is to construct that part in a variable before the heredoc. Which of the two is most readable depends on the length of the code needed in the loop as well as how many lines of heredoc you have before the line in question. – kasperd Nov 30 '17 at 14:10
  • @derHugo Command substitution invoking a shell function defined earlier in the script is a third approach which may be more readable if you need a significant amount of code to generate that one line. – kasperd Nov 30 '17 at 14:12
  • @derHugo: Note that: (1) You can have a loop that puts all the required commands into a variable; (2) You can use $( ...command...) inside a heredoc, to insert the output of those commands inline at that point in the heredoc; (3) Bash has array variables, which you can expand inline, and use substitutions to add things at the start/end if necessary. Together these make kasperd's suggestion much more powerful (and your script potentially much more readable). – psmears Nov 30 '17 at 15:56
  • I'm using the heredocs because of their kind of template behavior (wysiwyg). Producing the whole template in another place and than passing it to the heredoc in my opinion doesn't really make things easier to read/maintain. – derHugo Dec 21 '17 at 15:06
1

You could remove the newlines in whatever way you like, piping to tr would probably be simplest. This would remove all newlines, of course.

$ tr -d '\n' <<EOF 
if ((
EOF

But, the if statement you're building doesn't require that, the arithmetic expression (( .. )) should work fine even if it contains newlines.

So, you could do

cat <<EOF 
if ((
EOF
for name in x y; do 
    cat << EOF
    ( \$${name}_E != 0 && \$${name}_E != 1 ) ||
EOF
done
cat <<EOF
    0 )) ; then 
    echo do something
fi
EOF

producing

if ((
    ( $x_E != 0 && $x_E != 1 ) ||
    ( $y_E != 0 && $y_E != 1 ) ||
    0 )) ; then 
    echo do something
fi

Building it in a loop still makes ugly, though. The || 0 is there of course so that we don't need to special case the first or the last line.


Also, you have that | sudo tee ... in every output. I think we could get rid of that by opening a redirection only once (with process substitution), and using that for the later printing:

exec 3> >(sudo tee outputfile &>/dev/null)
echo output to the pipe like this >&3
echo etc. >&3
exec 3>&-         # close it in the end

And frankly, I still think that building variable names from variables in the outer script (like that \$${name}_E ) is a bit ugly, and it should probably be replaced with an associative array.

1

Heredocs always end in a new-line characters but you can simply remove that. The easiest way I know is with the head program:

head -c -1 <<EOF | sudo tee file > /dev/null
my
heredoc
content
EOF
  • Cool I didn't know that -c also takes negative values! Like that even more than the pearl solution – derHugo Dec 21 '17 at 15:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.