I created the following script:

cat > Top10 <<EOF
Linux Mint 17.2
Ubuntu 15.10
Debian GNU/Linux 8.2
Mageria 5
Fedora 23
openSUSE Leap 42.1
Arch Linux
CentOS 7.2-1511
PCLinuxOS 2014.12
Slackware Linux 14.1
sed -ri "s/^[^0-9]*$//" Top10
sed -r "s/(.*)([[:space:]][[:digit:]]*.*)$/\2\1/" Top10 | sed -r "s/([[:space:]])([[:digit:]])/\2/" | sed -r "s/([[:digit:]])([[:alpha:]])/\1 \2/" > Top10
sed -r -i "s/(.*)/\L\1/" Top10
sed -r -i "y/[aeiou]/[AEIOU]/" Top10
sort Top10 -g -o Top10
cat Top10

When I run it a few times the following happens:

As you can see sometimes the Top10 file turns out empty and sometimes it turn out the way I needed it to be. I know that the command which replaces the extensions from the end to the front of a line is done poorly. I ran this script on a VMware virtual machine. Could that be the reason?

  • I'm flagging to close this question as off-topic because as it currently stands, it's not about Ubuntu within the scope as defined in the help center. Please edit your question to help explain what your ubuntu-specific problem or question is. – cat Jan 9 '16 at 15:58
  • If you have a question about GNU Sed or shell scripting in general, you may be able to get help at Unix & Linux. – cat Jan 9 '16 at 15:58
  • 1
    Another thing to notice: You shouldn't type sh zadatak3.sh to invoke the script. Typing the name of the shell on the command line when invoking a script is not the proper way to invoke a script. The correct place to indicate which shell to use is the #! line at the start of the script. You shouldn't be overriding that. You should have typed ./zadatak3.sh instead. – kasperd Jan 9 '16 at 23:17

More specifically, pipes are not deterministic.

I.e. in a pipe such as this one:

command1 file | command2 | command3 >file

it's not guaranteed that command1 file will be executed before command3 >file.

So the race condition between command1 file and command3 >file makes it so that sometimes the file is first read by command1 file and sometimes the file is first truncated by command3 >file, giving the expected output in the first case and a giving an empty output in the second case.

This can be fixed by using sponge (in the moreutils package) to write the output to the file, to make sure that the output gets written to the file only after the rest of the pipe has finished executing:

command1 file | command2 | command3 | sponge file
  • 2
    I think one possible solution in addition to that would be to turn sed commands into one single script , instead of piping it over and over. – Sergiy Kolodyazhnyy Jan 9 '16 at 19:00
  • @Serg Sure. Actually that would be the way to go (my answer only wanted to address why the problem showed up and how to fix simlar problems in general). In this specific case the way to go is muru's answer. – kos Jan 10 '16 at 5:30
  • @StenSoft Since every command of the pipe is run in a subshell, each redirection is resolved by a different subshell, and since the order in which the subshells are to be run is not defined, there's a race condition between the first and the last command: running for((i=0;i<20;i++)); do echo foo >file | echo bar >file; cat file; done should be enough to be convinced of that. – kos Jan 12 '16 at 18:18
  • @StenSoft I'm afraid it's often but not always: paste.ubuntu.com/14479882. You problem is being convinced that the redirections are always resolved first. That is true for a single command, but not for commands in a pipe: in that case the resolution of the redirections for each single command is delegated to the subshell that executes the command. – kos Jan 12 '16 at 18:58
sed -r "s/(.*)([[:space:]][[:digit:]]*.*)$/\2\1/" Top10 | sed -r "s/([[:space:]])([[:digit:]])/\2/" | sed -r "s/([[:digit:]])([[:alpha:]])/\1 \2/" > Top10

causes the file to be first overwritten with > Top10 and only then processed by sed (the file is empty at that time)


There is no reason to run 6 sed commands when you can do it in one:

sed -ri 's/^[^0-9]*$//; s/(.*)/\L\1/; y/[aeiou]/[AEIOU]/; s/(.*)([[:space:]])([[:digit:]]*.*)$/\3 \1/' Top10

Note, in particular, how the second pipe of sed commands could be combined into one expression if you just grouped your matches differently. Since you wanted to discard the leading whitespace anyway, and then add a space after the version, you could do that in the original match itself, by grouping the leading space and the version separately.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.