I had imagined the simplest way to compare the contents of two similar directories would be something like

diff `ls old` `ls new`

But I see why this doesn't work; diff is being handed a big long list of files on the command line, rather than two streams like I had hoped. How do I pass the two outputs to diff directly?

up vote 199 down vote accepted

Command substitution `…` substitutes the output of the command into the command line, so diff sees the list of files in both directories as arguments. What you want is for diff to see two file names on its command line, and have the contents of these files be the directory listings. That's what process substitution does.

diff <(ls old) <(ls new)

The arguments to diff will look like /dev/fd/3 and /dev/fd/4: they are file descriptors corresponding to two pipes created by bash. When diff opens these files, it'll be connected to the read side of each pipe. The write side of each pipe is connected to the ls command.

  • 43
    echo <(echo) <(echo) never thought this could be so interesting :D – Aquarius Power Dec 13 '14 at 1:46
  • 3
    Process substitution is not supported by all shells, but pipe redirects are a neat workaround. – Irfan434 Sep 16 '17 at 14:38
  • 1
    Just to mention that parsing ls is not recommended unix.stackexchange.com/questions/128985/why-not-parse-ls – Katu Apr 12 at 7:59
  • @Katu The problem with ls is that it mangles file names. Parsing its output is fragile (it doesn't work with “weird” file names). For comparing two directory listings, it's ok as long as the output is unambiguous. With arbitrary file names, this would require an option such as --quoting-style=escape. – Gilles Apr 12 at 11:03

Fish shell

In Fish shell you have to pipe into psub. Here is an example of heroku and dokku config comparison with Beyond Compare:

bcompare (ssh me@myapp.pl dokku config myapp | sort | psub) (heroku config -a myapp | sort | psub)
  • Another graphical diff tool is meld which is open source and available in the Ubuntu and EPEL repositories. meldmerge.org – phiphi Jun 18 at 7:36

For zsh, using =(command) automatically creates a temporary file and replaces =(command) with the path of the file itself. With Command Substitution, $(command) is replaced with the output of the command.

So there are three options:

  1. Command Substitution: $(...)
  2. Process Substitution: <(...)
  3. zsh-Flavored Process Substitution: =(...)

zsh flavored process subsitution, #3, is very useful and can be used like so to compare the output of two commands using a diff tool, for example Beyond Compare:

bcomp  =(ulimit -Sa | sort) =(ulimit -Ha | sort)

For Beyond Compare, note that you must use bcomp for the above (instead of bcompare) since bcomp launches the comparison and waits for it to complete. If you use bcompare, that launches comparison and immediately exits due to which the temporary files created to store the output of the commands disappear.

Read more here: http://zsh.sourceforge.net/Intro/intro_7.html

Also notice this:

Note that the shell creates a temporary file, and deletes it when the command is finished.

and the following which is the difference between the two types of Process substitution supported by zsh (i.e #2 and #3):

If you read zsh's man page, you may notice that <(...) is another form of process substitution which is similar to =(...). There is an important difference between the two. In the <(...) case, the shell creates a named pipe (FIFO) instead of a file. This is better, since it does not fill up the file system; but it does not work in all cases. In fact, if we had replaced =(...) with <(...) in the examples above, all of them would have stopped working except for fgrep -f <(...). You can not edit a pipe, or open it as a mail folder; fgrep, however, has no problem with reading a list of words from a pipe. You may wonder why diff <(foo) bar doesn't work, since foo | diff - bar works; this is because diff creates a temporary file if it notices that one of its arguments is -, and then copies its standard input to the temporary file.

Reference: https://unix.stackexchange.com/questions/393349/difference-between-subshells-and-process-substitution

  • 2
    $(...) is not process substitution, it's command substitution. <(...) is process substitution. That's why the quoted passage doesn't mention $(...) at all. – muru Jun 29 at 9:10

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.