307

What's the difference between <<, <<< and < < in bash?

8
  • 42
    At least in these Google-centric times, it's difficult to search for these symbol-based operators. Is there a search engine where you could plug in "<< <<< < <" and get anything useful? Sep 27, 2015 at 17:17
  • 29
    @DanielGriscom There's SymbolHound.
    – Dennis
    Sep 28, 2015 at 5:48
  • 5
    @DanielGriscom Stack Exchange used to support searching symbols, but then something broke and no one ever fixed it.
    – muru
    Sep 28, 2015 at 7:17
  • 3
    @muru classic stack exchange
    – qwr
    Jun 30, 2021 at 4:09
  • 7
    @DanielGriscom I just googled bash <<< and this came up first. So looks like they improved something in the last 7 years ;)
    – Jorn
    Jan 28, 2022 at 8:39

4 Answers 4

356
+50

Here document

<< is known as here-document structure. You let the program know what will be the ending text, and whenever that delimiter is seen, the program will read all the stuff you've given to the program as input and perform a task upon it.

Here's what I mean:

$ wc << EOF
> one two three
> four five
> EOF
 2  5 24

In this example we tell wc program to wait for EOF string, then type in five words, and then type in EOF to signal that we're done giving input. In effect, it's similar to running wc by itself, typing in words, then pressing CtrlD

In bash these are implemented via temp files, usually in the form /tmp/sh-thd.<random string>, while in dash they are implemented as anonymous pipes. This can be observed via tracing system calls with strace command. Replace bash with sh to see how /bin/sh performs this redirection.

$ strace -e open,dup2,pipe,write -f bash -c 'cat <<EOF
> test
> EOF'

Here string

<<< is known as here-string. Instead of typing in text, you give a pre-made string of text to a program. For example, with such program as bc we can do bc <<< 5*4 to just get output for that specific case, no need to run bc interactively. Think of it as the equivalent of echo '5*4' | bc.

Here-strings in bash are implemented via temporary files, usually in the format /tmp/sh-thd.<random string>, which are later unlinked, thus making them occupy some memory space temporarily but not show up in the list of /tmp directory entries, and effectively exist as anonymous files, which may still be referenced via file descriptor by the shell itself, and that file descriptor being inherited by the command and later duplicated onto file descriptor 0 (stdin) via dup2() function. This can be observed via

$ ls -l /proc/self/fd/ <<< "TEST"
total 0
lr-x------ 1 user1 user1 64 Aug 20 13:43 0 -> /tmp/sh-thd.761Lj9 (deleted)
lrwx------ 1 user1 user1 64 Aug 20 13:43 1 -> /dev/pts/4
lrwx------ 1 user1 user1 64 Aug 20 13:43 2 -> /dev/pts/4
lr-x------ 1 user1 user1 64 Aug 20 13:43 3 -> /proc/10068/fd

And via tracing syscalls (output shortened for readability; notice how temp file is opened as fd 3, data written to it, then it is re-opened with O_RDONLY flag as fd 4 and later unlinked, then dup2() onto fd 0, which is inherited by cat later ):

$ strace -f -e open,read,write,dup2,unlink,execve bash -c 'cat <<< "TEST"'
execve("/bin/bash", ["bash", "-c", "cat <<< \"TEST\""], [/* 47 vars */]) = 0
...
strace: Process 10229 attached
[pid 10229] open("/tmp/sh-thd.uhpSrD", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
[pid 10229] write(3, "TEST", 4)         = 4
[pid 10229] write(3, "\n", 1)           = 1
[pid 10229] open("/tmp/sh-thd.uhpSrD", O_RDONLY) = 4
[pid 10229] unlink("/tmp/sh-thd.uhpSrD") = 0
[pid 10229] dup2(4, 0)                  = 0
[pid 10229] execve("/bin/cat", ["cat"], [/* 47 vars */]) = 0
...
[pid 10229] read(0, "TEST\n", 131072)   = 5
[pid 10229] write(1, "TEST\n", 5TEST
)       = 5
[pid 10229] read(0, "", 131072)         = 0
[pid 10229] +++ exited with 0 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=10229, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
+++ exited with 0 +++

Opinion: potentially because here strings make use of temporary text files, it is the possible reason why here-strings always insert a trailing newline, since text file by POSIX definition has to have lines that end in newline character.

Process Substitution

As tldp.org explains,

Process substitution feeds the output of a process (or processes) into the stdin of another process.

So in effect this is similar to piping stdout of one command to the other , e.g. echo foobar barfoo | wc . But notice: in the bash manpage you will see that it is denoted as <(list). So basically you can redirect output of multiple (!) commands.

Note: technically when you say < < you aren't referring to one thing, but two redirection with single < and process redirection of output from <( . . .).

Now what happens if we do just process substitution?

$ echo <(echo bar)
/dev/fd/63

As you can see, the shell creates temporary file descriptor /dev/fd/63 where the output goes (which according to Gilles's answer, is an anonymous pipe). That means < redirects that file descriptor as input into a command.

So very simple example would be to make process substitution of output from two echo commands into wc:

$ wc < <(echo bar;echo foo)
      2       2       8

So here we make shell create a file descriptor for all the output that happens in the parenthesis and redirect that as input to wc .As expected, wc receives that stream from two echo commands, which by itself would output two lines, each having a word, and appropriately we have 2 words, 2 lines, and 6 characters plus two newlines counted.

How is process substitution implemented ? We can find out using the trace below (output shortened for brevity)

$ strace -e clone,execve,pipe,dup2 -f bash -c 'cat <(/bin/true) <(/bin/false) <(/bin/echo)'
execve("/bin/bash", ["bash", "-c", "cat <(/bin/true) <(/bin/false) <"...], 0x7ffcb96004f8 /* 50 vars */) = 0
pipe([3, 4])                            = 0
dup2(3, 63)                             = 63
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f6c12569a10) = 8954
strace: Process 8954 attached
[pid  8953] pipe([3, 4])                = 0
[pid  8953] dup2(3, 62)                 = 62
[pid  8953] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f6c12569a10) = 8955
strace: Process 8955 attached
[pid  8953] pipe([3, 4])                = 0
[pid  8953] dup2(3, 61)                 = 61
[pid  8953] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f6c12569a10) = 8956
[pid  8953] execve("/bin/cat", ["cat", "/dev/fd/63", "/dev/fd/62", "/dev/fd/61"], 0x55ab7566e8f0 /* 50 vars */) = 0

The above trace on Ubuntu (which also implies on Linux in general) suggests that process substitution is implemented by repeatedly forking multiple subprocesses (so process 8953 forks multiple child processes 8954,8955,8956,etc). Then all these subprocesses communicate back via their stdout , but that is duplicated (that is copied) onto the stack of next available file descriptors starting at 63 downwards. Why start at 63 ? That may be a good question for the developers. It is known for a fact that bash can use fd 255 for saving file descriptors for the "main" command/pipeline when its streams are redirected.

Side Note: Process substitution may be referred to as a bashism (a command or structure usable in advanced shells like bash, but not specified by POSIX), but it was implemented in ksh before bash's existence as ksh man page and this answer suggest. Shells like tcsh and mksh however do not have process substitution. So how could we go around redirecting output of multiple commands into another command without process substitution? Grouping plus piping!

$ (echo foo;echo bar) | wc
      2       2       8

Effectively this is the same as above example, However, this is different under the hood from process substitution, since we make stdout of the whole subshell and stdin of wc linked with the pipe. On the other hand, process substitution makes a command read a temporary file descriptor.

So if we can do grouping with piping, why do we need process substitution? Because sometimes we cannot use piping. Consider the example below - comparing outputs of two commands with diff (which needs two files, and in this case we are giving it two file descriptors)

diff <(ls /bin) <(ls /usr/bin)
12
  • 9
    < < is used when one is getting stdin from a process substitution. Such a command might look like: cmd1 < <(cmd2). For example, wc < <(date)
    – John1024
    Sep 27, 2015 at 8:05
  • 4
    Yes, process substitution is supported by bash, zsh, and AT&T ksh{88,93} but not by pdksh/mksh.
    – John1024
    Sep 27, 2015 at 8:12
  • 2
    < < isn't a thing by itself, in the case of process substitution it's just a < followed by something else that happens to start with <
    – user253751
    Sep 27, 2015 at 8:40
  • 1
    @muru As far as I know, <<< was first implemented by the Unix port of Plan 9 rc shell then later adopted by zsh, bash and ksh93. I wouldn't then call it a bashism.
    – jlliagre
    Sep 27, 2015 at 11:19
  • 5
    Another example of where piping cannot be used: echo 'foo' | read; echo ${REPLY} will not return foo, because read is started in a sub-shell — piping starts a sub-shell. However, read < <(echo 'foo'); echo ${REPLY} correctly returns foo, because there is no sub-shell. Sep 29, 2015 at 10:38
48

< < is a syntax error:

$ cat < <
bash: syntax error near unexpected token `<'

< <() is process substitution (<()) combined with redirection (<):

A contrived example:

$ wc -l < <(grep ntfs /etc/fstab)
4
$ wc -l <(grep ntfs /etc/fstab)
4 /dev/fd/63

With process substitution, the path to the file descriptor is used like a filename. In case you don't want to (or can't) use a filename directly, you combine process substitution with redirection.

To be clear, there is no < < operator.

2
  • i get by your answer that , < <() more useful than <() right?
    – solfish
    Aug 17, 2017 at 13:28
  • 4
    @solfish <() gives a filename-like thing, so it's more generally useful - < <() is replacing the stdin where it might not be necessary. In wc, the latter happens to be more useful. It might be less useful elsewhere
    – muru
    Aug 17, 2017 at 15:07
21

< < is a syntax error, you probably mean command1 < <( command2 ) which is a simple input redirection followed by a process substitution and is very similar but not equivalent to :

command2 | command1

The difference assuming you are running bash is command1 is run in a subshell in the second case while it is run in the current shell in the first one. That means variables set in command1 would not be lost with the process substitution variant.

6
  • Isn't in the first the subshell? Don't we start a new subshell by ()?
    – ado sar
    Mar 7, 2022 at 14:54
  • @adosar Sorry, I don't understand your question. Can you rephrase it clarifying which "first" you are referring to?
    – jlliagre
    Mar 7, 2022 at 18:00
  • Apologize. I was referring to the subshell. Isn't a subshell created in the first case < <(command2) ?
    – ado sar
    Mar 7, 2022 at 18:39
  • @adosar Yes, command2 is run in a subshell in the first example. What is run in the current shell is command1. No parenthesis are needed in the second example, the pipeline puts both command1 and command2 in subshells. That's bash default behavior.
    – jlliagre
    Mar 7, 2022 at 18:49
  • 1
    @adosar The reason why the shell put all components of a pipeline in a subshell is it's easier to implement by shell developers. There is no potential conflicts between commands to solve. That's also the defaut POSIX behavior. (POSIX: Additionally, each command of a multi-command pipeline is in a subshell environment)
    – jlliagre
    Mar 8, 2022 at 0:43
17

< < will give an syntax error. Proper use is as follows:

Explaining with the help of examples:

Example for < <():

while read line;do
   echo $line
done< <(ls)

In the above example, the input to the while loop will come from the ls command which can be read line by line and echoed in the loop.

<() is used for process substitution. More information and example for <() can be found at this link:

Process substitution and pipe

0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .