63

When looping recursively through folders with files containing spaces the shell script I use is of this form, copied from the internet:

    while IFS= read -r -d $'\0' file; do
      dosomethingwith "$file"        # do something with each file
    done < <(find /bar -name *foo* -print0)

I think I understand the IFS bit, but I don't understand what the '< <(...)' characters mean. Obviously there's some sort of piping going on here.

It's very hard to Google "< <" or "<(", you see. I tried "angle bracket parenthesis" and "less-than parenthesis" but didn't find anything.

5
  • "man sh" is your friend, in any case. Commented Mar 14, 2010 at 17:52
  • 7
    It's not < < but it's the < and the <(...) operator, if i remember right Commented Mar 14, 2010 at 17:55
  • dosomething "$file" is definitely misleading. the reason why this construct was used is because in the original page an array was modified inside the loop
    – knittl
    Commented Mar 14, 2010 at 18:00
  • 1
    thanks for editing the question Jonathan. Realising that the pattern is "< <(..)", not "< <" makes a lot of difference.
    – stib
    Commented Mar 15, 2010 at 10:50
  • 2
    I think, for better understanding and recall, that the "process substitution" operator should be called the penguin operator Commented Sep 21, 2020 at 12:50

4 Answers 4

75

<() is called process substitution in the manual, and is similar to a pipe but passes an argument of the form /dev/fd/63 instead of using stdin.

< reads the input from a file named on command line.

Together, these two operators function exactly like a pipe, so it could be rewritten as

find /bar -name *foo* -print0 | while read line; do
  ...
done
3
  • 10
    it's not the same if you must not start a subshell
    – knittl
    Commented Mar 14, 2010 at 18:18
  • 7
    +1 because you have the correct name for the notation. As @knittl points out, it is not quite identical to your rewrite because the loop will run in a sub-shell, and any changes made in the variables by the loop only affect the sub-shell, not the main script. You can deal with that by redirecting the output of 'find' into '{ while ...; do ...; done; ...rest of script...; }', using the braces to group the whole of the rest of the script - loop and other material - into a single sub-shell. Commented Mar 14, 2010 at 18:39
  • 1
    thanks for that. I can feel my skull expanding over the part of the brain that does shell scripting. Having a precise term to google makes it much easier. Now I'm going to go find out what a subshell is.
    – stib
    Commented Mar 15, 2010 at 10:46
26

<( command ) is process substitution. Basically, it creates a special type of file called a "named pipe," then redirects the output of the command to be the named pipe. So for example, suppose you want to page through a list of files in an extra-big directory. You could do this:

ls /usr/bin | more

Or this:

more <( ls /usr/bin )

But NOT this:

more $( ls /usr/bin )

The reason for this becomes clear when you investigate further:

~$ echo $( ls /tmp )
gedit.maxtothemax.436748151 keyring-e0fuHW mintUpdate orbit-gdm orbit-maxtothemax plugtmp pulse-DE9F3Ei96ibD pulse-PKdhtXMmr18n ssh-wKHyBU1713 virtual-maxtothemax.yeF3Jo
~$ echo <( ls /tmp )
/dev/fd/63
~$ cat <( ls /tmp )
gedit.maxtothemax.436748151
keyring-e0fuHW
mintUpdate
orbit-gdm
orbit-maxtothemax
plugtmp
pulse-DE9F3Ei96ibD
pulse-PKdhtXMmr18n
ssh-wKHyBU1713
virtual-maxtothemax.yeF3Jo

/dev/fd/whatever acts like a text file with the output of the command between the parenthesis.

2
  • more <( ls /usr/bin ) shows /dev/fd/11 is not a regular file (use -f to see it) on my zsh and shows /dev/fd/63 is not a regular file (use -f to see it) on my bash.
    – aafulei
    Commented Aug 3, 2022 at 8:18
  • 2
    The second shell should be more < <(ls /usr/bin), since <(ls /usr/bin) converts stdout to file and return file path as args and ` < ` receives file path and convert it to stdin.
    – Yari
    Commented Aug 9, 2022 at 3:42
4

< redirects to stdin.

<() seems to be some sort of a reverse pipe, as mentioned on the page:

find /bar -name *foo* -print0 | \
while IFS= read -r -d $'\0' file; do
  dosomethingwith "$file"        # do something with each file
done

will not work, because the while loop will be executed in a subshell, and you'll lose changes made in the loop

1
  • This answer would benefit from an x-ref to 'process substitution' and a URL such as gnu.org/software/bash/manual/bashref.html#Process-Substitution that explains it. The key point is the sub-shell and changes made to variables in the sub-shell. The other way of dealing with it is: find ... | { while ...; do ...; done; ...rest of script...; }, using the braces to run all the rest of the script in a sub-shell instead of just the while loop. Commented Mar 14, 2010 at 18:35
0

You use the process substitution construct (as per https://www.gnu.org/software/bash/manual/html_node/Process-Substitution.html#Process-Substitution) mostly when you need to pass a file to a command but using the output of another command instead of a file, thus avoiding the need to create and populate that file in advance.

For ex. say that you want to compare the output of 2 commands, you can do like this:

diff <(command1) <(command2)

2
  • This question already has a similar accepted answer!
    – Csisanyi
    Commented Jan 13, 2023 at 16:30
  • Yes but it doesn’t explain at all why it could convenient to use such a construct Commented Jan 14, 2023 at 17:04

Not the answer you're looking for? Browse other questions tagged or ask your own question.