5 
$ bash --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)

First, I create 5 files

$ touch a b c d e
$ ls
a  b  c  d  e

In the following example, xargs seems to get all the results from find via the pipe all at once:

$ find . -type f | xargs -t echo | wc -w
echo ./c ./b ./a ./d ./e
5

In the following example, xargs seems to get all the results from find via the pipe one at a time (line by line):

$ find . -type f | xargs -t -i mv {} {}.bak | wc  -w
mv ./c ./c.bak
mv ./b ./b.bak
mv ./a ./a.bak
mv ./d ./d.bak
mv ./e ./e.bak
0

In the following example, xargs seems to get all the results from 'find' via the pipe one at a time (line by line); and wc seems to get all the results from xargs via the pipe all at once.

Otherwise, if wc got the results line by line, you would see 1 appear 5 times instead of a 5 at the end.

$ find . -type f | xargs -t -i echo {} | wc -w
echo ./e.bak
echo ./b.bak
echo ./c.bak
echo ./a.bak
echo ./d.bak
5

So do pipes get results from previous command One at a Time (Line by Line) or All at Once?

Improve this question

2 Answers 2

Reset to default
4

The behavior you are observing comes from the xargs options you are using. The -i option specifically instructs xargs to create one new process for each input token, and replace the {} placeholder with that token.

Pipes by themselves do not stipulate any specific behavior; many kernels will pass one full I/O buffer at a time (hence the many questions about I/O buffering with pipes, along the lines of "this command doesn't seem to do anything in a pipe" when the actual symptom is just that the output buffer isn't full yet).

Improve this answer
4
  • find . -type f | xargs -t -i echo {} | xargs -t echo This command demonstrates what you wrote is right. -i changes the behavior of xargs.
    – J.Joe
    Commented Apr 6, 2018 at 5:42
  • 1
    Yes, exactly. (By the by, the default action of xargs is echo so you don't actually need to spell it out.) The first xargs prints one at a time as you requested and the second collects them all back on one line, which is the primary use case for xargs.
    – tripleee
    Commented Apr 6, 2018 at 5:44
  • Does it also mean xargs -t echo actually replace all the newline after each result with a space? while cat preserves the newline as in find . -type f | xargs -t -i echo {} | cat
    – J.Joe
    Commented Apr 6, 2018 at 5:52
  • Again, the primary purpose of xargs is to collect as many arguments as possible on the command line of the command it is spawning. The spaces are supplied by echo when it receives these five arguments in its argv array.
    – tripleee
    Commented Apr 6, 2018 at 5:57
2

| simply passes stdout from a process to the stdin of the next process. For example, find . -type f | grep "bla" translates to (roughly): Run find . -type f, and whatever this process prints to stdout, pass it as it is to grep "bla" as stdin.

It is up to the receiving process to determine what to do with it. xargs applies the given substitution on each line. wc, on the other hand, keeps track of various statistics (words, lines etc.) until there is no more input, and then prints it out.

It might be useful to read up on stdout, stdin etc. - http://www.learnlinux.org.za/courses/build/shell-scripting/ch01s04.html

Improve this answer
1
  • This sounds a plausible explanation. It is up to the command after the pipe to decide how to use the data passed on from the previous command(process). So it seems the data are sometimes processed line by line or sometimes all at once( as the wc example shows).
    – J.Joe
    Commented Apr 6, 2018 at 5:24

Not the answer you're looking for? Browse other questions tagged or ask your own question.