Why diff3 on inputs from process substitution finds a difference while diff3 on same data in files finds no difference?

Question

diff3 on files finds no difference:

$ grep -P '\[\[.*?\]\]' -o intro.tex | sort > A.txt
$ grep -P '\[\[.*?\]\]' -o intro.tex | sort | uniq > B.txt
$ grep '\\pnum %% \[\[' intro.tex | sed 's/\\pnum %% //' | sort > C.txt
$ diff3 A.txt B.txt C.txt | wc -l
0

diff3 on process substitutions running the same commands finds a difference:

$ diff3 \
  <(grep -P '\[\[.*?\]\]' -o intro.tex | sort) \
  <(grep -P '\[\[.*?\]\]' -o intro.tex | sort | uniq) \
  <(grep '\\pnum %% \[\[' intro.tex | sed 's/\\pnum %% //' | sort) | wc -l
95

Why? Any ideas?

A minimum reproducer:

$ echo test > a
$ diff3 a a a
$ diff3 <(cat a) <(cat a) <(cat a)
====1
1:1c
  test
2:0a
3:0a

Answer 1

If we run diff3 under strace -f:

$ strace -qqfe execve -e signal=none diff3 a b c
execve("/usr/bin/diff3", ["diff3", "a", "b", "c"], 0x7ffef2bb2d78 /* 53 vars */) = 0
[pid 13360] execve("/usr/bin/diff", ["diff", "--horizon-lines=100", "--", "b", "c"], 0x7ffc019c6d50 /* 53 vars */) = 0
[pid 13361] execve("/usr/bin/diff", ["diff", "--horizon-lines=100", "--", "a", "c"], 0x7ffc019c6d50 /* 53 vars */) = 0

As you can see diff3 calls diff twice, and the third file is one of the operands for both invocations.

With ksh-style <(...) process substitution, the file is a pipe.

cmd1 <(cmd2)

is the same as:

cmd2 | cmd1 /dev/fd/0

except that a fd other than 0 is used.

So, for the third argument to diff3, the first diff will consume the whole input, the second will have nothing left to read, so the file will appear empty.

So the third argument at least cannot be a pipe¹.

If using zsh, you can use the =(...) form of process substitution that uses a temporary file instead of a pipe:

diff3 <(cmd1) <(cmd2) =(cmd3)

(the first 2 can still be pipes).

In your case:

$ diff3 \
  <(grep -Po '\[\[.*?\]\]' intro.tex | sort) \
  <(grep -Po '\[\[.*?\]\]' intro.tex | sort -u) \
  =(sed -n 's/\\pnum %% \[\[/[[/p' intro.txt | sort)

(I've also removed the superfluous grep and uniq and moved the -o option before the non-options).

You could even do some factorisation with:

(){ diff3 $1 <(uniq<$1) $2; } =(grep -Po '\[\[.*?\]\]' intro.tex) \
                              =(sed -n 's/\\pnum %% \[\[/[[/p' intro.txt | sort)

---

^{¹ if the third argument is - which diff3 interprets as reading stdin, then it's the second argument that cannot be a pipe as that's the one that it passes to both diff invocations in that case. You'll notice that the manual says: "At most one of these three file names may be -, which tells diff3 to read the standard input for that file."}