Tar new volume script fails with "Syntax error: Bad fd number"

Question

My shell script (see below) is failing partway through execution with "/bin/sh: 1: Syntax error: Bad fd number." This thread suggests that the problem is using the >& construction in sh, which in Ubuntu is linked to dash, which does not support >&. I am puzzled, because while the error message suggests that my script is running in sh, I am pretty confident that it is running in bash:

• The login shell from which I am executing it is bash, as indicated by echo $0.
• The first line of the script is #!/bin/bash
• If I insert readlink /proc/$$/exe into my script (as suggested here), the output is /usr/bin/bash.
• Both /bin/bash and /usr/bin/bash are actual executables, not links to other files.

So, if I'm wrong and it's actually running in sh, why is it doing so? And if it is actually running in bash, why is it returning an error when that syntax is supposed to work in bash?

I'm running this in Ubuntu 22.04 using a 6.5.0-28 kernel.

Any feedback is appreciated.

---

The purpose of this script is to create a multipart tar (using this method) from a list of files and directories in a text file, capping the part size at 512 Gb. This should generate four files, but it consistently fails after the third part:

#!/bin/bash
tar -cvf /home/me/archive.tar --multi-volume --tape-length=512000 --new-volume-script='echo -n "/home/me/archive-${TAR_VOLUME}.tar" >&${TAR_FD}' --exclude-tag-all=donotcopy --exclude-backups -T /home/me/file-list.txt

The full error text at failure is:

bin/sh: 1: Syntax error: Bad fd number
tar: ‘echo -n "/home/me/archive-${TAR_VOLUME}.tar" >&${TAR_FD}’ command failed
tar: Error is not recoverable: exiting now

The format of file-list.txt is as follows:

/home/me/foo
/home/me/bar/file-1.xyz
/home/me/bar/file-2.xyz

Answer 1

The tar command uses /bin/sh to execute the command line given to --new-volume-script.

Evidence?

tar -cf /tmp/archive.tar --multi-volume --tape-length=1024 --new-volume-script='ps -f >/tmp/x' /etc 2>/dev/null
cat /tmp/x

Example output clearly showing that PID 25662 is executed with /bin/sh:

UID        PID  PPID  C STIME TTY          TIME CMD
chris    25210 25207  0 00:58 pts/0    00:00:00 -bash
chris    25651 25210  5 01:03 pts/0    00:00:00 tar -cf /tmp/archive.tar --multi-volume --tape-length=1024 --new-volume-script=ps -f >/tmp/x /etc
chris    25662 25651  0 01:03 pts/0    00:00:00 /bin/sh -c ps -f >/tmp/x
chris    25663 25662 99 01:03 pts/0    00:00:00 ps -f

Having established that tar invokes sh, and that sh is often implemented by dash rather than bash, we can move on to information about the file descriptor number:

tar -cf /home/me/archive.tar --multi-volume --tape=length=512000 --new-volume-script='echo "TAR_VOLUME=$TAR_VOLUME, TAR_FD=$TAR_FD." >&2; echo -n "/home/me/archive-${TAR_VOLUME}.tar" >&${TAR_FD}' --exclude-tag-all=donotcopy --exclude-backups -T /home/me/file-list.txt

The resulting output is

TAR_VOLUME=2, TAR_FD=9.
TAR_VOLUME=3, TAR_FD=9.
TAR_VOLUME=4, TAR_FD=10.

And fails

TAR_VOLUME=5, TAR_FD=9.
TAR_VOLUME=6, TAR_FD=10.

And fails

Empirically we can suggest that a two digit file descriptor is causing the issue. Reading man dash leads us here:

The [n] is an optional number between 0 and 9, as in 3 (not [3]), that refers to a file descriptor

In other words a file descriptor for dash must be in the range 0 to 9, and file descriptors 10 and above cannot be referenced by number.

The solution here is to step outside dash into a shell with more flexibility, such as bash (I've split the command across several lines for improved readability):

tar -cvf /home/me/archive.tar --multi-volume --tape-length=512000 --new-volume-script='
      exec bash -c "echo \"/home/me/archive-$TAR_VOLUME.tar\" >&$TAR_FD"
    ' --exclude-tag-all=donotcopy --exclude-backups -T /home/me/file-list.txt