File descriptors & shell scripting

Question

I am having a very hard time understanding how does one use file descriptors in shell scripts.

I know the basics such as

exec 5 > /tmp/foo

So fd 5 is attached to foo for writing.

exec 6 < /tmp/bar

… for reading.

exec 5>&-

… close fd.

Now what does this do?

#!/bin/bash

exec 5 > /tmp/foo 
exec 6 < /tmp/bar 

cat <&6 | while read a
do
     echo $a >&5
done

As i understand &5 closes the fd, so how is the output still being re-directed successfully after each call?

This is a copy pasta from : Here

It claims using this over a simple echo $a > file would make it much quicker, I however fail to understand. I would appreciate any links to decent tutorial. I google powers seem to be failing me.

Answer 1

First, note that the syntax for closing is 5>&- or 6<&-, depending on whether the file descriptor is being read for writing or for reading. There seems to be a typo or formatting glitch in that blog post.

Here's the commented script.

exec 5>/tmp/foo       # open /tmp/foo for writing, on fd 5
exec 6</tmp/bar       # open /tmp/bar for reading, on fd 6
cat <&6 |             # call cat, with its standard input connected to
                      # what is currently fd 6, i.e., /tmp/bar
while read a; do      # 
  echo $a >&5         # write to fd 5, i.e., /tmp/foo
done                  # 

There's no closing here. Because all the inputs and outputs are going to the same place in this simple example, the use of extra file descriptors is not necessary. You could write

cat </tmp/bar |
while read a; do
  echo $a
done >/tmp/foo

Using explicit file descriptors becomes useful when you want to write to multiple files in turn. For example, consider a script that outputs data to a data output file and logging data to a log file and possibly error messages as well. That means three output channels: one for data, one for logs and one for errors. Since there are only two standard descriptors for output, a third is needed. You can call exec to open the output files:

exec >data-file
exec 3>log-file
echo "first line of data"
echo "this is a log line" >&3
…
if something_bad_happens; then echo error message >&2; fi
exec >&-  # close the data output file
echo "output file closed" >&3

The remark about efficiency comes in when you have a redirection in a loop, like this (assume the file is empty to begin with):

while …; do echo $a >>/tmp/bar; done

At each iteration, the program opens /tmp/bar, seeks to the end of the file, appends some data and closes the file. It is more efficient to open the file once and for all:

while …; do echo $a; done >/tmp/bar

When there are multiple redirections happening at different times, calling exec to perform redirections rather than wrapping a block in a redirection becomes useful.

exec >/tmp/bar
while …; do echo $a; done

You'll find several other examples of redirection by browsing the io-redirection tag on this site.

Answer 2

Each open file gets assigned a file descriptor and the file descriptor for stdin is 0 stdout is 1
stderr is 2 For opening additional files, there remain descriptors 3 to 9.

Take for example

exec 3>&1 4>&2
exec 1>  /proc/1/fd/1 2>&1

3>&1 creates a new file descriptor and redirect it to 1 which is STDOUT 4>&2 also new file descriptor and redirect it to 2 which is STDERR

1> /proc/1/fd/1 uses a socket file descriptor 2>&1 also redirects everything from this file descriptor to 1 which is STDOUT

Ive also learned this pretty recently, so feel free to point out if i am missing something

References for using exec and redirecting using file descriptions https://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/x13082.html