Using bash, how to pass filename arguments to a command sorted by date and dealing with spaces and other special characters?

Question

I am using the bash shell and want to execute a command that takes filenames as arguments; say the cat command. I need to provide the arguments sorted by modification time (oldest first) and unfortunately the filenames can contain spaces and a few other difficult characters such as "-", "[", "]". The files to be provided as arguments are all the *.txt files in my directory. I cannot find the right syntax. Here are my efforts.

Of course, cat *.txt fails; it does not give the desired order of the arguments.

cat `ls -rt *.txt`

The `ls -rt *.txt` gives the desired order, but now the blanks in the filenames cause confusion; they are seen as filename separators by the cat command.

cat `ls -brt *.txt`

I tried -b to escape non-graphic characters, but the blanks are still seen as filename separators by cat.

cat `ls -Qrt *.txt`

I tried -Q to put entry names in double quotes.

cat `ls -rt --quoting-style=escape *.txt`

I tried this and other variants of the quoting style.

Nothing that I've tried works. Either the blanks are treated as filename separators by cat, or the entire list of filenames is treated as one (invalid) argument. Please advise!

Answer 1

Using --quoting-style is a good start. The trick is in parsing the quoted file names. Backticks are simply not up to the job. We're going to have to be super explicit about parsing the escape sequences.

First, we need to pick a quoting style. Let's see how the various algorithms handle a crazy file name like "foo 'bar'\tbaz\nquux". That's a file name containing actual single and double quotes, plus a space, tab, and newline to boot. If you're wondering: yes, these are all legal, albeit unusual.

$ for style in literal shell shell-always shell-escape shell-escape-always c c-maybe escape locale clocale; do printf '%-20s <%s>\n' "$style" "$(ls --quoting-style="$style" '"foo '\''bar'\'''$'\t''baz '$'\n''quux"')"; done
literal              <"foo 'bar'    baz 
quux">
shell                <'"foo '\''bar'\'' baz 
quux"'>
shell-always         <'"foo '\''bar'\'' baz 
quux"'>
shell-escape         <'"foo '\''bar'\'''$'\t''baz '$'\n''quux"'>
shell-escape-always  <'"foo '\''bar'\'''$'\t''baz '$'\n''quux"'>
c                    <"\"foo 'bar'\tbaz \nquux\"">
c-maybe              <"\"foo 'bar'\tbaz \nquux\"">
escape               <"foo\ 'bar'\tbaz\ \nquux">
locale               <‘"foo 'bar'\tbaz \nquux"’>
clocale              <‘"foo 'bar'\tbaz \nquux"’>

The ones that actually span two lines are no good, so literal, shell, and shell-always are out. Smart quotes aren't helpful, so locale and clocale are out. Here's what's left:

shell-escape         <'"foo '\''bar'\'''$'\t''baz '$'\n''quux"'>
shell-escape-always  <'"foo '\''bar'\'''$'\t''baz '$'\n''quux"'>
c                    <"\"foo 'bar'\tbaz \nquux\"">
c-maybe              <"\"foo 'bar'\tbaz \nquux\"">
escape               <"foo\ 'bar'\tbaz\ \nquux">

Which of these can we work with? Well, we're in a shell script. Let's use shell-escape.

There will be one file name per line. We can use a while read loop to read a line at a time. We'll also need IFS= and -r to disable any special character handling. A standard line processing loop looks like this:

while IFS= read -r line; do ... done < file

That "file" at the end is supposed to be a file name, but we don't want to read from a file, we want to read from the ls command. Let's use <(...) process substitution to swap in a command where a file name is expected.

while IFS= read -r line; do
    # process each line
done < <(ls -rt --quoting-style=shell-escape *.txt)

Now we need to convert each line with all the quoted characters into a usable file name. We can use eval to have the shell interpret all the escape sequences. (I almost always warn against using eval but this is a rare situation where it's okay.)

while IFS= read -r line; do
    eval "file=$line"
done < <(ls -rt --quoting-style=shell-escape *.txt)

If you wanted to work one file at a time we'd be done. But you want to pass all the file names at once to another command. To get to the finish line, the last step is to build an array with all the file names.

files=()

while IFS= read -r line; do
    eval "files+=($line)"
done < <(ls -rt --quoting-style=shell-escape *.txt)

cat "${files[@]}"

There we go. It's not pretty. It's not elegant. But it's safe.

Answer 2

Does this do what you want?

for i in $(ls -rt *.txt); do echo "FILE: $i"; cat "$i"; done