How to cat files together, adding missing newlines at end of some files

Question

I have a bunch of .text files, most of which end with the standard nl.

A couple don't have any terminator at end. The last physical byte is (generally) an alphameric character.

I was using cat *.text >| /tmp/joined.text, but then noticed a couple of places in joined.text where the first line of a file appeared at the end of the last line of a previous file. Inspecting the previous file, I saw there wasn't a line terminator -- concatenation explained.

That raised the question, what's the easiest way to concatenate, sticking in the missing newline? What about these options?

1. A solution that might effectively add a blank line to some input files. For me, that's not a problem as the processing of joined.text can handle it.
2. A solution that adds the cr/fl only to files that do not already end that way.

Answer 1

Another command that can add newlines if needed is awk, so:

awk 1 ./*.txt

The 1 here is the simplest way to get a true condition in awk, which works for this purpose since awk default action on true conditions is to print the input lines.

Answer 2

With some cut implementations like GNU cut, you can do:

cut -b 1- ./*.text > output

as it will add the missing newline if missing.

Answer 3

This handy Perl one-liner can do the job of adding the missing newline only if not already there:

perl -lpe '' ./*.text > output

Answer 4

You could use this:

grep -h "" ./*.txt

-h will remove the filename printout

Answer 5

The first approach that comes to mind is to loop over the files and just print their contents with an appended newline:

for f in *text; do
    printf '%s\n' "$(cat < "$f")"
done > /tmp/joined.text

The $() will strip any already existing newline characters so this will result in just one \n at the end of each file.