find log files recursively and print "INVALID" if no log found (bash)

Question

I have ~750 log directories I need to search for the latest log file available:

These are captured by the allLogs.txt file and each entry contains a wild card '*' so that ONLY the log with the latest date is grabbed (there can be many logs of the same name with different dates. e.g.

import_Asbeg_DA_Config3_*.log

import_test_deploy_*.log

import_Plus_DQ_Config1_*.log

e.g. I know this is not pretty but it works great to accomplish the goal in a loop.

for LATEST in `cat allLogs.txt` ;
do
    find -name "${LATEST}" -type f | sort -n | tail -1

done < infile > outfile

This finds ~350 records because all the directories might not have a log file.

"allLogs.txt" contains all the ~750 paths to the log file for that directory.

What I am trying to figure out how to run this command in my loop BUT, maintain consistency even if no log file is found. Perhaps by appending 'INVALID' to the records that DO NOT contain a log file. I've tried combinations of echo with the find command but no luck. Any ideas?

Answer 1

Based on your question, I think what you are really asking is to print out the latest file in the directory, if there is any file in the directory. If there are no files in any directory, then print out INVALID, or something similar.

If the above is a correct summarization, you can add an if clause to your loop that checks to see if there is only a single line outputted when you do a find on that directory. If there is only a single line, that is the directory. Hence, there are no files. You can then skip any further commands within the loop and continue to the next item in the list.

Here is the updated script:

#!/bin/bash

for LATEST in $(cat allLogs.txt); do
    if [[ $(find ${LATEST} -print | wc -l) -eq 1 ]]; then
        echo "${LATEST} has no files"
        continue
    fi
    find ${LATEST} -type f | sort -n | tail -1
done

exit 0

I did not want to just print INVALID as you would not know which directory was printing out INVALID.

I wasn't sure why you were redirecting stdin for the loop. I did not redirect stdout, but that is easy enough to add.

Answer 2

As in the answer you already have, I assume that you have many directories, each one containing zero or more files whose names are composed by a common part plus a number for ordering, and that the common parts of those names are listed in your file allLogs.txt.

You can use something like this to add a line to outfile for each name in allLogs.txt that has no matches found by find:

#!/bin/bash

while read -r LATEST; do
  find -name "${LATEST}" -type f | tee >( [ "$(wc -l)" = 0 ] && printf '%s\n' "${LATEST} INVALID" )  | sort -n | tail -1
done < allLogs.txt > outfile

tee and the process substitution will be executed even if find finds nothing, in which case a message is printed.

I changed the for loop into a while loop to let it manage file names with spaces. The input redirecton is changed accordingly, but this may not be what you want. Anyway, it is not relevant to your main question (how to add the "INVALID" notice).

Answer 3

All thanks tremendously with the help!

"printf" is my buddy!

This code worked to solve my initial... outfile maintains the line integrity of ~750... outfile is a little messy but nothing that cannot be processed with some awk, sed, etc.

 for LATEST in `cat ${TMPDIR}/AllLogs.txt` ;
    do
      printf "`echo ${LATEST}` `find -name "${LATEST}" -type f | sort -n | tail -1`\n"

    done < infile > outfile