Running Checks on all files in all subdirectories

Question

I have looked through MANY questions asked and none of the solutions are working for me. I want to run an md5sum check on all files in all subdirectories from that specified in cd [directory]. Example:

cd [directory]

for f in *; do 
    if [[ -f "$f" ]]; then
        (md5sum -- "$f" > "md5_${f}") 
    fi
done

Thanks to a commentor, I understood my initial error which took my command as a directory instead of running it.

The issue now is that it runs successfully in the first directory it is looking at, but does not run correctly in sub-directories.

Per comment

find . - type f ! - name /'md5_*' -execdir sh -c /
    for f; do
         b="$(basename "$f")";
         [ ! -e "md5_$b" ] && md5sum "$b" > "md5$b"
    done

output find: paths must precede expression: - Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]

Answer 1

This can be done with one find command:

find . -type f ! -name 'md5_*' -execdir sh -c \
  'for f; do
     b="$(basename "$f")"
     [ ! -e "md5_$b" ] || [ "$b" -nt "md5_$b" ] && md5sum "$b" >  "md5_$b"
   done' sh {} +

The ! -name 'md5_* option excludes md5_* files, and the -execdir option causes find to cd to each directory containing files, and then running the sh -c '....' command in it. It runs as few sh -c ... commands per directory as possible (usually only one, depending on the maximum command line limit on your system - typically 2MB or more these days).

The [ ! -e "md5_$b" ] || [ "$b" -nt "md5_$b" ] test in the shell script ensures that md5sum is only run for files where either the md5_ file does not exist or if the file is newer than its md5_ file.

---

If you wanted to have only one md5sums.txt file in each directory, it would be simpler:

find . -type f ! -name 'md5sums.txt' -execdir sh -c \
  'md5sum "$@" >md5sums.txt' sh {} +

and even simpler if you wanted just one md5sums.txt file listing all files in all subdirectories:

rm -f md5sums.txt
find . -type f -exec md5sum {} + >> md5sums.txt

---

This version should work with busybox's version of find, and maybe other minimalist versions:

find . -type f ! -name 'md5_*' -exec sh -c \
  'for f; do
     cd "$(dirname "$f")"
     b="$(basename "$f")"
     [ ! -e "md5_$b" ] || [ "$b" -nt "md5_$b" ] && md5sum "$b" >  "md5_$b"
     cd - >/dev/null
   done' sh {} +

It'll run a bit slower because it has to do the directory changing in the shell script...and it has to do cd twice for every file, rather than just once per directory.

I would have used pushd and popd to change directories, but if you're running busybox find or similar, you're probably also running a minimalist sh like dash.

Answer 2

This will find all files in a /dir and run md5sum /dir/file

find /path/to/directory -type f -print0 | xargs -0I{} md5sum {}

Answer 3

This looks like a job for find:

find /directory -type f -exec sh -c 'a={}; md5sum -- $a > $a.md5' \;