0

The source & destination paths have >100k different directories. I would like to compare the files between the source & destination without accounting for the differences between the directories.

Put another way, I want to match if the files in the source path are in the destination path i.e. using the files they contain, no matter where in the hierarchy they are.

If there are differences, these would be outputted to a text file for example.

I need to be able to use a cross platform solution e.g. Windows & Linux.

How can I achieve this?

Improve this question
2
  • I still don't understand your criteria, please edit your question and make an example (or two, or three). Do you want to compare directories, using the files they contain, no matter where in the hierarchy they are? Do you want to compare some subtree on both systems for the files they contain, no matter where are the files in the subtree? Or something else?
    – dirkt
    Commented Sep 12, 2021 at 9:41
  • @dirkt - I've updated the post. I want to be able to compare files no matter where they are in the hierarchy.
    – Motivated
    Commented Sep 12, 2021 at 18:55

3 Answers 3

Reset to default
0

Create a file that contains all the paths in the same format, on both systems.

Use e.g. find on Linux, and find in WSL or Cygwin on Windows. Or write a simple tool for that that you can install on both systems.

Then use diff to compare the two text files with all the paths.

Improve this answer
3
  • Do you mean create a file e.g. source.txt and destination.txt with the different paths for instance x:\source\path1\path2\filename1 and y:\destination\path3\path4\filename1 using find? If yes, will diff only show the file matches/differences irrespective of the paths?
    – Motivated
    Commented Sep 12, 2021 at 8:42
  • Do you want to compare the complete paths ("without accounting for the differences between the directories"), only the file names ("with accounting for the differences between the directories"), or something else? If you just want to compare file names, you can also make a list of file names only.
    – dirkt
    Commented Sep 12, 2021 at 8:53
  • The comparison is without accounting for differences between the hierarchical structure or depth. As long as the files match between the source & destination, it's a successful verification. If they differ, it returns an output highlighting the variance.
    – Motivated
    Commented Sep 12, 2021 at 8:58
0

You could use file hashes to compare the files and find them even if the folder structure has changed.

Many hash utilities can be found. For example, on Windows you could use hellzerg/hashing, with a review in the article Generate and compare file hashes with Hashing for Windows:

enter image description here

Improve this answer
4
  • I was considering generating a hash of the files on the source and comparing these with the hashes on the destination. Does the tool you suggested recursively generate hashes and compare these without accounting for the directories?
    – Motivated
    Commented Sep 12, 2021 at 18:58
  • The doc says it will do entire folders. I have no personal experience with this tool.
    – harrymc
    Commented Sep 12, 2021 at 19:37
  • As I need to perform these on both Windows & Linux, is there a tool that supports these?
    – Motivated
    Commented Sep 12, 2021 at 21:17
  • See for example this post.
    – harrymc
    Commented Sep 13, 2021 at 7:53
0

On Windows you can use Total Commander (Sync dirs option) and Samba connection to Linux/Mac:

  • you will see which files are missing and which are existing on one side
  • you will see files size
  • you will see dates and times of last modification for both sides
  • you can see left file and right file (text, image)
  • TC will allow you to configure comparison options for files attributes
  • in result you can synchronize both sides of the comparison

https://www.ghisler.com/download.htm

Double Commander also has these options: https://sourceforge.net/p/doublecmd/wiki/Download/

Improve this answer
1
  • It seems that it'll compare files within a given directory. I would like to compare files no matter where there are in the structure.
    – Motivated
    Commented Sep 12, 2021 at 18:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .