I am using diff file1 file2 and getting memory exhausted exception. file1 and file 2 are 10 GB each.
I tried diff --speed-large-files file1 file2 but that didn't work.
How can I compare two files file1 and file2 and see what changed in file2 and write the changed content on file2 to another file?
Gnu diff requires to be able to load the input files into RAM.
According to this question, bdiff is available as part of the heirloom toolchest and doesn't require reading the files into RAM. You'll have to compile it yourself though as it doesn't seem to be available on most Linux distros.
Other questions on CSV files and large files have alternative solutions but none of them seem to quite do what you want.
It may be easier to break the files into parts and compare them in order. For instance, split out each 10 GB file to about 100 numbered hundred-megabyte files. Run the diff on the files in order, comparing outputA01 to outputB01, then outputA02 to outputB02, and son on. Related question : https://unix.stackexchange.com/questions/1588/break-a-large-file-into-smaller-pieces
