0

Verifiying checksums inside a file usually works like this using Linux and md5sum.

    md5sum -c file.md5

However, using it like this on a Windows file reveals errors, that took me a long while to figure out. The first error was easy: Windows uses backslash instead of slash and different line endings. This can be fixed!

    sed 's/\r$//' file.md5 | sed 's_\\_/_g' | md5sum -c

The first sed takes care of the line endings and the second one of the backslashes.
And while this does solve the major issue, it always ignores the first line of the file, saying unhelpfully at the very end:

md5sum: WARNING: 1 line is improperly formatted

Switching lines around doesn't change this. The first line of the file is always ignored.

4
  • What's actually in that 1st line? Run hexdump -C foo.md5 and check. Commented Apr 29, 2021 at 12:30
  • Weird binary characters, see my answer below. Only it took me way longer than you to think of that. Should have asked here in the first place and save me weeks of trouble.
    – Ocean
    Commented Apr 29, 2021 at 12:32
  • Do they look like a byte order mark? Commented Apr 29, 2021 at 13:09
  • Yes. Using :%!xxd in vim reveals that the file starts with "efbb bf39 6631", while the actual checksum starts with 9f1 which in hex is "396631".
    – Ocean
    Commented Apr 29, 2021 at 16:47

1 Answer 1

0

I finally found the answer after noticing that copying the entire content into a new file does not produce this error.

So it turns out that in Windows the file has some binary characters in front that are not shown in a text editor, but are interpreted by md5sum.

So the solution is then to get rid of these:

 sed 's/\r$//' file.md5 | sed 's_\\_/_g' | sed -e '1s/^.//' | md5sum -c

The third sed removes the first binary blob, finally enabling me to check md5 files generated on Windows!

And because this has been a major ordeal that I didn't find much help online for, I've decided to share it with the world.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .