Skip to main content
added 1741 characters in body
Source Link
Gabriel Staples
  • 2.1k
  • 2
  • 27
  • 39

Get the latest version of my hex2xxdhex function in my eRCaGuy_dotfiles repo here: .bash_useful_functions. Copy and paste that function to the bottom of your ~/.bashrc file. Then, re-source your ~/.bashrc file with . ~/.bashrc. Finally, use my hex2xxdhex function like this:

# pass multiple `.hex` files to convert to `.bin`, `.xxd.hex`, and
# `.xxd_short.hex` files
hex2xxdhex path/to/myfile1.hex path/to/myfile2.hex
# then compare the two output ".xxd.hex" or ".xxd_short.hex" files with `meld`
meld path/to/myfile1.xxd_short.hex path/to/myfile2.xxd_short.hex
meld path/to/myfile1.xxd.hex path/to/myfile2.xxd.hex

In the above comparison of the two *.xxd*.hex files, you'll see all of the hex chars, followed by the binary/ASCII chars in a column on the right-hand-side, allowing you to more easily identify hex file differences between the two files.

The .xxd_short.hex files are simply the same as the .xxd.hex files, except with all lines containing only zeros removed. This way, if your hex file places portions of your firmware at drastically different address locations, all of the padded zeros between the two address locations are removed.

If your initial .hex file is 3.5 MB, your .bin file might be 45 MB, your .xxd.hex file might be 200 MB, and your .xxd_short.hex file (with all rows of pure zeros) might be 5 MB. meld can compare two 5 MB files just fine, but it struggles with 200 MB files. That's why I generate the .xxd_short.hex version too.

Other options:

Get the latest version of my hex2xxdhex function in my eRCaGuy_dotfiles repo here: .bash_useful_functions. Copy and paste that function to the bottom of your ~/.bashrc file. Then, re-source your ~/.bashrc file with . ~/.bashrc. Finally, use my hex2xxdhex function like this:

# pass multiple `.hex` files to convert to `.bin`, `.xxd.hex`, and
# `.xxd_short.hex` files
hex2xxdhex path/to/myfile1.hex path/to/myfile2.hex
# then compare the two output ".xxd.hex" or ".xxd_short.hex" files with `meld`
meld path/to/myfile1.xxd_short.hex path/to/myfile2.xxd_short.hex
meld path/to/myfile1.xxd.hex path/to/myfile2.xxd.hex

In the above comparison of the two *.xxd*.hex files, you'll see all of the hex chars, followed by the binary/ASCII chars in a column on the right-hand-side, allowing you to more easily identify hex file differences between the two files.

The .xxd_short.hex files are simply the same as the .xxd.hex files, except with all lines containing only zeros removed. This way, if your hex file places portions of your firmware at drastically different address locations, all of the padded zeros between the two address locations are removed.

If your initial .hex file is 3.5 MB, your .bin file might be 45 MB, your .xxd.hex file might be 200 MB, and your .xxd_short.hex file (with all rows of pure zeros) might be 5 MB. meld can compare two 5 MB files just fine, but it struggles with 200 MB files. That's why I generate the .xxd_short.hex version too.

Other options:

added 274 characters in body
Source Link
Gabriel Staples
  • 2.1k
  • 2
  • 27
  • 39
added 362 characters in body
Source Link
Gabriel Staples
  • 2.1k
  • 2
  • 27
  • 39

If you really need to use non-GUI tools, such as through an ssh session, here are some more options:. Alternatively, you could just scp the file back to your local machine over ssh, and then use meld as described above.

Pure CLI tools for binary comparison:

  1. ReferUse diff: refer back to @kenorb's answer above]. Here are some of my own spins on those commands which I think are more useful:

    # For short output
    diff -u --color=always <(xxd file1.bin) <(xxd file2.bin)
    
    # If your output is really long, pipe to `less -RFX`, like `git` does
    diff -u --color=always <(xxd file1.bin) <(xxd file2.bin) | less -RFX
    

    Example run and output:

    eRCaGuy_hello_world/c$ diff -u --color=always <(xxd file1.bin) <(xxd file2.bin) 
    --- /dev/fd/63  2023-06-21 23:16:51.649582608 -0700
    +++ /dev/fd/62  2023-06-21 23:16:51.649582608 -0700
    @@ -53,8 +53,8 @@
     00000340: 0500 0000 474e 5500 0200 00c0 0400 0000  ....GNU.........
     00000350: 0300 0000 0000 0000 0280 00c0 0400 0000  ................
     00000360: 0100 0000 0000 0000 0400 0000 1400 0000  ................
    -00000370: 0300 0000 474e 5500 5a84 e8c3 58b3 e81e  ....GNU.Z...X...
    -00000380: d731 5fd2 0c0a 1aaf be99 ec8b 0400 0000  .1_.............
    +00000370: 0300 0000 474e 5500 23ea bebd 1106 9feb  ....GNU.#.......
    +00000380: 14a4 4f55 e90b d6b0 bf57 e851 0400 0000  ..OU.....W.Q....
     00000390: 1000 0000 0100 0000 474e 5500 0000 0000  ........GNU.....
     000003a0: 0300 0000 0200 0000 0000 0000 0000 0000  ................
     000003b0: 0200 0000 0600 0000 0100 0000 0600 0000  ................
    @@ -510,7 +510,7 @@
     00001fd0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
     00001fe0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
     00001ff0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
    -00002000: 0100 0200 4865 6c6c 6f20 576f 726c 642e  ....Hello World.
    +00002000: 0100 0200 4865 4c6c 6f20 576f 526c 642e  ....HeLlo WoRld.
     00002010: 0a00 0000 011b 033b 3000 0000 0500 0000  .......;0.......
     00002020: 0cf0 ffff 6400 0000 2cf0 ffff 8c00 0000  ....d...,.......
     00002030: 3cf0 ffff a400 0000 4cf0 ffff bc00 0000  <.......L.......
    

    Here's a screenshot so you can more easily see the colored changes:

    enter image description here

  2. UseUse vbindiff: I first learned about this tool here: vbindiffHow-To Geek: How to Compare Binary Files on Linux.

    Here is how to install and use it:

    # 1. Install it
    sudo apt update
    sudo apt install vbindiff
    
    # 2. use it; note that we do NOT need `xxd` here
    vbindiff file1.bin file2.bin
    
    # You should study the manual/help pages too. It is very useful:
    vbindiff -h
    man vbindiff
    

    Here's what it looks like on the 2nd change shown above. As you can see, it has highlighted the exact character differences in red. It's a pretty nice tool:

    enter image description here

    Navigation:

    1. See man vbindiff for details. It's a short and easy manual.
    2. Press Space or Enter to go to the next difference. I hit Space twice to go to the 2nd difference in the screenshot above.
    3. Press Q or Esc to quit.
    4. You can scroll with the mouse scroll wheel, arrow keys, PageUp/Down keys, etc.
    5. Press T to toggle on scrolling only the top window.
    6. Press B to toggle on scrolling only the bottom window.
    7. Press F to find.

    The navigation is really pretty limited. I don't see a way to go back up and find a previous change. Just quit and start again.

If you really need to use non-GUI tools, such as through an ssh session, here are some more options:

  1. Refer back to @kenorb's answer above]. Here are some of my own spins on those commands which I think are more useful:

    # For short output
    diff -u --color=always <(xxd file1.bin) <(xxd file2.bin)
    
    # If your output is really long, pipe to `less -RFX`, like `git` does
    diff -u --color=always <(xxd file1.bin) <(xxd file2.bin) | less -RFX
    

    Example run and output:

    eRCaGuy_hello_world/c$ diff -u --color=always <(xxd file1.bin) <(xxd file2.bin) 
    --- /dev/fd/63  2023-06-21 23:16:51.649582608 -0700
    +++ /dev/fd/62  2023-06-21 23:16:51.649582608 -0700
    @@ -53,8 +53,8 @@
     00000340: 0500 0000 474e 5500 0200 00c0 0400 0000  ....GNU.........
     00000350: 0300 0000 0000 0000 0280 00c0 0400 0000  ................
     00000360: 0100 0000 0000 0000 0400 0000 1400 0000  ................
    -00000370: 0300 0000 474e 5500 5a84 e8c3 58b3 e81e  ....GNU.Z...X...
    -00000380: d731 5fd2 0c0a 1aaf be99 ec8b 0400 0000  .1_.............
    +00000370: 0300 0000 474e 5500 23ea bebd 1106 9feb  ....GNU.#.......
    +00000380: 14a4 4f55 e90b d6b0 bf57 e851 0400 0000  ..OU.....W.Q....
     00000390: 1000 0000 0100 0000 474e 5500 0000 0000  ........GNU.....
     000003a0: 0300 0000 0200 0000 0000 0000 0000 0000  ................
     000003b0: 0200 0000 0600 0000 0100 0000 0600 0000  ................
    @@ -510,7 +510,7 @@
     00001fd0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
     00001fe0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
     00001ff0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
    -00002000: 0100 0200 4865 6c6c 6f20 576f 726c 642e  ....Hello World.
    +00002000: 0100 0200 4865 4c6c 6f20 576f 526c 642e  ....HeLlo WoRld.
     00002010: 0a00 0000 011b 033b 3000 0000 0500 0000  .......;0.......
     00002020: 0cf0 ffff 6400 0000 2cf0 ffff 8c00 0000  ....d...,.......
     00002030: 3cf0 ffff a400 0000 4cf0 ffff bc00 0000  <.......L.......
    

    Here's a screenshot so you can more easily see the colored changes:

    enter image description here

  2. Use vbindiff:

    # 1. Install it
    sudo apt update
    sudo apt install vbindiff
    
    # 2. use it; note that we do NOT need `xxd` here
    vbindiff file1.bin file2.bin
    
    # You should study the manual/help pages too. It is very useful:
    vbindiff -h
    man vbindiff
    

    Here's what it looks like on the 2nd change shown above. As you can see, it has highlighted the exact character differences in red. It's a pretty nice tool:

    enter image description here

    Navigation:

    1. See man vbindiff for details. It's a short and easy manual.
    2. Press Space or Enter to go to the next difference. I hit Space twice to go to the 2nd difference in the screenshot above.
    3. Press Q or Esc to quit.
    4. You can scroll with the mouse scroll wheel, arrow keys, PageUp/Down keys, etc.
    5. Press T to toggle on scrolling only the top window.
    6. Press B to toggle on scrolling only the bottom window.
    7. Press F to find.

    The navigation is really pretty limited. I don't see a way to go back up and find a previous change. Just quit and start again.

If you really need to use non-GUI tools, such as through an ssh session, here are some more options. Alternatively, you could just scp the file back to your local machine over ssh, and then use meld as described above.

Pure CLI tools for binary comparison:

  1. Use diff: refer back to @kenorb's answer above]. Here are some of my own spins on those commands which I think are more useful:

    # For short output
    diff -u --color=always <(xxd file1.bin) <(xxd file2.bin)
    
    # If your output is really long, pipe to `less -RFX`, like `git` does
    diff -u --color=always <(xxd file1.bin) <(xxd file2.bin) | less -RFX
    

    Example run and output:

    eRCaGuy_hello_world/c$ diff -u --color=always <(xxd file1.bin) <(xxd file2.bin) 
    --- /dev/fd/63  2023-06-21 23:16:51.649582608 -0700
    +++ /dev/fd/62  2023-06-21 23:16:51.649582608 -0700
    @@ -53,8 +53,8 @@
     00000340: 0500 0000 474e 5500 0200 00c0 0400 0000  ....GNU.........
     00000350: 0300 0000 0000 0000 0280 00c0 0400 0000  ................
     00000360: 0100 0000 0000 0000 0400 0000 1400 0000  ................
    -00000370: 0300 0000 474e 5500 5a84 e8c3 58b3 e81e  ....GNU.Z...X...
    -00000380: d731 5fd2 0c0a 1aaf be99 ec8b 0400 0000  .1_.............
    +00000370: 0300 0000 474e 5500 23ea bebd 1106 9feb  ....GNU.#.......
    +00000380: 14a4 4f55 e90b d6b0 bf57 e851 0400 0000  ..OU.....W.Q....
     00000390: 1000 0000 0100 0000 474e 5500 0000 0000  ........GNU.....
     000003a0: 0300 0000 0200 0000 0000 0000 0000 0000  ................
     000003b0: 0200 0000 0600 0000 0100 0000 0600 0000  ................
    @@ -510,7 +510,7 @@
     00001fd0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
     00001fe0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
     00001ff0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
    -00002000: 0100 0200 4865 6c6c 6f20 576f 726c 642e  ....Hello World.
    +00002000: 0100 0200 4865 4c6c 6f20 576f 526c 642e  ....HeLlo WoRld.
     00002010: 0a00 0000 011b 033b 3000 0000 0500 0000  .......;0.......
     00002020: 0cf0 ffff 6400 0000 2cf0 ffff 8c00 0000  ....d...,.......
     00002030: 3cf0 ffff a400 0000 4cf0 ffff bc00 0000  <.......L.......
    

    Here's a screenshot so you can more easily see the colored changes:

    enter image description here

  2. Use vbindiff: I first learned about this tool here: How-To Geek: How to Compare Binary Files on Linux.

    Here is how to install and use it:

    # 1. Install it
    sudo apt update
    sudo apt install vbindiff
    
    # 2. use it; note that we do NOT need `xxd` here
    vbindiff file1.bin file2.bin
    
    # You should study the manual/help pages too. It is very useful:
    vbindiff -h
    man vbindiff
    

    Here's what it looks like on the 2nd change shown above. As you can see, it has highlighted the exact character differences in red. It's a pretty nice tool:

    enter image description here

    Navigation:

    1. See man vbindiff for details. It's a short and easy manual.
    2. Press Space or Enter to go to the next difference. I hit Space twice to go to the 2nd difference in the screenshot above.
    3. Press Q or Esc to quit.
    4. You can scroll with the mouse scroll wheel, arrow keys, PageUp/Down keys, etc.
    5. Press T to toggle on scrolling only the top window.
    6. Press B to toggle on scrolling only the bottom window.
    7. Press F to find.

    The navigation is really pretty limited. I don't see a way to go back up and find a previous change. Just quit and start again.

added 87 characters in body
Source Link
Gabriel Staples
  • 2.1k
  • 2
  • 27
  • 39
Loading
added 47 characters in body
Source Link
Gabriel Staples
  • 2.1k
  • 2
  • 27
  • 39
Loading
Source Link
Gabriel Staples
  • 2.1k
  • 2
  • 27
  • 39
Loading