Can Archive Manager distinguish between wrong password and corrupted file?

Question

I have two old encrypted RAR files. I know their passwords, but something interesting happens:

When I try to open each file in File Roller (Ubuntu's default archive manager) and enter the wrong password, it shows the message "Wrong password", as expected.
When I try to open the files in File Roller with the correct password, it shows the error "An error occurred while loading the archive / Could not find the volume a.r00". (Note: the files are not split in parts, they are single complete RAR files.)
When I try to open the files with other tools, I always get errors like "Checksum error in the encrypted file a.rar. Corrupt file or wrong password.", whether I enter the right or wrong password. I have tried Unrar and 7-zip.

From point 3, I'd assume I actually forgot the password, but point 2 is leaving me very puzzled because it seems, from the message, that the password is correct but the file fails to be extracted for some other reason (it expects there's a second part of the archive, when there is not).

There are many questions around related to recovery of corrupt RAR files, so my question here is not about that. What I'd like to understand is: how is it possible that File Roller has a different error message for when the password is correct while the other tools have a general message that either the password is incorrect or the file is corrupt?

My final goal is to find out whether I have the wrong password or the file is corrput.

I am not completely familiar with rars encryption but I am with the crypto aspect of it. #3 is probably the most accurate. The archive tool has no way to know weather the key is wrong or if the data is corrupt. This is the nature of crypto. You run the encrypted data through an api call with the key and bytes pop out. If those bytes have a valid checksum (or whatever hashing they use) then the file is good. If not.. the devs that wrote those errors don't know either. You don't want your crypto to let someone know which it is. "Wrong pwd" would mean keep the dictionary attack going. — Señor CMasMas, Commented Jun 14, 2022 at 5:13
@SeñorCMasMas Yes, that makes sense. But what an extraordinary coincidence that the error changes exactly and only when I enter the correct password!? The passwords are complex and distinct. Since the files were encrypted with File Roller as well, I thought of the possibility that it could have some extra information to check for password correctness. — Raphael, Commented Jun 14, 2022 at 12:52

Mofi · Accepted Answer · 2022-07-03 13:06:17Z

There must be taken into account that WinRAR.exe (Windows GUI version) and Rar.exe (Windows console version) support two different types of encryption on creating a RAR archive:

file data encryption and
encryption of really all data including file names, sizes, attributes, comments and other blocks.

In first case a RAR archive file can be opened to get the list of file names, their original file sizes, their packed sizes, the last modification dates, etc. without the need of the password which means with decryption of any data.

In second case the password is even necessary to get the list of files and folders inside the archive file and all the other metadata.

There must be also taken into account if a RAR archive file is in RAR format 4 or in RAR format 5. The reason is explained below.

I created on Windows three RAR archive files using licensed Rar.exe.

The RAR file FileDataEncrypted.rar with just file data encrypted using the command line:

"%ProgramFiles%\WinRAR\Rar.exe" a -cfg- -ep -idq -m5 -ma5 -mt1 -psecret FileDataEncrypted.rar "%ProgramFiles%\WinRAR\Rar.txt"

The RAR file FullyEncrypted.rar with all data encrypted using the command line:

"%ProgramFiles%\WinRAR\Rar.exe" a -cfg- -ep -idq -m5 -ma5 -mt1 "-hpmore secret" FullyEncrypted.rar "%ProgramFiles%\WinRAR\Rar.txt"

The corrupted RAR file CorruptedArchive.rar with using first the command line:
```
copy FileDataEncrypted.rar CorruptedArchive.rar
```
Then I opened the file CorruptedArchive.rar in text editor UltraEdit in hex edit mode and modified 16 bytes at hexadecimal offset 2000 to hexadecimal value FF and saved the modified file.

See the text file Rar.txt for the meaning of the uses switches.

The three RAR archive files are all in RAR format 5 (switch -ma5).

Then I executed in the Windows command prompt window a batch file with the following command lines:

@echo off
"%ProgramFiles%\WinRAR\UnRAR.exe" x -idcdp -o+ -psecret FileDataEncrypted.rar
echo Exit code of FileDataEncrypted.rar extraction with correct password is: %ERRORLEVEL%
"%ProgramFiles%\WinRAR\UnRAR.exe" x -idcdp -o+ -pwrong FileDataEncrypted.rar
echo Exit code of FileDataEncrypted.rar extraction with wrong password is: %ERRORLEVEL%
"%ProgramFiles%\WinRAR\UnRAR.exe" x -idcdp -o+ "-hpmore secret" FullyEncrypted.rar
echo Exit code of FullyEncrypted.rar extraction with correct password is: %ERRORLEVEL%
"%ProgramFiles%\WinRAR\UnRAR.exe" x -idcdp -o+ "-hpwrong secret" FullyEncrypted.rar
echo Exit code of FullyEncrypted.rar extraction with wrong password is: %ERRORLEVEL%
"%ProgramFiles%\WinRAR\UnRAR.exe" x -idcdp -o+ -psecret CorruptedArchive.rar
echo Exit code of CorruptedArchive.rar extraction with correct password is: %ERRORLEVEL%
"%ProgramFiles%\WinRAR\UnRAR.exe" x -idcdp -o+ -psecret CorruptedArchive.rar
echo Exit code of CorruptedArchive.rar extraction with wrong password is: %ERRORLEVEL%

The output was:

Extracting from FileDataEncrypted.rar

Extracting  Rar.txt                                                   OK
Exit code of FileDataEncrypted.rar extraction with correct password is: 0

Extracting from FileDataEncrypted.rar

Incorrect password for Rar.txt
Exit code of FileDataEncrypted.rar extraction with wrong password is: 11

Extracting from FullyEncrypted.rar

Extracting  Rar.txt                                                   OK
Exit code of FullyEncrypted.rar extraction with correct password is: 0

Incorrect password for FullyEncrypted.rar
Exit code of FullyEncrypted.rar extraction with wrong password is: 11

Extracting from CorruptedArchive.rar

Extracting  Rar.txt
Rar.txt              - checksum error
Exit code of CorruptedArchive.rar extraction with correct password is: 3

Extracting from CorruptedArchive.rar

Extracting  Rar.txt
Rar.txt              - checksum error
Exit code of CorruptedArchive.rar extraction with wrong password is: 3

The text file Rar.txt explains the exit values at bottom with:

   Code   Description   

    0     Successful operation.
    1     Non fatal error(s) occurred.
    2     A fatal error occurred.
    3     Invalid checksum. Data is damaged.
    4     Attempt to modify an archive locked by 'k' command.
    5     Write error.
    6     File open error.
    7     Wrong command line option.
    8     Not enough memory.
    9     File create error
   10     No files matching the specified mask and options were found.
   11     Wrong password.
   12     Read error.
  255     User stopped the process.

The usage of the wrong password on the command line for the two valid RAR archives results in an output of the error message Incorrect password for ... of the archived file inside the RAR archive file (first case) or the entire RAR archive file (second case) and an exit with the value 11.

The error message is always checksum error and the exit code is always 3 on RAR archive file being corrupted independent on using correct or wrong password.

The RAR archive format 5 makes it possible for the RAR extraction code to determine if the password for decryption is wrong or the archive file itself is corrupted.

I created next the three RAR archive files once again, but using the switch -ma4 instead of -ma5 to create two valid and a corrupted RAR archive files in RAR archive format 4.

Next I executed the same batch file for extraction as before. The output was this time:

Extracting from FileDataEncrypted.rar

Extracting  Rar.txt                                                   OK
Exit code of FileDataEncrypted.rar extraction with correct password is: 0

Extracting from FileDataEncrypted.rar

Extracting  Rar.txt
Checksum error in the encrypted file Rar.txt. Corrupt file or wrong password.
Exit code of FileDataEncrypted.rar extraction with wrong password is: 3

Extracting from FullyEncrypted.rar

Extracting  Rar.txt                                                   OK
Exit code of FullyEncrypted.rar extraction with correct password is: 0

Checksum error in the encrypted file FullyEncrypted.rar. Corrupt file or wrong password.
No files to extract
Exit code of FullyEncrypted.rar extraction with wrong password is: 3

Extracting from CorruptedArchive.rar

Extracting  Rar.txt
Checksum error in the encrypted file Rar.txt. Corrupt file or wrong password.
Exit code of CorruptedArchive.rar extraction with correct password is: 3

Extracting from CorruptedArchive.rar

Extracting  Rar.txt
Checksum error in the encrypted file Rar.txt. Corrupt file or wrong password.
Exit code of CorruptedArchive.rar extraction with wrong password is: 3

It can be seen on the error messages as well as on always same exit code 3 that it is not possible for the extraction code to determine if the password is wrong or there is a corruption of the archive file data on RAR archive file being in format 4.

The currently latest UnRAR C++ source code can be downloaded from the WinRAR and RAR archiver addons page as it is freeware, see license.txt in the source archive file. There are also UnRAR binaries available for download many platforms on same page.

UnRAR and 7-Zip used on Ubuntu output the same error message as UnRAR.exe and of course also Rar.exe on Windows on extraction of a RAR archive file in RAR format 4 being either corrupted or on which the wrong password is entered by the user. It is not possible to find out the cause of the checksum error as it can be seen above. UnRAR and 7-Zip are compiled with using the UnRAR source code.

The Ubuntu documentation page for File Roller in chapter Supported Formats in the section Not installed by default in Ubuntu the information that format RAR is supported only by File Roller with using unrar (non-free) or unrar-free package to extract .rar archives. So it depends on how old is the extra installed unrar package for extraction of RAR archive file and how new is the RAR archive file.