Based on the versions you provide in your comment, it sounds like what @MattCaswell indicated should be the indeed the culprit: OpenSSL changed the default digest from md5 to sha-256 in version 1.1.0, see in the History text of the documentation. Based on this, your 1.0.2k version must be using the md5 digest.
Try passing -md md5
as an argument to your decrypting command to decrypt old files, like this:
openssl aes-256-cbc -d -salt -md md5 -in ~/my_file.txt.enc -pass pass:[redacted] > my_file.txt
If you are encrypting new files, it is better to add -md sha-256
in the encryption command instead, to keep it more portable, since new versions use this by default (and sha-256 is presumably more secure than the older md5 anyway).
Update
Ignore the previous update (striked out below), it turns out both openssl 1.0.2 and LibreSSL 3.3.6 use the old key derivation function by default anyway, nevertheless it is a good idea to add -pbkdf2
in both your encryption and decryption commands anyway, in order to use a better key derivation function.
Regarding the actual problem, I tried the exact versions you mentioned, and both these combinations work (encryption in 1.0.2k-fips, decryption in LibreSSL 3.3.6):
Using md5 (no extra argument in 1.0.2k-fips, -md md5
in LibreSSL):
cat test.txt | openssl aes-256-cbc -salt -out test.enc -pass pass:passwd
openssl enc -d -aes-256-cbc -pass pass:passwd -md md5 -in ~/test.enc
Using sha256 (-md sha256
in openssl 1.0.2k-fips, no argument in LibreSSL):
cat test.txt | openssl aes-256-cbc -salt -md sha256 -out test2.enc -pass pass:passwd
openssl enc -d -aes-256-cbc -pass pass:passwd -in ~/test2.enc
So it looks like the md md5
option should actually help. Could you double check your commands, maybe you mistyped the password or checking the wrong files? Of course it also wouldn't harm to use the -md xxx
option in both the encryption and decryption anyway, just to be on the safe side.
Based on your comment that -md md5
did not help, it looks that this is not the only problem. Additionally, it looks like the newer versions are using the PBKDF key derivation function, which older versions (like 1.0.2) were not using.
The best approach will be of course, if you have the possibility to update your amazon linux to a newer version, and use the safer PBKDF algorithm.
If this is not possible, you could use an actual key and iv for encryption and decryption instead of a password (which is transformed to a key and iv by the key derivation function, which is different in new and old openssl). Assuming you have a good random source, instead of passing -pass pass:[redacted]
, use -K <64 random hex characters>
and -iv <32 random hex characters>
, and pass the same key and iv in both encryption and decryption.
That is of course assuming you want a command to encrypt in amazon linux and decrypt in osx. If you already have the encrypted files and look for a way to decrypt them in OSX, you need to find a way to extract the actual key and iv from the old password (not sure what algorithm was used in old version of openssh, but I think it was some kind of simple hash, will require some research).
openssl version
on each machine. Most likely there are differences in the default digest used to generate the key from the password due to the version of openssl used.