The Problem: Private Key Decryption Failure
Managing SSL/TLS certificates often requires converting or unlocking private keys. You might be stripping a passphrase, converting PEM to PKCS#12, or just checking a key's integrity. You enter your password, confident it's correct, only to have OpenSSL reject it with this message:
error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt
This error hits during the final validation step. OpenSSL successfully processed the data, but the resulting plaintext failed the padding check. Essentially, the math doesn't add up because the key was decrypted with the wrong parameters.
Analysis: Why Decryption Fails
While "bad decrypt" usually signals a typo, OpenSSL often fails for more nuanced technical reasons:
- Shell Character Escaping: If your password contains
$,!, or#, your terminal might be stripping or changing those characters before they ever reach OpenSSL. - Line Ending Corruption: Moving a key from Windows to Linux often introduces
\r\ncarriage returns. These hidden characters can break the Base64 parsing. - OpenSSL 3.0 Legacy Deprecation: Modern OS versions like Ubuntu 22.04 or macOS Sonoma use OpenSSL 3.x. This version disables older, "insecure" algorithms like MD5-based key derivation by default. If your key was generated years ago, OpenSSL 3.x simply won't try to decrypt it without a specific flag.
- Truncated Files: Even a single missing character in the Base64 block—often caused by a bad copy-paste—will trigger a padding failure.
Step-by-Step Fixes
1. Eliminate Terminal Interference
Stop typing your password directly into the prompt to avoid shell escaping issues. Instead, use a temporary file to pass the passphrase cleanly.
# Use printf to avoid adding a trailing newline
printf "your_password_here" > pass.txt
# Attempt to read the key using the password file
openssl rsa -in encrypted.key -passin file:pass.txt -text -noout
# Delete the file immediately
rm pass.txt
Using printf is safer than echo because it gives you exact control over whitespace. Even one extra space at the end of your password will cause a 06065064 error.
2. Enable the Legacy Provider (OpenSSL 3.x)
If you are using a recent Linux distribution or macOS and you're certain the password is correct, the issue is likely a legacy cipher. Older keys frequently use pbeWithMD5AndDES-CBC, which OpenSSL 3.x ignores for security reasons.
Force OpenSSL to load the legacy module with these flags:
# Unlock an older RSA key
openssl rsa -in encrypted.key -out decrypted.key -legacy -provider default -provider legacy
If you are working with a .p12 or .pfx bundle, the command looks like this:
openssl pkcs12 -in bundle.p12 -nodes -out certs.pem -legacy
3. Strip Hidden Windows Characters
Hidden carriage returns (CRLF) are a common culprit. Use tr to sanitize the file before trying to decrypt it again:
tr -d '\r' < encrypted.key > clean_encrypted.key
openssl rsa -in clean_encrypted.key -check
You can verify if these characters exist by running cat -A encrypted.key. If you see ^M at the end of lines, your file has Windows-style endings.
Permanent Fix: Modernize Your Encryption
If you had to use the -legacy flag, your key is a security liability. Modern tools like Nginx 1.24+ or newer Java environments may reject it. You should upgrade the encryption to AES-256 immediately.
First, decrypt the old key:
openssl rsa -in old_key.key -out temp.key -legacy
Then, re-encrypt it using modern standards:
openssl rsa -in temp.key -aes256 -out new_secure.key
This ensures compatibility with OpenSSL 3.x without needing special overrides in your production config.
Verification: Confirming the Key Integrity
Once you think you've fixed it, run a consistency check. This verifies the mathematical relationship between the private and public components of the key.
openssl rsa -in your_key.key -check -noout
If you see "RSA key ok", you're done. If the file is unencrypted, the header should read -----BEGIN RSA PRIVATE KEY----- without any secondary Proc-Type or DEK-Info headers inside the text.
Error Code Breakdown
Component
Description
`06065064`
The numeric identity for an EVP decryption failure.
`EVP_DecryptFinal_ex`
The function that checks if the data was decrypted cleanly.
`bad decrypt`
Confirmation that the password or algorithm didn't match the data.