Recovering from backups encrypted with an inaccessible key

TL;DR — You have backup files, but restore fails because the decrypting key is missing, disabled, deleted, or wrapped by a KMS you can no longer use. The fastest path is to identify whether the backup was encrypted with a customer-managed key you still control somewhere else; if the key material is truly gone and no plaintext copy or exported key exists, the backup is cryptographically unrecoverable. Reading time: ~6 min

The scenario

It is Tuesday at 3:40 PM. You are doing a restore drill or an actual recovery after a bad migration, and the backup object is right there in object storage with the expected size and checksum. But the restore job dies immediately with a decryption error: your old KMS key alias points nowhere, the IAM role lost decrypt permission, or the only person who had the GPG private key left six months ago. Product is asking for an ETA, and you need to know quickly whether this is a permissions problem, a key-state problem, or a hard stop with no cryptographic recovery path.

Symptoms

• Restore command fails before any data is read into the target system.
• Common stderr/messages:

gpg: decryption failed: No secret key

gpg: encrypted with 1 passphrase
gpg: decryption failed: Bad session key

Error decrypting keyset: permission denied

AccessDeniedException: User is not authorized to perform: kms:Decrypt on resource ...

InvalidCiphertextException: 

KMSInvalidStateException: arn:... is pending deletion

Cannot restore backup: failed to unwrap data key

• Backup file downloads fine, but piping through decrypt fails:

curl -fSL https://storage.example.com/db.dump.enc | age -d -i key.txt
age: error: no identity matched any of the recipients

• Logs show metadata references to a key ID/alias you do not recognize or can no longer resolve.
• In provider dashboards, the key exists but is disabled, scheduled for deletion, or the current role/user cannot use it.

Likely causes

| Key is disabled, pending deletion, or in the wrong region/account | Very common | ```bash
aws kms describe-key --key-id alias/your-key
``` |
| You have the wrong private key/passphrase for file-level encryption (GPG/age/OpenSSL) | Common | ```bash
gpg --list-secret-keys --keyid-format LONG
``` |
| Backup metadata points to an old alias or rotated key version you no longer retained | Common | ```bash
jq . backup-manifest.json
``` |
| The actual key material was deleted and never exported/escrowed | Less common, highest impact | ```bash
find /secure /etc /home -maxdepth 3 \( -name '*.asc' -o -name '*.gpg' -o -name 'age*key*' -o -name '*.p12' \) 2>/dev/null
``` |
| Corrupted or truncated encrypted backup mistaken for a key problem | Less common | ```bash
file backup.tar.enc && sha256sum backup.tar.enc
``` |

## Step-by-step diagnosis
1. Identify the encryption method and key reference.
   ```bash
file backup.enc
strings -a backup.enc | head -50
jq . backup-manifest.json 2>/dev/null || cat backup-manifest.json

This is your problem if you find explicit references like alias/prod-backup-key, a GPG recipient fingerprint, age-encryption.org/v1, or openssl enc. Jump to the matching fix section.

1. If the backup uses a cloud KMS envelope key, check key state first.

aws kms describe-key --key-id alias/prod-backup-key

   Problem indicators:
   ```json
   {"KeyMetadata":{"KeyState":"Disabled"}}

or:

{"KeyMetadata":{"KeyState":"PendingDeletion","Arn":"arn:aws:kms:us-east-1:..."}}

If disabled/pending deletion/wrong region-account, jump to Key is disabled, pending deletion, or in the wrong region/account.

1. Test decrypt authorization separately from the restore tool.

aws sts get-caller-identity aws kms decrypt --key-id alias/prod-backup-key --ciphertext-blob fileb://wrapped-key.bin --output text --query Plaintext >/dev/null

   This is your problem if you get:
   ```bash
   An error occurred (AccessDeniedException) when calling the Decrypt operation: User is not authorized to perform: kms:Decrypt

Jump to Missing decrypt permission to the KMS/HSM/key service.

1. For GPG backups, verify the secret key exists locally and matches the recipient.

   gpg --list-packets backup.sql.gpg | sed -n '1,20p'
   gpg --list-secret-keys --keyid-format LONG
   

   This is your problem if the recipient key ID in --list-packets is not present in --list-secret-keys, or decryption says No secret key. Jump to You have the wrong private key/passphrase for file-level encryption.

2. For age/OpenSSL backups, validate identities and headers.

   head -5 backup.tar.age
   age -d -i ./keys.txt backup.tar.age >/dev/null
   

   or:

   openssl enc -d -aes-256-cbc -pbkdf2 -in backup.enc -out /dev/null
   echo $?
   

   If age says no identity matched any of the recipients or OpenSSL returns bad decrypt, jump to You have the wrong private key/passphrase for file-level encryption.

3. Check whether the backup metadata references an alias/version that changed.

   jq '{key_id, key_alias, recipient, created_at, tool_version}' backup-manifest.json
   

   This is your problem if the manifest points to an alias that now targets a different key, or to a deleted key version. Jump to Backup metadata points to an old alias or rotated key version you no longer retained.

4. Rule out corruption before declaring the data unrecoverable.

   sha256sum -c backup.sha256
   file backup.enc
   tail -c 64 backup.enc | hexdump -C
   

   If checksum fails or the file is unexpectedly short, jump to Corrupted or truncated encrypted backup mistaken for a key problem.

5. If none of the above restores access, search for escrow/exported key material.

   ⚠️ Do not copy private keys into chat, tickets, or shared paste tools. Handle recovered key material only on an approved admin workstation and rotate it after the incident.

   find /secure /etc /home -maxdepth 3 \( -name '*.asc' -o -name '*.gpg' -o -name 'age*key*' -o -name '*.p12' -o -name '*backup*key*' \) 2>/dev/null
   

   If you find no exported private key, no escrow, and the KMS key is actually deleted, jump to The actual key material was deleted and never exported/escrowed.

Fixes

Missing decrypt permission to the KMS/HSM/key service

Grant decrypt on the exact key to the restore principal, then retry the decrypt test before rerunning the full restore.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["kms:Decrypt", "kms:DescribeKey"],
      "Resource": "arn:aws:kms:us-east-1:123456789012:key/11111111-2222-3333-4444-555555555555"
    }
  ]
}

If your org uses key policies instead of IAM-only grants, add the principal there too. Then test:

aws kms decrypt --key-id arn:aws:kms:us-east-1:123456789012:key/11111111-2222-3333-4444-555555555555 --ciphertext-blob fileb://wrapped-key.bin --output text --query Plaintext >/dev/null && echo OK

Verify it worked: the command exits 0 and prints OK.

Key is disabled, pending deletion, or in the wrong region/account

For a disabled key, re-enable it:

aws kms enable-key --key-id arn:aws:kms:us-east-1:123456789012:key/11111111-2222-3333-4444-555555555555
aws kms describe-key --key-id arn:aws:kms:us-east-1:123456789012:key/11111111-2222-3333-4444-555555555555 --query 'KeyMetadata.KeyState'

If the manifest references us-east-1 but you are querying us-west-2, switch region/profile explicitly:

AWS_PROFILE=prod AWS_REGION=us-east-1 aws kms describe-key --key-id alias/prod-backup-key

If the key is PendingDeletion, cancel deletion if your provider still allows it:

aws kms cancel-key-deletion --key-id arn:aws:kms:us-east-1:123456789012:key/11111111-2222-3333-4444-555555555555

Trade-off: re-enabling or canceling deletion may violate your normal retirement process; document the exception and rotate after recovery. Verify it worked: describe-key shows "Enabled" and a decrypt test succeeds.

You have the wrong private key/passphrase for file-level encryption

For GPG, import the correct secret key and retry decryption:

gpg --import team-backup-private.asc
gpg --list-secret-keys --keyid-format LONG
gpg --decrypt backup.sql.gpg > /dev/null

Typical success shape:

gpg: encrypted with 4096-bit RSA key, ID 1234ABCD5678EF90, created 2025-02-10
      "backup-restore@example.com"

For age:

age -d -i ~/.config/age/keys.txt backup.tar.age > /dev/null

For OpenSSL passphrase-based backups, use the exact original flags; mismatched digest/KDF options often look like a bad password:

openssl enc -d -aes-256-cbc -pbkdf2 -md sha256 -in backup.enc -out /dev/null

Edge case: if the old process used openssl enc without -pbkdf2, you must match that legacy behavior exactly. Verify it worked: decrypt exits 0 and writes plausible plaintext or a valid archive header.

Backup metadata points to an old alias or rotated key version you no longer retained

Resolve aliases to immutable key IDs and restore using the original key ID, not the current alias target. Example manifest patch for a restore script:

{
  "key_alias": "alias/prod-backup-key",
  "key_id": "arn:aws:kms:us-east-1:123456789012:key/11111111-2222-3333-4444-555555555555"
}

Then invoke the restore with the immutable ID:

./restore.sh --manifest backup-manifest.json --kms-key-id arn:aws:kms:us-east-1:123456789012:key/11111111-2222-3333-4444-555555555555

If your rotation process destroyed old private subkeys, recovery depends on whether the old decrypting key still exists anywhere. Rotation is safe only when old decrypt keys are retained for historical backups. Verify it worked: the restore tool logs the original key ID and proceeds past key unwrap.

The actual key material was deleted and never exported/escrowed

If the KMS key is permanently deleted or the only private key was destroyed and never escrowed, there is no practical cryptographic recovery. Stop spending time on restore tooling and move to incident containment:

printf '%s
' 'Backup unrecoverable: decrypt key material unavailable' | tee incident-summary.txt

Actions:

• Preserve the encrypted backup and all metadata for audit.
• Confirm whether any older backups used a different still-accessible key.
• Rebuild from replicas, snapshots, downstream exports, or application-level rehydration if available.
• Open an internal incident and record the exact key ARN/fingerprint and deletion timeline. Verify it worked: you have a documented recovery/no-recovery decision and an alternate data source plan.

Corrupted or truncated encrypted backup mistaken for a key problem

Re-download or re-copy the object and verify checksums before retrying decrypt.

aws s3 cp s3://my-bucket/backups/backup.tar.age ./backup.tar.age
sha256sum -c backup.tar.age.sha256

If no checksum exists, compare object size against inventory/logs:

aws s3api head-object --bucket my-bucket --key backups/backup.tar.age --query '{Length:ContentLength,ETag:ETag}'

Trade-off: some encryption formats fail with generic bad decrypt on truncated input, so checksum validation is cheaper than repeated key debugging. Verify it worked: checksum passes and decryption error changes or succeeds.

Prevention

1. Store immutable key identifiers in backup manifests, not just aliases.

{
  "backup_id": "2026-10-30T14:00:00Z",
  "kms_key_id": "arn:aws:kms:us-east-1:123456789012:key/11111111-2222-3333-4444-555555555555",
  "kms_alias": "alias/prod-backup-key",
  "sha256": "..."
}

This avoids alias drift during rotation.

1. Add a scheduled restore test that performs an actual decrypt, not just object existence.

#!/usr/bin/env bash
set -euo pipefail
latest=$(aws s3 ls s3://my-bucket/backups/ | tail -1 | awk '{print $4}')
aws s3 cp "s3://my-bucket/backups/$latest" - | age -d -i /run/secrets/age-key >/dev/null

Alert on non-zero exit code.

1. Escrow decrypt-capable key material under dual control for non-KMS schemes.

gpg --export-secret-keys --armor BACKUP-RESTORE-KEYID > backup-restore-secret.asc
age-keygen -o age-backup-recovery.txt

Store escrow in your approved secret vault/offline medium with access logging; do not leave the only copy on one engineer laptop.

1. Block destructive key lifecycle actions when backups still depend on the key.

aws kms list-aliases --query 'Aliases[?TargetKeyId==`11111111-2222-3333-4444-555555555555`]'

Tie this into your change process so deletion requires proof that all backups encrypted under that key have expired or been re-encrypted.

1. Record encryption parameters alongside the backup.

{
  "format": "openssl-enc",
  "cipher": "aes-256-cbc",
  "kdf": "pbkdf2",
  "digest": "sha256"
}

This prevents false bad decrypt incidents caused by missing flags.

1. Add CI/pipeline checks that fail backup changes without restore metadata.

jq -e '.kms_key_id and .sha256 and .created_at' backup-manifest.json >/dev/null

Fail the job if required fields are absent.

This article was written by an AI system and published pending human review. Verify anything you intend to act on.