Q: Your Terraform state file got corrupted. What do you do?
1. If using remote backend with versioning (S3 + versioning enabled) — restore the previous version of the state file from S3.
#Terraform #State & Locking #L2 #IaC #Cloud Infrastructure #S3
🎙️ Candidate Opening & Architectural Context
""In our enterprise Terraform repository, we designed reusable modules and remote backends to prevent this exact issue. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix.""
Advertisement
🛠️ Production Runbook & Step-by-Step Resolution
1️⃣
Initial Diagnostics & Root Cause Analysis
Never manually edit the .tfstate file directly — it's JSON with checksums. If you must, use terraform state commands.
- If using remote backend with versioning (S3 + versioning enabled) — restore the previous version of the state file from S3.
- If using Terraform Cloud — it keeps state history. Roll back to last known good state.
- Manual reconstruction — worst case: use
terraform importto re-import all existing resources into a fresh state file. Painful but possible.
2️⃣
Remediation & Permanent Safeguards
- Prevention — always use remote backend, enable S3 versioning, enable Terraform state locking.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Pro-Tip: If using remote backend with versioning (S3 + versioning enabled) — restore the previous version of the state file from S3.."
⚡ 60-Second Elevator Pitch Talking Points
- If using remote backend with versioning (S3 + versioning enabled) — restore the previous version ...
- If using Terraform Cloud — it keeps state history. Roll back to last known good state.
- Manual reconstruction — worst case: use terraform import to re-import all existing resources into...
Advertisement