Q: How do you prevent accidental destruction of production resources in Terraform?
Multiple layers: lifecycle { prevent_destroy = true } on critical resources. Pipeline policy that fails if plan contains destroys. AWS Config rules that alert on resource deletion. S3 MFA Delete for the state bucket itself.
Want to master this scenario in a live sandbox? KodeKloud's HashiCorp Certified Terraform Associate (003) Interactive Labs covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: "In our enterprise Terraform repository, we designed reusable modules and remote backends to prevent this exact issue. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix."
Production Solution & Architecture
Multiple layers: lifecycle { prevent_destroy = true } on critical resources. Pipeline policy that fails if plan contains destroys. AWS Config rules that alert on resource deletion. S3 MFA Delete for the state bucket itself.
- Immediate Triage: Multiple layers: lifecycle { prevent_destroy = true } on critical resources. Pipeline policy th
- Run targeted verification commands before modifying configuration.
- Automate permanent guardrails (CI check, alerts, IaC policy) to prevent recurrence.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Relying on legacy bucket ACLs instead of modern S3 Block Public Access and IAM bucket policies when configuring prevent accidental destruction AWS recommends disabling ACLs (Bucket Owner Enforced) in favor of centralized declarative bucket policies and account-level Block Public Access controls.
-
Confusing SSE-S3 encryption with KMS customer-managed key (SSE-KMS) authorization policies when configuring prevent accidental destruction
Accessing SSE-KMS encrypted objects requires both
s3:GetObjectandkms:Decryptpermissions; neglecting KMS key policies causes silent 403 Forbidden errors. - Using sequential timestamps or predictable prefixes for high-throughput S3 read and write keys when configuring prevent accidental destruction Sequential key prefixes limit S3 horizontal partitioning; randomized hash prefixes distribute workloads across partitions to achieve 3,500 PUT and 5,500 GET requests/sec.
- Failing to audit S3 object delete markers when troubleshooting missing versioned objects when configuring prevent accidental destruction Issuing a DELETE on a versioned bucket only adds a delete marker; engineers must retrieve previous versions or purge markers to restore data.