⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Terraform & IaC Interview Questions Scenario 105 of 120 in Terraform & IaC
Staff SRE / Principal Architect [L3] Terraform Terraform Modules & State Staff SRE Scenario [L3]
🎯 Target Role / Context: Terraform Scenario #100 (L3) · Use VPC ID from another module

Q: A Terraform apply introduced a bad infrastructure change in production. What is the rollback process?

Revert the Terraform code to the last known good version and run a new plan to see what Terraform will change back.

#Terraform #L3 #IaC #Cloud Infrastructure #Terraform State
🎙️ Candidate Opening & Architectural Context
""Treat Terraform code with the same rigor as application code: pre-merge plans, state locks, and automated drift detection. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix.""
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's HashiCorp Certified Terraform Associate (003) Interactive Labs covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: "Treat Terraform code with the same rigor as application code: pre-merge plans, state locks, and automated drift detection. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix."

1️⃣

Production Solution & Architecture

Revert the Terraform code to the last known good version and run a new plan to see what Terraform will change back. Apply that reviewed rollback plan through the normal approval path unless the incident process allows emergency approval. Restore state only if the state itself is wrong or corrupted; for a bad but successful infrastructure change, state usually reflects reality and the fix is another controlled apply. --- *More Terraform scenarios added periodically. PRs welcome.*

💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Pro-Tip: Revert the Terraform code to the last known good version and run a new plan to see what Terraform will change back. Apply that rev."
⚡ 60-Second Elevator Pitch Talking Points
  • Immediate Triage: Revert the Terraform code to the last known good version and run a new plan to see what Terrafo
  • Run targeted verification commands before modifying configuration.
  • Automate permanent guardrails (CI check, alerts, IaC policy) to prevent recurrence.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Executing Terraform applies without inspecting '# forces replacement' warnings in plan output when configuring Terraform apply introduced Attributes that force resource replacement destroy and recreate critical infrastructure (databases, load balancers), inducing unannounced production outages.
  2. Omitting remote state locking mechanisms (DynamoDB or backend native locks) for shared state files when configuring Terraform apply introduced Without state locking, concurrent CI/CD pipeline executions or team applies create race conditions that corrupt the state file.
  3. Structuring entire enterprise cloud footprints into a single monolithic Terraform root module when configuring Terraform apply introduced Monolithic state files slow plan execution dramatically, increase blast radius during failed applies, and create severe pipeline locking contention.
  4. Failing to pin required_providers versions, causing breaking provider upgrades in CI pipelines when configuring Terraform apply introduced Unpinned provider versions automatically download breaking major upgrades during CI runs, introducing unexpected schema validation failures.
Advertisement
Want more Terraform & IaC scenarios?
Explore our complete collection of scenario-based Terraform & IaC interview runbooks.
Browse All Terraform & IaC Questions →