⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Terraform & IaC Interview Questions Scenario 7 of 120 in Terraform & IaC
Senior DevOps / SRE [L2] Terraform State & Locking Production Scenario [L2]
🎯 Target Role / Context: Terraform Scenario #2 (L2) · State & Locking

Q: Two developers ran `terraform apply` at the same time on the same workspace. What happened and how do you prevent it?

This is a race condition. The last write wins — whichever apply finishes last overwrites the state file. This can cause state corruption and out-of-sync infrastructure.

#Terraform #State & Locking #L2 #IaC #Cloud Infrastructure #S3
🎙️ Candidate Opening & Architectural Context
""When terraform plan shows unexpected changes, my golden rule is: never apply blindly. Investigate the diff first. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix.""
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's HashiCorp Certified Terraform Associate (003) Interactive Labs covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: "When terraform plan shows unexpected changes, my golden rule is: never apply blindly. Investigate the diff first. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix."

1️⃣

Initial Diagnostics & Root Cause Analysis

This is a race condition. The last write wins — whichever apply finishes last overwrites the state file. This can cause state corruption and out-of-sync infrastructure.

  • Use an S3 backend with DynamoDB locking: Terraform writes a lock entry to DynamoDB before applying. If another apply is running, the lock is already taken and the second apply waits or fails.
  • Terraform Cloud/HCE automatically handles locking.
  • Never use local state files for team work — they can't be locked.
2️⃣

Remediation & Permanent Safeguards

Prevention — state locking: Best practice: Run Terraform only from CI/CD pipelines, never from developer laptops directly. The pipeline enforces sequential execution.

💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Pro-Tip: Use an S3 backend with DynamoDB locking: Terraform writes a lock entry to DynamoDB before applying. If another apply is running, t."
⚡ 60-Second Elevator Pitch Talking Points
  • Use an S3 backend with DynamoDB locking: Terraform writes a lock entry to DynamoDB before applyin...
  • Terraform Cloud/HCE automatically handles locking.
  • Never use local state files for team work — they can't be locked.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Failing to configure lifecycle expiration rules for incomplete multipart S3 uploads Aborted or incomplete multipart upload parts remain in storage indefinitely, accumulating storage costs without appearing in standard console bucket listings.
  2. Attempting cross-account S3 object copies without configuring destination bucket ownership overrides By default, objects uploaded across accounts remain owned by the uploader account unless bucket-owner-full-control is explicitly enforced.
  3. Relying on public internet S3 endpoints from inside private VPC subnets instead of S3 Gateway Endpoints VPC S3 Gateway Endpoints eliminate NAT Gateway data processing fees and ensure high-throughput storage traffic remains entirely on the private AWS network.
  4. Enabling S3 Cross-Region Replication (CRR) without verifying versioning status on both source and destination buckets in terraform-q002-two-developers-ran-terraform-apply- CRR strictly requires S3 Versioning to be enabled on both source and destination buckets to synchronize object revisions and delete markers.
Advertisement
Want more Terraform & IaC scenarios?
Explore our complete collection of scenario-based Terraform & IaC interview runbooks.
Browse All Terraform & IaC Questions →