Q: Two developers ran `terraform apply` at the same time on the same workspace. What happened and how do you prevent it?
This is a race condition. The last write wins — whichever apply finishes last overwrites the state file. This can cause state corruption and out-of-sync infrastructure.
Want to master this scenario in a live sandbox? KodeKloud's HashiCorp Certified Terraform Associate (003) Interactive Labs covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: "When terraform plan shows unexpected changes, my golden rule is: never apply blindly. Investigate the diff first. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix."
Initial Diagnostics & Root Cause Analysis
This is a race condition. The last write wins — whichever apply finishes last overwrites the state file. This can cause state corruption and out-of-sync infrastructure.
- Use an S3 backend with DynamoDB locking: Terraform writes a lock entry to DynamoDB before applying. If another apply is running, the lock is already taken and the second apply waits or fails.
- Terraform Cloud/HCE automatically handles locking.
- Never use local state files for team work — they can't be locked.
Remediation & Permanent Safeguards
Prevention — state locking: Best practice: Run Terraform only from CI/CD pipelines, never from developer laptops directly. The pipeline enforces sequential execution.
- Use an S3 backend with DynamoDB locking: Terraform writes a lock entry to DynamoDB before applyin...
- Terraform Cloud/HCE automatically handles locking.
- Never use local state files for team work — they can't be locked.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Failing to configure lifecycle expiration rules for incomplete multipart S3 uploads Aborted or incomplete multipart upload parts remain in storage indefinitely, accumulating storage costs without appearing in standard console bucket listings.
-
Attempting cross-account S3 object copies without configuring destination bucket ownership overrides
By default, objects uploaded across accounts remain owned by the uploader account unless
bucket-owner-full-controlis explicitly enforced. - Relying on public internet S3 endpoints from inside private VPC subnets instead of S3 Gateway Endpoints VPC S3 Gateway Endpoints eliminate NAT Gateway data processing fees and ensure high-throughput storage traffic remains entirely on the private AWS network.
- Enabling S3 Cross-Region Replication (CRR) without verifying versioning status on both source and destination buckets in terraform-q002-two-developers-ran-terraform-apply- CRR strictly requires S3 Versioning to be enabled on both source and destination buckets to synchronize object revisions and delete markers.