⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Terraform & IaC Interview Questions Scenario 19 of 120 in Terraform & IaC
Staff SRE / Principal Architect [L3] Terraform Workspaces & CI/CD Staff SRE Scenario [L3]
🎯 Target Role / Context: Terraform Scenario #14 (L3) · Workspaces & CI/CD

Q: How do you run Terraform safely in a CI/CD pipeline? What are the guardrails?

State handling:

#Terraform #Workspaces & CI/CD #L3 #IaC #Cloud Infrastructure #S3
🎙️ Candidate Opening & Architectural Context
""When terraform plan shows unexpected changes, my golden rule is: never apply blindly. Investigate the diff first. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix.""
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's HashiCorp Certified Terraform Associate (003) Interactive Labs covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: "When terraform plan shows unexpected changes, my golden rule is: never apply blindly. Investigate the diff first. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix."

1️⃣

Initial Diagnostics & Root Cause Analysis

State handling:

  • Remote backend (S3 + DynamoDB locking) — never local state in CI.
  • Each pipeline run acquires lock before apply, releases after.
  • terraform plan -out=plan.tfplan in one stage.
  • Human reviews the plan (or automated check for unexpected destroys).
  • terraform apply plan.tfplan in a separate stage.
  • Fail the pipeline if plan shows any destroy without explicit override.
  • Run terraform fmt -check to fail on unformatted code.
2️⃣

Remediation & Permanent Safeguards

Plan before apply: Guardrails: No developer applies directly:

  • Run terraform validate to check syntax.
  • Run tflint for provider-specific lint rules.
  • Run tfsec or checkov for security misconfigurations.
  • Separate pipelines for different environments. Production requires manual approval.
  • All Terraform runs go through CI.
  • Developers open PRs → plan runs → review → merge → apply runs.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Pro-Tip: Remote backend (S3 + DynamoDB locking) — never local state in CI.."
⚡ 60-Second Elevator Pitch Talking Points
  • Remote backend (S3 + DynamoDB locking) — never local state in CI.
  • Each pipeline run acquires lock before apply, releases after.
  • terraform plan -out=plan.tfplan in one stage.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Relying on legacy bucket ACLs instead of modern S3 Block Public Access and IAM bucket policies when configuring Terraform safely CI/CD AWS recommends disabling ACLs (Bucket Owner Enforced) in favor of centralized declarative bucket policies and account-level Block Public Access controls.
  2. Confusing SSE-S3 encryption with KMS customer-managed key (SSE-KMS) authorization policies when configuring Terraform safely CI/CD Accessing SSE-KMS encrypted objects requires both s3:GetObject and kms:Decrypt permissions; neglecting KMS key policies causes silent 403 Forbidden errors.
  3. Using sequential timestamps or predictable prefixes for high-throughput S3 read and write keys when configuring Terraform safely CI/CD Sequential key prefixes limit S3 horizontal partitioning; randomized hash prefixes distribute workloads across partitions to achieve 3,500 PUT and 5,500 GET requests/sec.
  4. Failing to audit S3 object delete markers when troubleshooting missing versioned objects when configuring Terraform safely CI/CD Issuing a DELETE on a versioned bucket only adds a delete marker; engineers must retrieve previous versions or purge markers to restore data.
Advertisement
Want more Terraform & IaC scenarios?
Explore our complete collection of scenario-based Terraform & IaC interview runbooks.
Browse All Terraform & IaC Questions →