Q: Engineers in your team run 'terraform apply' locally from their laptops, leading to concurrent state file collisions, unreviewed infrastructure changes, and lost drift history. How do you design and deploy Atlantis on Kubernetes to automate Terraform / Terragrunt planning and applying directly via GitHub PR comments with strict approval gates?
Engineering a collaborative, secure Infrastructure as Code (IaC) deployment pipeline using Atlantis for Terraform & Terragrunt, with automated PR comment planning, state locking, and apply controls.
#CI/CD #Atlantis #Terraform #Terragrunt #GitOps #IaC #Automation
🎙️ Candidate Opening & Architectural Context
"Running Terraform locally from engineer workstations is an operational anti-pattern that leads to unreviewed production changes and leaked state files. We deployed Atlantis on Kubernetes to turn pull requests into the sole control plane for infrastructure execution."
Advertisement
⚡ Recommended Practice Lab
Want to master this scenario in a live sandbox? KodeKloud's Enterprise GitOps with ArgoCD & Kubernetes Rollouts covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
1️⃣
Deploy Atlantis StatefulSet on Kubernetes with Cloud IAM Roles
Establish a secure, persistent Atlantis execution engine:
- StatefulSet Installation: Deployed Atlantis StatefulSet with persistent volume storage for checked-out Git repositories and Terraform workspaces.
- Cloud Authentication: Authenticated Atlantis pod via AWS IAM Roles for Service Accounts (IRSA) / Azure Workload Identity, eliminating static cloud credentials.
Pro Tip: Using IRSA grants Atlantis temporary cloud credentials with full audit logging in AWS CloudTrail.
2️⃣
Configure atlantis.yaml Project Directory Layout for Terragrunt
Define automated plan and apply execution workflows:
- atlantis.yaml Spec: Configured project declarations mapping directories (e.g.
live/prod/vpc,live/prod/eks). - Terragrunt Workflow: Defined custom workflow steps:
plan: { steps: [{ run: 'terragrunt plan -out $PLANFILE' }] }andapply: { steps: [{ run: 'terragrunt apply $PLANFILE' }] }.
Pro Tip: Configuring explicit atlantis.yaml projects ensures that plans execute in the correct dependency order across multi-module Terragrunt structures.
Advertisement
3️⃣
Execute Automated PR Planning & Distributed State Locking
Provide transparency and prevent concurrent modifications to the same infrastructure:
- Automated PR Plan: Opening a PR triggers Atlantis to automatically run
terragrunt plan, posting the formatted diff directly as a comment on the GitHub PR. - Atlantis Lock: Atlantis locks the modified directory; if another engineer opens a PR touching the same module, Atlantis rejects the plan until the first PR is merged or unlocked.
Pro Tip: Directory-level locking in Atlantis prevents race conditions between multiple engineers editing the same Terraform state simultaneously.
4️⃣
Enforce Mandatory Peer Approvals & In-PR Apply Execution
Ensure changes are reviewed by senior engineers before hitting production:
- Apply Requirements: Configured
apply_requirements: [approved, mergeable]; Atlantis refuses to executeatlantis applyunless the PR has at least 2 peer approvals and passes CI checks. - In-PR Apply: Approved engineer comments
atlantis apply; Atlantis executes the apply and automatically merges the PR and closes the branch upon success.
Pro Tip: Requiring peer approvals before allowing 'atlantis apply' guarantees 100% compliance with corporate SOC 2 change control policies.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Atlantis turns GitHub pull requests into an auditable GitOps control plane for Terraform and Terragrunt, featuring automated PR plan comments, directory-level locks, and mandatory peer-approval apply gates."
⚡ 60-Second Elevator Pitch Talking Points
- Deploy Atlantis on Kubernetes with AWS IRSA / Azure Workload Identity.
- Configure atlantis.yaml with custom Terragrunt plan and apply workflows.
- Lock modified project directories automatically to prevent concurrent state collisions.
- Enforce mandatory peer approvals before unlocking the 'atlantis apply' PR comment command.
Advertisement