⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All CI/CD & GitOps Interview Questions Scenario 1 of 187 in CI/CD & GitOps
Senior DevOps / SRE CI/CD Pipelines & Delivery Pipeline Triage
🎯 Target Role / Context: CI/CD & Delivery Engineering

Q: Pipeline succeeds but new version is not deployed — how would you debug?

Diagnostic guide for when a pipeline succeeds not deployed: troubleshooting image tag caching, branch rules, and GitOps sync gaps.

#CI/CD #Docker #Image Tagging #GitOps #Kubernetes #GitHub Actions
🎙️ Candidate Opening & Architectural Context
"When a pipeline succeeds not deployed in Kubernetes, the root cause typically stems from mutable image tag caching, skipped execution stages, or GitOps reconciliation gaps."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's Enterprise GitOps with ArgoCD & Kubernetes Rollouts covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: When a pipeline succeeds not deployed in Kubernetes, the root cause typically stems from mutable image tag caching, skipped execution stages, or GitOps reconciliation gaps.

1️⃣

Verify What is Actually Running in Production

Start by inspecting the live cluster/server before checking pipeline scripts:

  • Run: kubectl get deployment <app> -o jsonpath='{.spec.template.spec.containers[0].image}'. If pods fail to schedule, review the Kubernetes Pod Stuck in Pending runbook.
  • Check the image tag and digest. Does it match the newly built Git commit SHA?
  • Hit the application's version endpoint: curl https://app.example.com/version.
2️⃣

The ':latest' Tag & imagePullPolicy Trap

The single most common root cause in container CI/CD:

  • If the pipeline pushes myapp:latest and the Kubernetes Deployment manifest says image: myapp:latest with imagePullPolicy: IfNotPresent:
  • Kubernetes checks if a tag named latest exists locally on the node. If yes, it never pulls the new image from the registry!
  • Furthermore, Kubernetes detects no change in the Deployment manifest (the image string is still myapp:latest), triggering zero rollout. If a bad image was cached, see our Kubernetes CrashLoopBackOff triage guide.
  • Fix: Always use immutable image tags based on Git SHA or semantic release (e.g. myapp:sha-7f3a9b2).
Advertisement
3️⃣

Pipeline Step Conditions & Environment Mismatch

Audit pipeline execution steps:

  • Skipped Deploy Step: Did the build/test job succeed, but the deploy job was skipped because of a condition like if: github.ref == 'refs/heads/main' when building a feature branch?
  • Target Environment Mismatch: Did the pipeline deploy to Staging instead of Production due to environment variable configuration?
  • Manual Approval Gate: Is the pipeline waiting on manual approval in GitHub Actions Environments / GitLab Protected Environments?
4️⃣

GitOps Manifest Repo & Controller Audit

If using a separate manifest repository (ArgoCD / Flux):

  • Did the CI pipeline successfully commit and push the updated image tag to the config repo? (Check git credentials and branch protection rules).
  • Check ArgoCD sync status: Is the application in OutOfSync or Sync Failed state? For rapid rollbacks, review our Kubernetes Deployment Rollback runbook.
  • Check for Kubernetes manifest validation failure (e.g. invalid YAML or unaccepted CPU limit).
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"When a pipeline succeeds not deployed, inspect live running container digests first, avoid mutable tags, and verify GitOps manifest commit chains."
⚡ 60-Second Elevator Pitch Talking Points
  • Verify running container image: 'kubectl get deploy <app> -o jsonpath={..image}'.
  • Check image tagging: Avoid ':latest' with 'imagePullPolicy: IfNotPresent' which ignores new image pushes.
  • Audit CI logs: Ensure the deploy job actually executed and was not skipped by branch/tag conditions.
  • Check GitOps repo: Verify CI successfully pushed new image tag commit to the manifest repository.
  • Check ArgoCD/Flux: Inspect sync status, controller errors, or paused auto-sync.
🎬 60-Second Video Walkthrough
Watch the Animated Breakdown & Practice Live

Mastered this procedure? Subscribe to @naveedkumbhar01 on YouTube for daily 60-second SRE incident runbooks, Kubernetes triage shortcuts, and high-impact interview elevator pitches.

🔔 Subscribe on YouTube (Free) ▶ Watch Short (60s) ↗
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Promoting application releases to production without an automated, canary-validated rollback mechanism when configuring CI/CD Pipeline Succeeds Pipelines that lack automated rollback mechanisms force engineering teams to scramble manual hotfixes during severe production regressions.
  2. Running concurrent deployment pipeline jobs without stage mutex locking on shared target environments when configuring CI/CD Pipeline Succeeds Concurrent pipeline runs on the same environment race to apply state changes, causing out-of-order commits and unpredictable deployment state.
  3. Omitting dependency lockfiles (package-lock.json, go.sum, Cargo.lock) from continuous integration builds when configuring CI/CD Pipeline Succeeds Omitting lockfiles allows upstream package updates to break builds unpredictably without any changes to application codebase.
  4. Failing to define dynamic cache invalidation keys for CI package managers and layer caches when configuring CI/CD Pipeline Succeeds Stale build caches mask underlying dependency incompatibilities or pull outdated libraries, while un-cached runs slow feedback loops.
Advertisement
Want more CI/CD & GitOps scenarios?
Explore our complete collection of scenario-based CI/CD & GitOps interview runbooks.
Browse All CI/CD & GitOps Questions →