Q: Pipeline succeeds but new version is not deployed — how would you debug?
Diagnostic guide for when a pipeline succeeds not deployed: troubleshooting image tag caching, branch rules, and GitOps sync gaps.
Pipeline Succeeds Not Deployed? Debug Like a Senior SRE
Watch Naveed break down how to troubleshoot when a CI/CD pipeline succeeds with green checkmarks but the new version is not deployed—triaging live running container image digests, avoiding mutable :latest tags and IfNotPresent pull caching traps, auditing skipped pipeline stages, and inspecting GitOps ArgoCD/Flux sync reconciliation in 60 seconds.
- Verify live running container image digest: kubectl get deploy <app> -o jsonpath='{..image}'
- Avoid mutable :latest tag & IfNotPresent caching traps that skip pulling newly pushed layers
- Audit CI pipeline job conditions to ensure deployment stages were not silently skipped
- Inspect GitOps controller & manifest repo: verify commit SHA and ArgoCD / Flux sync state
Want to master this scenario in a live sandbox? KodeKloud's Enterprise GitOps with ArgoCD & Kubernetes Rollouts covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: When a pipeline succeeds not deployed in Kubernetes, the root cause typically stems from mutable image tag caching, skipped execution stages, or GitOps reconciliation gaps.
Verify What is Actually Running in Production
Start by inspecting the live cluster/server before checking pipeline scripts:
- Run:
kubectl get deployment <app> -o jsonpath='{.spec.template.spec.containers[0].image}'. If pods fail to schedule, review the Kubernetes Pod Stuck in Pending runbook. - Check the image tag and digest. Does it match the newly built Git commit SHA?
- Hit the application's version endpoint:
curl https://app.example.com/version.
The ':latest' Tag & imagePullPolicy Trap
The single most common root cause in container CI/CD:
- If the pipeline pushes
myapp:latestand the Kubernetes Deployment manifest saysimage: myapp:latestwithimagePullPolicy: IfNotPresent: - Kubernetes checks if a tag named
latestexists locally on the node. If yes, it never pulls the new image from the registry! - Furthermore, Kubernetes detects no change in the Deployment manifest (the image string is still
myapp:latest), triggering zero rollout. If a bad image was cached, see our Kubernetes CrashLoopBackOff triage guide. - Fix: Always use immutable image tags based on Git SHA or semantic release (e.g.
myapp:sha-7f3a9b2).
Pipeline Step Conditions & Environment Mismatch
Audit pipeline execution steps:
- Skipped Deploy Step: Did the build/test job succeed, but the deploy job was skipped because of a condition like
if: github.ref == 'refs/heads/main'when building a feature branch? - Target Environment Mismatch: Did the pipeline deploy to Staging instead of Production due to environment variable configuration?
- Manual Approval Gate: Is the pipeline waiting on manual approval in GitHub Actions Environments / GitLab Protected Environments?
GitOps Manifest Repo & Controller Audit
If using a separate manifest repository (ArgoCD / Flux):
- Did the CI pipeline successfully commit and push the updated image tag to the config repo? (Check git credentials and branch protection rules).
- Check ArgoCD sync status: Is the application in
OutOfSyncorSync Failedstate? For rapid rollbacks, review our Kubernetes Deployment Rollback runbook. - Check for Kubernetes manifest validation failure (e.g. invalid YAML or unaccepted CPU limit).
- Verify running container image: 'kubectl get deploy <app> -o jsonpath={..image}'.
- Check image tagging: Avoid ':latest' with 'imagePullPolicy: IfNotPresent' which ignores new image pushes.
- Audit CI logs: Ensure the deploy job actually executed and was not skipped by branch/tag conditions.
- Check GitOps repo: Verify CI successfully pushed new image tag commit to the manifest repository.
- Check ArgoCD/Flux: Inspect sync status, controller errors, or paused auto-sync.
Mastered this procedure? Subscribe to @naveedkumbhar01 on YouTube for daily 60-second SRE incident runbooks, Kubernetes triage shortcuts, and high-impact interview elevator pitches.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Promoting application releases to production without an automated, canary-validated rollback mechanism when configuring CI/CD Pipeline Succeeds Pipelines that lack automated rollback mechanisms force engineering teams to scramble manual hotfixes during severe production regressions.
- Running concurrent deployment pipeline jobs without stage mutex locking on shared target environments when configuring CI/CD Pipeline Succeeds Concurrent pipeline runs on the same environment race to apply state changes, causing out-of-order commits and unpredictable deployment state.
- Omitting dependency lockfiles (package-lock.json, go.sum, Cargo.lock) from continuous integration builds when configuring CI/CD Pipeline Succeeds Omitting lockfiles allows upstream package updates to break builds unpredictably without any changes to application codebase.
- Failing to define dynamic cache invalidation keys for CI package managers and layer caches when configuring CI/CD Pipeline Succeeds Stale build caches mask underlying dependency incompatibilities or pull outdated libraries, while un-cached runs slow feedback loops.