⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Kubernetes Interview Questions Scenario 3 of 207 in Kubernetes
Senior DevOps / SRE Kubernetes Troubleshooting & Debugging Core K8s Scenario

Q: Pod is in CrashLoopBackOff — how would you troubleshoot?

Exhaustive triage process for debugging CrashLoopBackOff: decoding container exit codes (137 OOMKill, 1 app error, 143 SIGTERM), fetching previous container logs, and fixing probe failures.

#Kubernetes #CrashLoopBackOff #OOMKilled #kubectl logs #Probes #Exit Codes
🎙️ Candidate Opening & Architectural Context
"CrashLoopBackOff means the container started, crashed, and Kubernetes is backing off before restarting it. My troubleshooting sequence always follows: Exit Code analysis -> Previous container logs -> Liveness probe evaluation."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: CrashLoopBackOff means the container started, crashed, and Kubernetes is backing off before restarting it. My troubleshooting sequence always follows: Exit Code analysis -> Previous container logs -> Liveness probe evaluation.

1️⃣

Inspect Exit Code via kubectl describe

Run kubectl describe pod <pod-name> and examine the Last State: Terminated block:

  • Exit Code 137 (OOMKilled): Reason: OOMKilled. The Linux kernel OOM Killer terminated the container for exceeding its limits.memory. Fix: bump memory limit or fix memory leak.
  • Exit Code 1 or 255 (App Error): Application code crashed (uncaught exception, syntax error, missing environment variable, failed DB connection).
  • Exit Code 127 (Command Not Found): Container CMD/ENTRYPOINT executable does not exist inside the image.
  • Exit Code 143 (SIGTERM): Graceful shutdown signal was received, often from a failing liveness probe or preStop hook.
  • Exit Code 0 (Completed): Container finished its task and exited normally, but pod restartPolicy is Always instead of OnFailure/Never.
2️⃣

Fetch Previous Container Logs (--previous)

Because the crashing container has already terminated, standard logs may be empty or only show startup lines:

  • kubectl logs <pod-name> --previous: The golden command — reads the stdout/stderr from the crashed instance before it restarted.
  • If multi-container pod: kubectl logs <pod-name> -c <container-name> --previous.
  • Look at the last 20 lines for stack traces, database connection timeouts, or missing config keys.
Advertisement
3️⃣

Check Liveness & Startup Probes

A misconfigured liveness probe will actively kill a healthy container during slow startup:

  • Check events for: Liveness probe failed: HTTP probe failed with statuscode: 500.
  • If an app takes 45 seconds to initialize but initialDelaySeconds is 10, Kubernetes kills it repeatedly!
  • Fix: Implement a startupProbe with generous failureThreshold, giving the app time to start before liveness checks engage.
4️⃣

Interactive Debugging with Ephemeral Containers

If logs are silent and container crashes instantly:

  • Override entrypoint in a local manifest: change command to ['sh', '-c', 'sleep 3600'] to keep container alive, then kubectl exec -it to inspect files and environment.
  • Use kubectl debug -it <pod-name> --image=busybox --target=<container> to attach an ephemeral debugging container sharing the process namespace.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Look at the Exit Code first (137 = OOM, 1 = App crash). Use 'kubectl logs --previous' to capture the crash stack trace. Verify startupProbe isn't killing slow-starting applications."
⚡ 60-Second Elevator Pitch Talking Points
  • Run 'kubectl describe pod <name>' -> inspect Last State: Exit Code (137 = OOMKill, 1 = code exception, 127 = binary missing).
  • Run 'kubectl logs <name> --previous' to retrieve the stack trace from the crashed instance.
  • If Exit Code 137: Increase spec.resources.limits.memory or profile memory leak.
  • Check Probes: Verify livenessProbe isn't timing out during slow boots; add startupProbe.
  • If container crashes instantly: Override command with 'sleep 3600' or attach ephemeral container via 'kubectl debug'.
🎬 60-Second Video Walkthrough
Watch the Animated Breakdown & Practice Live

Mastered this procedure? Subscribe to @naveedkumbhar01 on YouTube for daily 60-second SRE incident runbooks, Kubernetes triage shortcuts, and high-impact interview elevator pitches.

🔔 Subscribe on YouTube (Free) ▶ Watch Short (60s) ↗
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Restarting a crashing container without examining preceding termination logs via kubectl logs --previous when configuring CrashLoopBackOff Diagnostic Root Restarting the pod flushes ephemeral container runtime state, destroying vital panic stack traces and fatal initialization logs.
  2. Misdiagnosing an application boot crash as a failure of aggressive liveness probes when configuring CrashLoopBackOff Diagnostic Root Liveness probes with short timeouts or high failure thresholds kill slow-starting applications repeatedly, causing false CrashLoopBackOff loops.
  3. Neglecting node taints, tolerations, and resource quotas when diagnosing pods stuck in Pending when configuring CrashLoopBackOff Diagnostic Root Pods remain Pending when scheduler predicates fail; kubectl describe pod reveals whether CPU/memory exhaustion or taints prevent placement.
  4. Invoking kubectl delete pod --force --grace-period=0 indiscriminately on unready pods when configuring CrashLoopBackOff Diagnostic Root Forced deletion bypasses kubelet cleanup and can cause multi-attach errors on PersistentVolumes (PVs) or split-brain on stateful workloads.
Advertisement
📥 FREE DOWNLOAD · 101-PAGE COMPANION HANDBOOK
Studying for Kubernetes & SRE Technical Rounds?
Download the complete 100-question PDF field guide covering all 11 core modules with offline diagnostic runbooks.
📥 Download PDF (Free) Read Online Guide →
Want more Kubernetes scenarios?
Explore our complete collection of scenario-based Kubernetes interview runbooks.
Browse All Kubernetes Questions →