Q: Your readiness probe keeps failing even though the app is working fine. What could be wrong?
1. Wrong port or path — probe is checking a different port/endpoint than the app actually serves.
Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: "In one of our high-traffic production clusters, our SRE team handled this incident using a standardized runbook. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix."
Initial Diagnostics & Root Cause Analysis
Debug: kubectl describe pod shows probe failure reason. Also exec into the pod and manually curl the readiness endpoint to see what it returns.
- Wrong port or path — probe is checking a different port/endpoint than the app actually serves.
- Probe timeout too short — if the app takes 2 seconds to respond and
timeoutSeconds: 1, it always times out. - App returns non-200 for the health path under load — the readiness endpoint has a bug.
Remediation & Permanent Safeguards
- initialDelaySeconds too short — app needs more time to start before readiness checks begin.
- Checking the wrong container port name — if using
port: httpand the port isn't named, it fails.
- Wrong port or path — probe is checking a different port/endpoint than the app actually serves.
- Probe timeout too short — if the app takes 2 seconds to respond and timeoutSeconds: 1, it always ...
- App returns non-200 for the health path under load — the readiness endpoint has a bug.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Over-granting cluster-admin permissions to pod ServiceAccounts instead of scoping namespace Roles when configuring readiness probe keeps Over-granting cluster-wide permissions violates least-privilege principles and allows compromised pods to escalate privileges across namespaces.
-
Referencing mutable latest container tags across production Kubernetes Deployment specs when configuring readiness probe keeps
Deployments referencing
:latestrisk inconsistent versions across replica pods and break deterministic rollback capabilities. - Omitting pod readiness probes, causing traffic to hit uninitialized application containers when configuring readiness probe keeps Readiness probes gate traffic delivery; missing probes cause Kubernetes to route user requests to pods before dependencies or caches are loaded.
- Executing diagnostic triage without analyzing chronological cluster events and describe outputs when configuring readiness probe keeps Cluster events provide sequential timestamps of scheduler decisions, probe failures, and image pull timeouts essential for rapid root-cause isolation.