Q: A container keeps restarting, but there are no application errors in the logs. What would you investigate next?
Troubleshooting checklist when a container repeatedly terminates and restarts while application logs remain completely empty or stop abruptly.
Want to master this scenario in a live sandbox? KodeKloud's Docker Certified Associate (DCA) Hands-On Lab Course covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: When a container restarts with empty application logs, the termination signal came from outside the application runtime—typically the Linux kernel OOM killer, Kubernetes probe failures, entrypoint shell script crashes, or cgroup memory limit violations.
Inspect Container Termination Exit Code & Reason
Check the exact exit code and termination reason using docker inspect or kubectl describe. Exit code 137 indicates SIGKILL (typically OOMKilled), while exit code 143 indicates SIGTERM (orchestrator shutdown).
kubectl describe pod <pod-name> | grep -E 'Last State|Exit Code|Reason'
# In Docker standalone:
docker inspect <container-id> --format 'ExitCode: {{.State.ExitCode}}, OOMKilled: {{.State.OOMKilled}}'
Audit Host Kernel Ring Buffer (dmesg)
Run dmesg on the host node. If the kernel Out-Of-Memory killer invoked SIGKILL, it leaves an indelible trace in dmesg showing the sacrificed process and its memory score.
dmesg -T | grep -i -E 'killed process|out of memory|oom-killer'
Verify Liveness Probe Configuration
Check if an aggressive liveness probe is timing out during heavy startup initialization. If initialDelaySeconds is too low, the kubelet kills the container before it finishes warming its cache.
- Check termination Exit Code: 137 = OOMKilled (SIGKILL), 143 = orchestrator shutdown (SIGTERM), 1 = entrypoint failure.
- Run dmesg -T | grep oom-killer on the worker node to verify kernel memory exhaustion.
- Review livenessProbe thresholds: ensure startupProbe protects slow-booting applications.
- Inspect container entrypoint script for set -e exits before logging initialized.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
-
Structuring Dockerfile instructions inefficiently by copying app source before package manifests when configuring Container Repeatedly Restarts
Copying code before package manifests (
package.json,go.mod,requirements.txt) invalidates the build cache on every commit, forcing slow full reinstalls. -
Neglecting the PID 1 process reap problem and omitting proper SIGTERM signal forwarding when configuring Container Repeatedly Restarts
If an application runs as PID 1 without an init system (like
tini), it fails to reap zombie child processes and ignoresSIGTERMsignals, causing unclean 10s timeouts. - Assembling single-stage images containing compilers, build SDKs, and header files in production when configuring Container Repeatedly Restarts Single-stage builds bloat image size by gigabytes and expose extensive package vulnerabilities; multi-stage builds isolate artifacts into minimal distroless images.
- Confusing container port forwarding on bridge networks with unisolated host networking mode when configuring Container Repeatedly Restarts Bridge networking utilizes iptables NAT translations which add latency overhead, whereas host networking bypasses the network namespace entirely.