⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Docker & Containers Interview Questions Scenario 122 of 159 in Docker & Containers
Senior DevOps Engineer Docker Container Runtime & Systems Engineering Production Scenario
🎯 Target Role / Context: Principal Docker Architect / Staff SRE Scenario

Q: A Kubernetes worker node loses connectivity with the control plane and kubelet becomes unresponsive. SREs logging into the node attempt to run `docker ps` but realize the node runs containerd. Running `ctr containers list` shows dozens of obscure UUIDs with no pod names. You must guide the team on how to use `crictl` for Kubernetes pod-aware triage, `nerdctl` for Docker-compatible developer tooling, and low-level `ctr` to directly inspect containerd runtime internals.

Master node-level container debugging using crictl, nerdctl, and ctr. Troubleshoot Pod sandboxes, pull images directly into the k8s.io namespace, and inspect container PID/cgroup states during kubelet outages.

#Docker #containerd #crictl #nerdctl #Kubernetes #Comparisons
🎙️ Candidate Opening & Architectural Context
"Master node-level container debugging using crictl, nerdctl, and ctr. Troubleshoot Pod sandboxes, pull images directly into the k8s.io namespace, and inspect container PID/cgroup states during kubelet outages."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's Docker Certified Associate (DCA) Hands-On Lab Course covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: Master node-level container debugging using crictl, nerdctl, and ctr. Troubleshoot Pod sandboxes, pull images directly into the k8s.io namespace, and inspect container PID/cgroup states during kubelet outages.

Step 1

Master the CRI Tooling Landscape: crictl vs nerdctl vs ctr

Map tool roles: crictl is the Kubernetes CRI validation and debugging tool; it understands Pods, Pod sandboxes, and K8s namespaces. ctr is containerd's bare-bones engineering CLI. nerdctl is the modern containerd-native drop-in replacement for Docker CLI.

# Tool Summary:
# crictl  -> CRI client, speaks to /run/containerd/containerd.sock (K8s pod-aware)
# nerdctl -> Docker-compatible CLI for containerd (supports compose, rootless, buildkit)
# ctr     -> Low-level containerd internal client (requires explicit -n <namespace>)
Pro Tip: Master the CRI Tooling Landscape: crictl vs nerdctl vs ctr
Step 2

Debug Pod Sandboxes and Containers Using crictl

Configure /etc/crictl.yaml and inspect pod sandboxes, container states, and container runtime logs directly from the node without requiring a functioning kubelet or api-server.

# Configure crictl endpoint
cat <<EOF | sudo tee /etc/crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
EOF

# List Pod Sandboxes
crictl pods

# Inspect pod sandbox network status and IP
crictl inspectp <POD_ID> | jq .status.network

# Inspect failed container logs directly from node disk
crictl logs <CONTAINER_ID>
Pro Tip: Debug Pod Sandboxes and Containers Using crictl
Advertisement
Step 3

Pull and Debug Images in the k8s.io Namespace with nerdctl

Use nerdctl to pull private images directly into the Kubernetes containerd namespace (k8s.io) and execute interactive debugging containers sharing the pod's network namespace.

# Pull image directly into Kubernetes namespace
nerdctl -n k8s.io pull registry.internal/debug-tools:latest

# Run interactive container inside the existing Kubernetes pod network namespace
nerdctl -n k8s.io run --rm -it \
  --net=container:<CONTAINER_ID> \
  nicolaka/netshoot
Pro Tip: Pull and Debug Images in the k8s.io Namespace with nerdctl
Step 4

Triage Low-Level containerd Tasks with ctr

When CRI is completely deadlocked, use ctr to check whether the underlying containerd daemon and runc tasks are alive.

# Check containerd tasks in k8s.io namespace
ctr -n k8s.io tasks list

# Check containerd plugins health
ctr plugins list | grep -E 'cri|io.containerd'
Pro Tip: Triage Low-Level containerd Tasks with ctr
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"crictl is the proper CLI for triaging Kubernetes nodes because it understands Pod sandboxes and CRI abstractions. nerdctl provides a Docker-compatible UX atop containerd, while ctr provides raw runtime access when the CRI plugin itself is unresponsive."
⚡ 60-Second Elevator Pitch Talking Points
  • W
  • h
  • e
  • n
  • t
  • r
  • i
  • a
  • g
  • i
  • n
  • g
  • c
  • o
  • n
  • t
  • a
  • i
  • n
  • e
  • r
  • d
  • -
  • b
  • a
  • s
  • e
  • d
  • K
  • u
  • b
  • e
  • r
  • n
  • e
  • t
  • e
  • s
  • n
  • o
  • d
  • e
  • s
  • d
  • u
  • r
  • i
  • n
  • g
  • c
  • o
  • n
  • t
  • r
  • o
  • l
  • p
  • l
  • a
  • n
  • e
  • o
  • u
  • t
  • a
  • g
  • e
  • s
  • ,
  • w
  • e
  • r
  • e
  • l
  • y
  • o
  • n
  • `
  • c
  • r
  • i
  • c
  • t
  • l
  • `
  • t
  • o
  • i
  • n
  • s
  • p
  • e
  • c
  • t
  • p
  • o
  • d
  • s
  • a
  • n
  • d
  • b
  • o
  • x
  • e
  • s
  • a
  • n
  • d
  • c
  • o
  • n
  • t
  • a
  • i
  • n
  • e
  • r
  • s
  • t
  • a
  • t
  • e
  • s
  • w
  • i
  • t
  • h
  • o
  • u
  • t
  • d
  • e
  • p
  • e
  • n
  • d
  • i
  • n
  • g
  • o
  • n
  • k
  • u
  • b
  • e
  • l
  • e
  • t
  • .
  • F
  • o
  • r
  • c
  • o
  • m
  • p
  • l
  • e
  • x
  • t
  • r
  • o
  • u
  • b
  • l
  • e
  • s
  • h
  • o
  • o
  • t
  • i
  • n
  • g
  • ,
  • `
  • n
  • e
  • r
  • d
  • c
  • t
  • l
  • `
  • a
  • l
  • l
  • o
  • w
  • s
  • d
  • e
  • v
  • e
  • l
  • o
  • p
  • e
  • r
  • s
  • t
  • o
  • r
  • u
  • n
  • n
  • e
  • t
  • s
  • h
  • o
  • o
  • t
  • c
  • o
  • n
  • t
  • a
  • i
  • n
  • e
  • r
  • s
  • s
  • h
  • a
  • r
  • i
  • n
  • g
  • p
  • o
  • d
  • n
  • e
  • t
  • w
  • o
  • r
  • k
  • n
  • a
  • m
  • e
  • s
  • p
  • a
  • c
  • e
  • s
  • ,
  • w
  • h
  • i
  • l
  • e
  • `
  • c
  • t
  • r
  • `
  • s
  • e
  • r
  • v
  • e
  • s
  • a
  • s
  • o
  • u
  • r
  • l
  • a
  • s
  • t
  • -
  • r
  • e
  • s
  • o
  • r
  • t
  • t
  • o
  • o
  • l
  • f
  • o
  • r
  • v
  • e
  • r
  • i
  • f
  • y
  • i
  • n
  • g
  • u
  • n
  • d
  • e
  • r
  • l
  • y
  • i
  • n
  • g
  • r
  • u
  • n
  • c
  • t
  • a
  • s
  • k
  • s
  • .
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Structuring Dockerfile instructions inefficiently by copying app source before package manifests when configuring Container Diagnostics crictl Copying code before package manifests (package.json, go.mod, requirements.txt) invalidates the build cache on every commit, forcing slow full reinstalls.
  2. Neglecting the PID 1 process reap problem and omitting proper SIGTERM signal forwarding when configuring Container Diagnostics crictl If an application runs as PID 1 without an init system (like tini), it fails to reap zombie child processes and ignores SIGTERM signals, causing unclean 10s timeouts.
  3. Assembling single-stage images containing compilers, build SDKs, and header files in production when configuring Container Diagnostics crictl Single-stage builds bloat image size by gigabytes and expose extensive package vulnerabilities; multi-stage builds isolate artifacts into minimal distroless images.
  4. Confusing container port forwarding on bridge networks with unisolated host networking mode when configuring Container Diagnostics crictl Bridge networking utilizes iptables NAT translations which add latency overhead, whereas host networking bypasses the network namespace entirely.
Advertisement
Want more Docker & Containers scenarios?
Explore our complete collection of scenario-based Docker & Containers interview runbooks.
Browse All Docker & Containers Questions →