Q: Your monitoring alerts trigger high file descriptor (FD) and process table exhaustion on several Kubernetes worker nodes. Running `ps aux | grep shim` reveals over 1,400 running `containerd-shim-runc-v2` processes, even though the node only hosts 40 active pods. Kubelet cannot schedule new containers. You must analyze why containerd creates a dedicated shim per container, triage the orphaned shims and leaked UNIX domain sockets, and restore the node to a clean operating state.

In OCI container runtimes, `runc` executes the container setup and exits. Without a shim, the daemon would have to remain the parent process of every container, preventing daemon restarts. The `shim-v2` stays alive to: act as sub-reaper for container processes, hold container PTY and stdio pipes open, report exit status codes, and allow the daemon to restart without terminating workloads.

<!-- containerd Architecture Flow -->
kubelet / docker CLI
  └── containerd (Daemon)
        └── containerd-shim-runc-v2 (1 per Pod/Container)
              ├── Holds stdio FIFOs & Unix Sockets
              ├── Monitors exit code
              └── runc (Runs setup, launches App, then exits!)
                    └── Application Process (Child of Shim)

containerd communicates with shims via TTRPC (a lightweight gRPC alternative optimized for low-memory environments) over local UNIX domain sockets located in `/run/containerd/io.containerd.runtime.v2.task/`.

# Inspect active shim sockets
ls -l /run/containerd/io.containerd.runtime.v2.task/k8s.io/

# Inspect open file descriptors of a shim process
lsof -p $(pgrep -f containerd-shim-runc-v2 | head -n 1)

When a container process crashes violently or experiences an unhandled kernel fault, the shim may fail to clean up its state, remaining stuck in the process table. Identify shims whose child processes no longer exist.

# Find shims without any child processes
for shim_pid in $(pgrep -f containerd-shim-runc-v2); do
    CHILDREN=$(pgrep -P $shim_pid | wc -l)
    if [ "$CHILDREN" -eq 0 ]; then
        echo "Orphaned shim detected: PID $shim_pid"
        # Inspect shim working directory
        ls -l /proc/$shim_pid/cwd
    fi
done

# Safely kill orphaned shims
# kill -9 <ORPHAN_PID>

Remove stale task directories from `/run/containerd` and purge orphaned FIFOs using `ctr tasks delete` to reclaim leaked file descriptors.

# Delete dead tasks directly in containerd
ctr -n k8s.io tasks delete -f <TASK_ID>

# Verify shim count returns to matching active pod count
pgrep -c -f containerd-shim-runc-v2