Q: What happens to pods when you run `kubectl drain` on a node?
kubectl drain does two things:
#Kubernetes #Advanced Scenarios #L2 #Container Orchestration #K8s
🎙️ Candidate Opening & Architectural Context
""When troubleshooting Kubernetes, I always follow a structured layered model: Pod status -> Events -> Logs -> Network. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix.""
Advertisement
🛠️ Production Runbook & Step-by-Step Resolution
1️⃣
Initial Diagnostics & Root Cause Analysis
kubectl drain does two things:
- Cordons the node — marks it unschedulable so no new pods land on it.
- Evicts all pods — sends eviction requests (not delete) for every pod. Eviction respects PodDisruptionBudgets.
- DaemonSet pods (need
--ignore-daemonsetsflag to proceed).
2️⃣
Remediation & Permanent Safeguards
Exceptions — these pods are NOT evicted by default: After drain: the node is empty and safe to maintain/terminate.
- Pods with local storage (need
--delete-emptydir-dataflag). - Pods not managed by a controller (standalone pods) — drain will fail unless you force it.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Pro-Tip: Cordons the node — marks it unschedulable so no new pods land on it.."
⚡ 60-Second Elevator Pitch Talking Points
- Cordons the node — marks it unschedulable so no new pods land on it.
- Evicts all pods — sends eviction requests (not delete) for every pod. Eviction respects PodDisrup...
- DaemonSet pods (need --ignore-daemonsets flag to proceed).
Advertisement