Q: How do you configure CPU and memory requests/limits?
Definitive architectural guide for configuring CPU and memory requests/limits: how kube-scheduler uses requests, how the Linux kernel enforces limits, CFS CPU throttling vs OOMKilled (exit 137), and QoS classes.
Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: Configuring requests and limits is not guess-work. Requests determine where the scheduler places pods; limits determine when the Linux kernel throttles or terminates them. Setting them incorrectly causes either cluster starvation or silent application slowness.
Requests vs Limits Mechanics
Understanding how the control plane and Linux kernel treat them differently:
- spec.resources.requests: The guaranteed reservation. The
kube-schedulercalculates node placement strictly based on sum of requests vs node capacity. If a node has 4 cores and 3.8 cores are requested, no new pod requesting 500m can fit. - spec.resources.limits: The hard ceiling. Enforced by Linux kernel cgroups. How the kernel responds depends on the resource type:
The 3 Kubernetes Quality of Service (QoS) Classes
Kubernetes automatically assigns QoS based on requests and limits:
- 1. Guaranteed (Highest Priority):
requests.cpu == limits.cpuANDrequests.memory == limits.memoryfor all containers. Last to be evicted when node is under resource pressure. Best for critical databases and core services. - 2. Burstable (Medium Priority): Requests are less than limits (e.g. request 500m, limit 2000m). Allowed to burst when extra node capacity exists. Standard for most web APIs.
- 3. BestEffort (Lowest Priority): No requests and no limits set. First to be killed instantly when a node experiences memory pressure.
Production Right-Sizing Best Practices
How senior SREs avoid common traps:
- Right-Sizing: Use Prometheus historical p95 usage or Vertical Pod Autoscaler (VPA in recommendation mode) to base requests on actual p95 peak load + 20% headroom.
- The CPU Limit Debate: Many high-scale teams (Google, Uber) remove CPU limits on latency-sensitive Burstable services to prevent CFS throttle penalties during micro-bursts, relying on HPA to scale out.
- Memory: Always set memory limits equal or close to requests to prevent noisy neighbors from consuming all node RAM.
- Requests: Guaranteed minimum reserved by kube-scheduler; placement depends on it.
- Limits: Maximum ceiling enforced by cgroups.
- CPU behavior: Compressible -> throttled via CFS quota (app slows down, no crash).
- Memory behavior: Incompressible -> killed immediately by OOM Killer (Exit Code 137).
- QoS Classes: Guaranteed (requests == limits, safest), Burstable (requests < limits), BestEffort (no values, evicted first).
- Best practice: Right-size with VPA recommendation mode; always set memory limits.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Restarting a crashing container without examining preceding termination logs via kubectl logs --previous when configuring Configuring Memory Requests Restarting the pod flushes ephemeral container runtime state, destroying vital panic stack traces and fatal initialization logs.
-
Misdiagnosing an application boot crash as a failure of aggressive liveness probes when configuring Configuring Memory Requests
Liveness probes with short timeouts or high failure thresholds kill slow-starting applications repeatedly, causing false
CrashLoopBackOffloops. -
Neglecting node taints, tolerations, and resource quotas when diagnosing pods stuck in Pending when configuring Configuring Memory Requests
Pods remain
Pendingwhen scheduler predicates fail;kubectl describe podreveals whether CPU/memory exhaustion or taints prevent placement. - Invoking kubectl delete pod --force --grace-period=0 indiscriminately on unready pods when configuring Configuring Memory Requests Forced deletion bypasses kubelet cleanup and can cause multi-attach errors on PersistentVolumes (PVs) or split-brain on stateful workloads.