⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Kubernetes Interview Questions Scenario 90 of 207 in Kubernetes
Senior DevOps / SRE [L2] Kubernetes Advanced Scenarios Production Scenario [L2]
🎯 Target Role / Context: Kubernetes Scenario #79 (L2) · Advanced Scenarios

Q: Horizontal Pod Autoscaler shows `unknown/50%` for current metric.

metrics-server not installed or pods have no resource requests set.

#Kubernetes #Advanced Scenarios #L2 #Container Orchestration #K8s
🎙️ Candidate Opening & Architectural Context
"When the <strong>Horizontal Pod Autoscaler shows unknown/50%</strong> as the current metric, the HPA controller cannot obtain a usable metric for the target pods, so it will not scale. The usual causes are no metrics-server (or an unhealthy metrics API), missing CPU requests on the pods, pods that are not ready yet, or a wrong target or selector. Interviewers ask this because utilization targets are relative to resource requests, a detail many engineers forget."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: When the <strong>Horizontal Pod Autoscaler shows unknown/50%</strong> as the current metric, the HPA controller cannot obtain a usable metric for the target pods, so it will not scale. The usual causes are no metrics-server (or an unhealthy metrics API), missing CPU requests on the pods, pods that are not ready yet, or a wrong target or selector. Interviewers ask this because utilization targets are relative to resource requests, a detail many engineers forget.

1️⃣

How the HPA Gets Its Metric

The HPA reads from the metrics API; the chain has several single points of failure.

  • Resource metrics: CPU and memory come from metrics-server through the metrics.k8s.io API, which is registered as an APIService.
  • Utilization is a percentage of requests: a 50% CPU target means 50% of the summed container CPU requests. No request, no percentage, so the value is unknown. If a pod has two containers and only one defines a CPU request, the HPA cannot compute pod utilization and reports the metric as unknown.
  • Custom and external metrics: these need an adapter such as prometheus-adapter or KEDA, which exposes custom.metrics.k8s.io or external.metrics.k8s.io.
  • Readiness and age: the HPA ignores recently started or unready pods in its calculation, which can briefly show unknown on a new deployment. A deployment that was just created can show unknown for the first scrape interval, usually under a minute, so wait before treating it as a fault.
  • Formula: desired replicas = ceil(current replicas x current metric / target metric), with a default tolerance of 10% around the target, which is why small fluctuations do not trigger scaling.
2️⃣

Hands-on: HPA Shows unknown/50% Diagnosis

The HPA's own conditions usually state the reason in plain text.

kubectl get hpa my-app -n prod
kubectl describe hpa my-app -n prod | sed -n '/Conditions/,/Events/p'
# Typical message: missing request for cpu in container my-app of Pod my-app-...

kubectl top pods -n prod
kubectl get apiservice v1beta1.metrics.k8s.io
kubectl -n kube-system get deploy metrics-server
kubectl -n kube-system logs deploy/metrics-server --tail=30
  • Read conditions: kubectl describe hpa lists AbleToScale and ScalingActive; a False ScalingActive with FailedGetResourceMetric names the missing piece. Look at the Events section too: repeated FailedGetResourceMetric and FailedComputeMetricsReplicas entries include the underlying API error text, such as the metrics API being unavailable.
  • Test the metrics API: kubectl top pods works only if metrics-server is healthy; an error means fix that first.
  • Check the APIService: kubectl get apiservice v1beta1.metrics.k8s.io should be Available True. If the APIService reports FailedDiscoveryCheck or MissingEndpoints, metrics-server is not reachable by the API server, often because of a webhook-style network policy or firewall between the control plane and the pod network on managed clusters.
Advertisement
3️⃣

Root Causes and the Matching Fix

Each signature has a one-step remedy.

# Deployment container
resources:
  requests:
    cpu: 200m
    memory: 256Mi

# HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-app
  namespace: prod
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50
  • Metrics-server missing or unhealthy: install it, or fix its kubelet connection. --kubelet-insecure-tls exists for lab clusters only; in production fix kubelet serving certificates. Check the version-specific install: metrics-server must run and be registered; on managed Kubernetes it is sometimes an add-on you must enable explicitly.
  • Missing CPU request: add resources.requests.cpu to every container in the pod, including sidecars, because the pod-level utilization needs all of them. On Kubernetes 1.30 and later the ContainerResource metric type can target a single container.
  • Wrong scaleTargetRef or selector: the HPA must reference the right Deployment name and kind, whose selector matches running pods.
  • Pods not ready or crashing: unready pods are excluded, so fix the pod health first.
4️⃣

Verification Under Load

Do not stop when the number appears; prove it scales.

  • Value appears: after a minute kubectl get hpa -w should show a real value such as 12%/50%.
  • Generate load: run a short load test (for example with hey or k6) and watch replicas rise and fall, remembering the scale-down stabilization window of 5 minutes by default. Compare against kubectl top pods during the test to see whether the metric the HPA uses matches what you observe, and remember that the HPA evaluates on a sync period of 15 seconds by default.
  • Events: SuccessfulRescale events confirm the controller acted.
5️⃣

Production Practice

Autoscaling is only as good as requests and metrics health.

  • Set requests from data: base them on observed usage; unrealistic requests make utilization targets meaningless. Set requests close to typical steady-state usage so 50% means a meaningful amount of headroom, not an arbitrary number.
  • Monitor the metrics pipeline: alert when the metrics APIService is unavailable or when HPA reports ScalingActive false.
  • Tune behavior: use spec.behavior to control scale-up and scale-down speed and avoid flapping.
  • Min and max sanity: if min equals max the HPA can never move, and if the target Deployment is scaled to zero by hand the HPA is disabled until replicas are above zero.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Pro-Tip: Read the HPA conditions with kubectl describe first; FailedGetResourceMetric usually says exactly whether metrics-server or a missing CPU request is the cause."
⚡ 60-Second Elevator Pitch Talking Points
  • Unknown means the HPA cannot compute utilization, usually because metrics-server is down or CPU requests are missing.
  • I describe the HPA, run kubectl top, check the metrics APIService, then confirm requests on every container.
  • I alert on the metrics API and HPA conditions and load-test to confirm scaling up and down.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Setting a CPU utilization target without CPU requests Utilization is calculated relative to requests, so there is nothing to divide by and the metric stays unknown.
  2. Forgetting a sidecar without requests One container missing a request makes the pod-level value incomputable.
  3. Using --kubelet-insecure-tls in production It disables certificate verification between metrics-server and kubelets.
  4. Expecting instant scale-down The default stabilization window delays scale-down, so waiting a few minutes is normal and not a failure.
Advertisement
📥 FREE DOWNLOAD · 101-PAGE COMPANION HANDBOOK
Studying for Kubernetes & SRE Technical Rounds?
Download the complete 100-question PDF field guide covering all 11 core modules with offline diagnostic runbooks.
📥 Download PDF (Free) Read Online Guide →
Want more Kubernetes scenarios?
Explore our complete collection of scenario-based Kubernetes interview runbooks.
Browse All Kubernetes Questions →