Q: How do you design High Availability and Autoscaling in Kubernetes?
Production engineering framework for building highly available, resiliently autoscaling Kubernetes workloads: Multi-AZ node spreading, PodDisruptionBudgets (PDB), podAntiAffinity, TopologySpreadConstraints, HPA, and Karpenter integration.
Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: I design Kubernetes HA by systematically removing single points of failure across both the control plane and data plane: Multi-AZ node groups, multiple workload replicas, PodDisruptionBudgets (PDBs), TopologySpreadConstraints across availability zones, and pod anti-affinity. Autoscaling is designed in two complementary tiers: horizontal pod scaling via HPA based on CPU/memory and custom metrics, paired with node autoscaling via Karpenter or Cluster Autoscaler to provision compute capacity dynamically.
Multi-AZ Workload Distribution & Disruption Budgets
Ensure zero downtime during node failures, AZ outages, and cluster maintenance:
# High-availability Deployment topology constraints
spec:
replicas: 6
template:
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: payment-service
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchLabels:
app: payment-service
topologyKey: kubernetes.io/hostname
- TopologySpreadConstraints: Enforce even pod distribution across availability zones using
topologyKey: topology.kubernetes.io/zonewithmaxSkew: 1. - Pod Anti-Affinity: Ensure pods of the same deployment do not land on the same physical worker node to prevent a single node crash from killing multiple replicas.
- PodDisruptionBudgets (PDB): Define
minAvailable: 60%ormaxUnavailable: 1so node draining and upgrades never violate minimum application capacity.
Two-Tier Autoscaling: HPA & Node Provisioning (Karpenter)
Coordinate application pod demand with physical cluster compute provisioning:
# HPA with scale-down stabilization
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: payment-hpa
namespace: prod
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: payment-service
minReplicas: 3
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
behavior:
scaleDown:
stabilizationWindowSeconds: 300
- HorizontalPodAutoscaler (HPA): Configure target CPU utilization (typically 70%) and custom business metrics (e.g. SQS queue depth or HTTP requests/sec via Prometheus Adapter).
- Stabilization Windows: Set scale-down stabilization windows (e.g., 300s) to prevent pod flapping/thrashing during erratic traffic bursts.
- Karpenter / Cluster Autoscaler: When HPA scales up pods and nodes run out of allocatable CPU/RAM, Karpenter detects the pending pods and launches right-sized EC2 nodes in under 45 seconds.
- Distribute pod replicas across availability zones using TopologySpreadConstraints and prevent co-location with podAntiAffinity.
- Protect production capacity during node upgrades and drain events with PodDisruptionBudgets (PDB).
- Couple HPA for workload scaling with Karpenter for sub-minute node provisioning and cost-efficient bin-packing.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Attempting multi-minor Kubernetes version jumps directly during cluster upgrades when configuring Designing High Availability Kubernetes explicitly does not support multi-minor version jumps; control plane and worker components require sequential hops to validate API deprecations.
- Upgrading kube-apiserver while leaving critical cluster add-ons and CNI plugins desynchronized when configuring Designing High Availability Upgrading kube-apiserver without synchronizing CNI daemonsets or CoreDNS manifests causes silent pod networking failure and DNS timeouts immediately post-upgrade.
- Replacing worker node pools simultaneously rather than adopting sequential rolling node group updates when configuring Designing High Availability In-place or simultaneous node pool replacements risk cluster-wide capacity starvation if new AMIs or kubelet versions encounter bootstrapping errors.
- Overlooking terminationGracePeriodSeconds when evicting stateful database or queue consumer pods when configuring Designing High Availability Prematurely terminating stateful pods before in-flight transactions commit causes data corruption and unclean lock releases.