Q: NodePort service not reachable from outside.
Check firewall/security group rules allow the NodePort (30000-32767) range.
Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: A <strong>NodePort service not reachable from outside</strong> almost always fails in one of four places: the Service has no endpoints, kube-proxy has not programmed the node port, a firewall or cloud security group blocks the port, or you are using an address that is not routable from your client. Test from the inside out: pod, ClusterIP, node itself, then external client. Interviewers ask this because it tests whether you understand that a NodePort is only a port opened on every node, forwarded by kube-proxy rules to pod endpoints.
Symptom and the Four Layers to Check
Reachability is a chain; find the first broken link instead of changing things at random.
- Layer 1, Service and endpoints: does the Service select ready pods, and does
targetPortmatch the container port? - Layer 2, node dataplane: has kube-proxy (iptables, IPVS or nftables mode) created rules for the node port on that node?
- Layer 3, network path: does a host firewall, cloud security group, NACL or on-prem firewall allow the port from your client to the node IP?
- Layer 4, addressing and policy: is the node IP reachable from the client, and does
externalTrafficPolicyor a NetworkPolicy change which pods answer? - What the rule does: kube-proxy matches traffic to any node IP on the node port and rewrites it to a pod endpoint, which may be on another node; the Service object itself runs no process.
Hands-on: NodePort Service Not Reachable From Outside
Run these in order. The first one that fails is your root cause.
kubectl get svc my-app -n prod -o wide # TYPE NodePort, PORT(S) 80:30080/TCP
kubectl get endpointslices -n prod -l kubernetes.io/service-name=my-app
kubectl get pods -n prod -l app=my-app -o wide # note node names and IPs
# From inside the cluster
kubectl run tmp --rm -it --image=curlimages/curl --restart=Never -- \
curl -sv http://my-app.prod.svc.cluster.local:80
# On a node (SSH or SSM)
curl -sv http://127.0.0.1:30080
sudo iptables -t nat -S KUBE-NODEPORTS | grep 30080 # iptables mode
# From your laptop
curl -sv --max-time 5 http://<node-external-ip>:30080
- Endpoints: an empty ENDPOINTS list means a selector or readiness problem, not a network problem.
- Inside the cluster: if the ClusterIP works from a debug pod but the NodePort does not, the problem is outside the Service object.
- On the node:
curltolocalhost:<nodePort>on the node isolates host rules from the external network path.
Root Causes and Fixes
Each failure has a signature that points to the fix.
- Empty endpoints: selector does not match pod labels, or pods fail readiness. Fix labels or the probe; the NodePort cannot work with no backends.
- Wrong targetPort: the Service forwards to a port the container does not listen on, giving connection refused or reset. Align
targetPortwith the container port or its name. - Firewall or security group: a timeout (no response) points here. On AWS, the node security group must allow inbound TCP on the node port; the default range is 30000-32767 unless changed with
--service-node-port-range. a timeout points to a firewall or routing problem, while connection refused points to a reachable host with nothing listening on the target, which is typically a targetPort or pod issue. - Private node IP: using an internal address from the internet times out. Use the node's public address, a bastion, or put a LoadBalancer or Ingress in front.
- externalTrafficPolicy: Local: only nodes that run a ready pod answer; other nodes drop the traffic. Query a node that hosts a pod, or use
Clusterand accept source IP masquerading.
Verification and Special Environments
Confirm the fix from the same place that failed.
# Which mode is kube-proxy running, and are there rules for the port?
kubectl -n kube-system get cm kube-proxy -o jsonpath='{.data.config\.conf}' | grep -E 'mode|nodePortAddresses'
sudo ipvsadm -Ln | grep 30080 # ipvs mode
# AWS: is the port open on the node security group?
aws ec2 describe-security-groups --group-ids sg-0123456789abcdef0 \
--query 'SecurityGroups[].IpPermissions[?FromPort<=`30080` && ToPort>=`30080`]'
- Retest externally: a successful
curl -svfrom the original client is the only real proof; node-local success is not enough. - Local clusters: on minikube, kind or Docker Desktop the node IP is inside a VM or Docker network. Use
minikube service my-app --url, extra port mappings in kind, orkubectl port-forward. - kube-proxy health:
kubectl -n kube-system get pods -l k8s-app=kube-proxyand its logs reveal a crashed or misconfigured proxy; some CNIs replace kube-proxy entirely, so rules look different. - NetworkPolicy: a policy selecting the pods can block the traffic; remember NodePort traffic may arrive with the node's IP after SNAT.
Production Practice: Prefer Load Balancer or Ingress
NodePort is a building block, not an internet-facing design.
- Front with a load balancer: use a LoadBalancer Service or Ingress/Gateway so clients do not depend on individual node IPs that change with autoscaling.
- Restrict exposure: limit the source CIDR in security groups and use
nodePortAddresseson kube-proxy if only specific interfaces should listen. - Pin the port deliberately: set
nodePortexplicitly only when an external system needs it, and document it to avoid collisions.
- A NodePort opens one port on every node and kube-proxy forwards it to the Service endpoints.
- I check endpoints, ClusterIP from a debug pod, curl on the node, then the external path and security groups.
- In production I avoid raw NodePort for public traffic and use a load balancer or Ingress with restricted source ranges.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Debugging firewalls before checking endpoints A Service with no ready endpoints fails no matter how open the network is.
- Confusing port, targetPort and nodePort Traffic is forwarded to targetPort on the pod, so a mismatch gives refused connections that look like network issues.
- Using the node's private IP from outside the VPC That address is not routable from the internet, so every test times out regardless of cluster health.
- Ignoring externalTrafficPolicy: Local Nodes without a local pod drop the traffic, so the Service works on some nodes and not others.