Q: Pod shows `ErrImagePull`.
Image not found or wrong tag. Check image name/tag. For private registry, ensure imagePullSecret is configured.
Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: A pod that shows <strong>ErrImagePull</strong> means the kubelet tried to pull the container image and failed; after retries the status flips to <code>ImagePullBackOff</code>, which is the same problem with an exponential delay (up to 5 minutes). The fix is always in the event message: wrong name or tag, missing credentials, registry rate limit, network or DNS trouble. Interviewers ask this to see if you start from <code>kubectl describe pod</code> rather than guessing.
Read the exact ErrImagePull error first
The Events section names the failing image and the registry's reason.
kubectl describe pod my-app-6f7d9-abcde -n prod | sed -n '/Events:/,$p'
# Failed to pull image "registry.example.com/my-app:1.4.O":
# rpc error: code = NotFound ... not found
kubectl get pod my-app-6f7d9-abcde -n prod \
-o jsonpath='{.spec.containers[*].image}{"\n"}'
- Describe:
kubectl describe pod <pod> -n prodand read the lastFailedevents. - Typical messages:
manifest unknownornot found(bad tag),unauthorizedorpull access denied(credentials),toomanyrequests(rate limit),no such hostor timeout (DNS or network). - Namespace events:
kubectl get events -n prod --sort-by=.lastTimestamp. - Spec image: confirm the image string in the pod spec, not in your Git repo.
Fix credentials with imagePullSecrets
Private registries need a Secret of type kubernetes.io/dockerconfigjson in the pod's namespace.
kubectl create secret docker-registry regcred -n prod \
--docker-server=registry.example.com \
--docker-username=ci-bot --docker-password="$REGISTRY_TOKEN"
kubectl patch serviceaccount default -n prod \
-p '{"imagePullSecrets":[{"name":"regcred"}]}'
kubectl rollout restart deploy/my-app -n prod
- Create the Secret:
kubectl create secret docker-registrywith registry server, user and token. - Reference it: add
imagePullSecretsto the pod spec or patch the ServiceAccount so all pods in the namespace use it. - Namespace scope: Secrets do not cross namespaces; create one per namespace.
- Cloud registries: ECR tokens expire after 12 hours, so use node IAM roles, kubelet credential providers or a refresh job.
Verify from the node, not only the API
Reproduce the pull exactly where the kubelet runs.
kubectl get pod my-app-6f7d9-abcde -n prod -o wide
# on the node:
sudo crictl pull registry.example.com/my-app:1.4.0
getent hosts registry.example.com
docker manifest inspect registry.example.com/my-app:1.4.0 >/dev/null && echo exists
- Which node:
kubectl get pod -o wideshows the node. - CRI pull: on a containerd node run
crictl pull <image>to see the registry response. - DNS and egress: check the node resolves and reaches the registry (proxy, firewall, NAT, VPC endpoint).
- Image exists: query the registry for the tag directly, for example
crane manifestordocker manifest inspect.
Less obvious causes
When the name and credentials look right:
- Rate limits: anonymous Docker Hub pulls are limited per IP; authenticate or use a registry mirror or pull-through cache.
- Architecture: a
no matching manifest for linux/arm64message means the image lacks that platform, common on mixed Graviton and x86 clusters. - Digest pins and tags: a tag deleted by a retention policy returns
not foundthough it worked yesterday. - Policy:
imagePullPolicy: Alwaysforces a registry contact on every start; a registry outage then blocks restarts. - TLS: private CAs must be trusted by the container runtime on every node.
Production practice: prevent pull failures
Treat image delivery as a dependency.
- Pin versions: deploy immutable tags or digests built by CI; never rely on
latest. - Pre-flight: CI verifies the tag exists in the registry before applying manifests.
- Mirror: use a pull-through cache or in-region registry for resilience and speed.
- Alert: alert on pods in
ImagePullBackOffvia kube-state-metricskube_pod_container_status_waiting_reason.
- ErrImagePull means the kubelet failed to pull the image; ImagePullBackOff is the same failure being retried with backoff.
- I read <code>kubectl describe pod</code> events, then confirm the tag, credentials and reachability, testing with <code>crictl pull</code> on the node.
- I prevent it with pinned tags, a CI tag-exists check, pull-through caching and an alert on waiting reason.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Treating ImagePullBackOff as a different problem from ErrImagePull It is the same pull failure with a growing retry delay; the cause is in the original event.
- Creating the pull secret in the wrong namespace imagePullSecrets are namespaced; a secret in default does nothing for pods in prod.
- Debugging only from your laptop Your workstation has different credentials, DNS and network than the node; reproduce on the node.
- Ignoring architecture mismatch A single-platform image fails on nodes of another CPU architecture with a manifest error, not an auth error.