Q: A CronJob job ran but the pod isn't showing in `kubectl get jobs`.
successfulJobsHistoryLimit may be 0 or 1 and old jobs were cleaned. Adjust to keep history.
Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: <strong>When a CronJob job ran but the pod isn't showing in kubectl get jobs, the first correction is that <code>kubectl get jobs</code> lists Job objects, not pods.</strong> A missing entry usually means the Job was garbage-collected (history limits or TTL), you are in the wrong namespace, or the CronJob never created a Job at all. Interviewers use this to test whether you know the CronJob to Job to Pod ownership chain and where each layer is cleaned up. The answer is a short diagnostic runbook: confirm the CronJob fired, find the Job, find the pod by label, then read events and logs.
Mental model: CronJob creates Job creates Pod
A CronJob does not run containers. It creates a Job on schedule, and the Job controller creates pods. Each layer has its own retention rules.
- CronJob: records
lastScheduleTimeandlastSuccessfulTimein status. If these are empty or old, no Job was created and you debug the schedule, not the pod. - Job: kept according to
successfulJobsHistoryLimit(default 3) andfailedJobsHistoryLimit(default 1). Older Jobs and their pods are deleted by the CronJob controller. - Pod: belongs to the Job through ownerReferences and carries the label
job-name=<job>(newer clusters also setbatch.kubernetes.io/job-name). Pods vanish when the Job is deleted or whenttlSecondsAfterFinishedexpires. - Different commands:
kubectl get jobsshows Jobs,kubectl get podsshows pods. Completed pods showCompletedstatus and are hidden by some UIs and by--field-selector=status.phase=Running. - Ownership check: run
kubectl get job <name> -o jsonpath='{.metadata.ownerReferences[0].name}'to prove which CronJob created a Job, which helps when several CronJobs share similar name prefixes in one namespace.
Hands-on: pod isn't showing in kubectl get jobs, find the run
Work down the chain in the right namespace and widen the search before concluding the job did not run.
kubectl -n prod get cronjob my-report -o wide
kubectl -n prod describe cronjob my-report | sed -n '/Events/,$p'
# Jobs created by this CronJob (newest last)
kubectl -n prod get jobs --sort-by=.metadata.creationTimestamp | grep my-report
# Pods for a specific Job, including Completed ones
kubectl -n prod get pods -l job-name=my-report-28774000 -o wide
# Events survive a little longer than the objects (default ~1h)
kubectl -n prod get events --sort-by=.lastTimestamp | grep my-report
- Namespace first: use
-nor-A. CronJob objects and their Jobs always live in the same namespace. - CronJob status:
kubectl describe cronjobshows last schedule time, active Jobs and Events such asSuccessfulCreateorMissSchedule. - All Jobs for it: Jobs are named
<cronjob>-<timestamp>, so grep by prefix, and sort by creation time. - Pods by label: if the Job exists, list pods through the
job-namelabel rather than guessing names. - Time window: compare
lastScheduleTimewith the current time and the cron expression. Cron schedules use the controller-manager's time zone unlessspec.timeZoneis set, so a job may have run at a different hour than you expected.
Root causes when the Job or pod is gone
Match what you see to a cause. Each one has a different fix.
- History limit cleanup: with
successfulJobsHistoryLimit: 0the Job is deleted as soon as it finishes, so it can never be listed afterwards. - TTL after finished:
spec.jobTemplate.spec.ttlSecondsAfterFinishedremoves the Job and its pods after the set seconds, which makes logs unrecoverable unless shipped elsewhere. - Never created:
suspend: true, a wrongscheduleortimeZone,concurrencyPolicy: Forbidwhile a previous run is stuck, orstartingDeadlineSecondsexceeded after controller downtime. - Created but no pod: a ResourceQuota, LimitRange or admission webhook rejecting the pod. The Job events then show
FailedCreatewith the reason, even though no pod exists. - Concurrency skips: with
concurrencyPolicy: Forbidthe controller records a skipped run in events rather than creating a Job, and withReplaceit deletes the still-running Job and its pods, which also explains pods that disappear mid-run.
Reproduce and verify
Don't wait for the next schedule. Create a Job on demand from the CronJob template and watch it.
kubectl -n prod create job --from=cronjob/my-report my-report-manual-1
kubectl -n prod wait --for=condition=complete job/my-report-manual-1 --timeout=300s
kubectl -n prod logs job/my-report-manual-1
kubectl -n prod get cronjob my-report -o jsonpath='{.status.lastSuccessfulTime}'
- Manual trigger:
kubectl create job --from=cronjob/my-report my-report-manual-1uses the same pod template, so quota or RBAC problems reproduce immediately. - Watch:
kubectl get pods -w -l job-name=my-report-manual-1and thenkubectl logs job/my-report-manual-1. - Failure states:
ImagePullBackOff,CrashLoopBackOffand exceededbackoffLimitall leave a failed Job whose pods remain until the failed history limit removes them. - Confirm the fix: after correcting the cause, check that
lastSuccessfulTimeadvances on the next scheduled run. - Evidence after failure: if the run failed and you only need to see why,
kubectl describe jobandkubectl logs job/<name> --all-containersstill work while the Job object exists, so capture output before retention expires.
Production practice
Make CronJob runs observable so you are never debugging after the evidence is deleted.
- Keep enough history: use
successfulJobsHistoryLimit: 3andfailedJobsHistoryLimit: 3or more for jobs you debug often, and avoid TTL values shorter than your alert response time. - Ship logs: send stdout to a log backend so output outlives the pod.
- Alert on absence: with kube-state-metrics, alert when
kube_cronjob_status_last_successful_timeis older than the expected interval, not only on failures. - Set deadlines: define
startingDeadlineSeconds,activeDeadlineSecondsand an explicittimeZoneso skips and hangs are predictable.
- kubectl get jobs lists Jobs, not pods, and a CronJob creates a Job which creates pods, each with its own cleanup rules.
- I check the CronJob status and events, look in the right namespace, find the Job and its pods with the job-name label, and read FailedCreate events for quota or webhook blocks.
- I automate it with generous history limits, shipped logs and an alert on stale lastSuccessfulTime.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Expecting pods in the output of kubectl get jobs The command lists only Job objects, so you must query pods separately by label.
- Setting successfulJobsHistoryLimit to 0 or a tiny TTL, then debugging later The Job and its pods are deleted immediately, so logs and events are gone.
- Ignoring concurrencyPolicy: Forbid A hung previous run silently makes the controller skip new schedules.
- Looking only at pod status for missing runs If quota or a webhook blocked creation, no pod ever existed; the evidence is in Job events.