Q: A production playbook run was aborted midway due to a network timeout. How does Ansible handle partial execution, and how do you guarantee idempotency when re-running?
Ansible is stateless and maintains no persistent backend state file. If a run drops midway, already executed tasks remain in their target state. Re-running the playbook relies on module idempotency to report ok and only apply pending tasks.
Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Understand Ansible's Stateless Execution Model
Unlike Terraform which tracks state in a JSON file, Ansible evaluates state directly on the remote system during each task execution. Tasks already applied before the timeout will evaluate to ok (no changes needed), while incomplete tasks will execute and report changed.
- No State Lock: There is no lock file preventing an immediate re-run once network connectivity is restored.
- Host List Filtering: Ansible generates a
.retryfile (if enabled) listing only the hosts that failed to complete.
Guard Custom Shell/Command Tasks
Declarative modules (package, file, service) are inherently idempotent. However, imperative shell tasks must be safeguarded to prevent duplicate side effects:
- Use
creates: /opt/app/installed.lockto skip execution if artifact already exists. - Use
removes: /tmp/setup.shto execute only if cleanup file is present. - Use
register: resultand check return codes before mutating state.
Validate and Re-Execute Playbook
Before full execution, execute in dry-run mode to inspect anticipated delta:
ansible-playbook -i inventory site.yml --check --diff- Verify that already completed steps report green (
ok) and only unfinished steps report yellow (changed). - Execute
ansible-playbook -i inventory site.yml --limit @site.retryto target only incomplete hosts.
- Ansible is inherently stateless — it queries the managed node's real-time configuration rather than reading a static state file.
- Tasks completed before the network timeout remain applied; declarative modules (service, file, package) safely report 'ok'.
- Guard imperative command and shell tasks using 'creates' or 'removes' arguments to prevent duplicate execution.
- Execute 'ansible-playbook site.yml --check --diff' to verify delta before committing changes to production.
- Re-run against failed nodes using the auto-generated retry file: '--limit @site.retry'.
- Implement automated pre-commit linting (ansible-lint) to block non-idempotent module patterns across CI pipelines.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Assuming Ansible has an automated rollback mechanism on task failure Ansible halts execution on failure; it does NOT roll back previous tasks automatically unless custom block/rescue structures are declared.
- Using raw shell modules to run echo or append commands without idempotency checks Appends duplicate configuration lines every time the playbook is retried, causing invalid syntax in config files.
- Re-running playbooks blindly without running --check --diff first Senior engineers always preview pending mutations during incident recovery to avoid race conditions.