Q: A developer fills out a Backstage Scaffolder form to create a new microservice. The UI displays 'Task Running...' for 20 minutes before timing out with 'Task failed: execution timed out'. The GitHub repository was never created. How do you debug Backstage Scaffolder task execution?
Diagnosing and resolving Backstage Scaffolder background job hangs caused by missing secrets and unhandled task runner timeouts.
Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Inspect Scaffolder Task Logs via API and Database
Query the Backstage API for the failed task ID to retrieve granular step execution logs that may not render in the frontend.
curl -s -H "Authorization: Bearer $TOKEN" \
https://backstage.acme.com/api/scaffolder/v2/tasks/<task-id> | jq .Verify GitHub Token Secret Mounting in Backstage Pod
Confirm that the Backstage backend pod has access to `GITHUB_TOKEN`. If using Kubernetes secrets via `envFrom`, verify that the secret exists and contains a valid PAT or GitHub App private key.
kubectl exec -it deployment/backstage-backend -n backstage -- env | grep GITHUB_TOKENConfigure Global Task Timeouts and Stale Task Reaper
In `app-config.yaml`, configure explicit task timeouts and enable the Scaffolder task reaper to prevent zombie jobs from exhausting database connection pools.
# app-config.yaml
scaffolder:
concurrentTasksLimit: 10
defaultTaskTimeout: { minutes: 5 }Implement Idempotent Scaffolder Actions
Ensure custom Scaffolder actions clean up temporary workspace files (`/tmp/backstage-scaffolder-*`) upon error to prevent disk exhaustion.
- Query Backstage Scaffolder task API directly to view unrendered step-level failure logs.
- Verify Git provider API credentials and secret mount permissions in the Backstage backend container.
- Set explicit concurrentTasksLimit and defaultTaskTimeout in app-config.yaml to prevent zombie jobs.