Q: Your organization has 4,500 GitHub repositories, but only 1,800 have valid catalog-info.yaml files in Backstage. How do you detect 'ghost services' actively running in Kubernetes clusters that lack catalog ownership, and automate catalog adoption?
Detecting and remediating orphaned repositories, decommissioned infrastructure, and shadow services missing catalog-info.yaml metadata.
Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: Catalog drift creates dangerous security and operational blind spots when production services have no registered on-call team or Slack channel. Remediation requires correlating active Kubernetes Deployment labels with Backstage catalog entities and enforcing automated admission gates.
Correlate Kubernetes Runtime Deployments with Backstage API
Run a nightly reconciliation script querying the Kubernetes API across all clusters. Extract app.kubernetes.io/name from active deployments and query Backstage's /api/catalog/entities/by-name/component/default/{name}. Any cluster deployment missing from Backstage is flagged as an unregistered ghost service.
# Query Backstage API to verify registration
curl -s -H "Authorization: Bearer $BACKSTAGE_TOKEN" \
https://backstage.acme.com/api/catalog/entities/by-name/component/default/checkout-api | jq .metadata.name
Automate Catalog-Info Generation via GitHub PRs
For repositories without catalog files, run an automated script that inspects package.json / pom.xml / go.mod and git commit history to infer service name and primary contributors, opening automated PRs containing a valid catalog-info.yaml.
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: checkout-api
description: Checkout core transaction service
spec:
type: service
lifecycle: production
owner: team-checkout
Enforce Admission Webhook for Production Namespaces
Deploy a Kyverno / OPA policy in production clusters blocking deployment manifests that do not include the annotation backstage.io/kubernetes-id matching a verified registered service in Backstage.
- Run automated cron auditors comparing active cluster deployment names against Backstage entity APIs.
- Generate automated pull requests creating default catalog-info.yaml files based on repository commit history.
- Enforce production Kyverno policies requiring verified Backstage catalog ownership annotations.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Engineering bespoke in-house developer tools instead of adopting standardized platform Golden Paths when configuring Service Catalog Drift Bespoke internal tools create long-term maintenance debt and fragmented workflows, whereas standardized Golden Paths accelerate delivery.
- Overlooking infrastructure cost visibility and FinOps allocation tagging across multi-tenant environments when configuring Service Catalog Drift Without granular resource tagging, engineering organizations cannot attribute cloud infrastructure costs to specific teams or business units.
- Failing to define explicit Service Level Objectives (SLOs) and Error Budgets to balance feature velocity with reliability when configuring Service Catalog Drift Without agreed SLOs, engineering and product teams lack an objective data-driven framework to prioritize technical debt over new features.
- Neglecting automated continuous drift detection between declarative configuration and deployed cloud state when configuring Service Catalog Drift Unmanaged drift accumulates over time, causing future automated deployments to fail or overwrite emergency manual adjustments.