⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Platform Engineering & IDP Interview Questions Scenario 22 of 50 in Platform Engineering & IDP
Staff Platform Engineer Platform Engineering Internal Developer Platforms & Catalogs Backstage Search
🎯 Target Role / Context: Staff Platform Engineer Loop · Platform Scalability Track

Q: As your Backstage catalog grew past 8,000 components and 25,000 TechDocs pages, developer searches began timing out. The Backstage backend CPU spikes to 100% due to the default in-memory Lunr search engine. How do you migrate and tune Backstage search using ElasticSearch?

Scaling Backstage search across 10,000 components, API specs, and TechDocs pages using ElasticSearch to resolve Lunr in-memory search timeouts.

#Platform Engineering #Backstage #ElasticSearch #Search #IDP #DevEx
🎙️ Candidate Opening & Architectural Context
"The default Lunr search engine in Backstage builds indexes in Node.js heap memory, crashing under large enterprise catalogs. Production enterprise search requires deploying the @backstage/plugin-search-backend-module-elasticsearch engine with dedicated indices for Catalog, TechDocs, and APIs."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: The default Lunr search engine in Backstage builds indexes in Node.js heap memory, crashing under large enterprise catalogs. Production enterprise search requires deploying the @backstage/plugin-search-backend-module-elasticsearch engine with dedicated indices for Catalog, TechDocs, and APIs.

1

Deploy ElasticSearch / OpenSearch Cluster on Kubernetes

Deploy a managed or operator-backed OpenSearch cluster with dedicated master and data nodes, enabling TLS and role-based access.

# app-config.yaml
search:
  engine:
    type: elasticsearch
    elasticsearch:
      node: https://opensearch.platform.acme.internal:9200
      auth:
        username: ${OPENSEARCH_USER}
        password: ${OPENSEARCH_PASSWORD}
2

Configure Dedicated Document Collators and Incremental Indexing

In packages/backend/src/plugins/search.ts, register the DefaultCatalogCollatorFactory and DefaultTechDocsCollatorFactory with batch sizes (e.g. 500 documents) and staggered schedules.

indexBuilder.addSearchEngine({
  engine: ElasticSearchSearchEngine.fromConfig({ config }),
});
indexBuilder.addCollator({
  schedule: env.scheduler.createScheduledTaskRunner({ frequency: { minutes: 15 }, timeout: { minutes: 5 } }),
  factory: DefaultCatalogCollatorFactory.fromConfig(config, { discovery: env.discovery }),
});
Advertisement
3

Tune Search Relevance and Tokenizers

Configure ngram tokenizers in OpenSearch for partial matching so developers finding auth-svc matches authentication-service accurately.

Pro Tip: DevEx Win: ElasticSearch drops p95 search latency from 8.2 seconds to 45 milliseconds across 50,000 indexed documents.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Replace Backstage's default in-memory Lunr search engine with ElasticSearch/OpenSearch and incremental collator scheduling to deliver sub-50ms search."
⚡ 60-Second Elevator Pitch Talking Points
  • Migrate from default in-memory Lunr engine to OpenSearch/ElasticSearch for enterprise catalog scale.
  • Configure staggered collator schedules to index TechDocs and Catalog entities asynchronously.
  • Implement partial-match ngram tokenizers to improve service discovery and developer search UX.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Engineering bespoke in-house developer tools instead of adopting standardized platform Golden Paths when configuring Internal Developer Portal Bespoke internal tools create long-term maintenance debt and fragmented workflows, whereas standardized Golden Paths accelerate delivery.
  2. Overlooking infrastructure cost visibility and FinOps allocation tagging across multi-tenant environments when configuring Internal Developer Portal Without granular resource tagging, engineering organizations cannot attribute cloud infrastructure costs to specific teams or business units.
  3. Failing to define explicit Service Level Objectives (SLOs) and Error Budgets to balance feature velocity with reliability when configuring Internal Developer Portal Without agreed SLOs, engineering and product teams lack an objective data-driven framework to prioritize technical debt over new features.
  4. Neglecting automated continuous drift detection between declarative configuration and deployed cloud state when configuring Internal Developer Portal Unmanaged drift accumulates over time, causing future automated deployments to fail or overwrite emergency manual adjustments.
Advertisement
Want more Platform Engineering & IDP scenarios?
Explore our complete collection of scenario-based Platform Engineering & IDP interview runbooks.
Browse All Platform Engineering & IDP Questions →