Design a highly available, production-grade cloud infrastructure for a microservices application handling millions of requests per day. Explain your choices around networking, load balancing, autoscaling, databases, observability, security, and disaster recovery.

To support millions of daily requests with 99.99% availability, the architecture is designed around multi-AZ redundancy, zero single points of failure, decoupling of state, automated progressive scaling, and zero-trust security. Key Takeaway: Achieving scale and 99.99% availability isn't about bigger machines: it's CloudFront/WAF edge caching, 3-AZ VPC with Karpenter + KEDA autoscaling, Aurora Multi-AZ with Redis caching, and zero-trust IAM with OpenTelemetry correlation.

For a project, how would you optimize costs for each environment? What strategies reduce cloud costs without affecting application availability?

Cost optimization is not about cutting resources blindly; it is about aligning infrastructure tiering to business risk. Lower environments can tolerate interruptions; Production demands zero downtime. Key Takeaway: Tier cost strategy by environment: Dev/QA get scheduled off-hours shutdowns and 100% Spot instances; Staging uses on-demand ephemeral environments; Production achieves 40%+ savings via Compute Savings Plans, ARM64 Graviton instances, and Karpenter consolidation without touching uptime SLAs.

How would you use Kubernetes resource requests/limits to control costs?

In Kubernetes, you pay for what you REQUEST, not what you use. If a developer requests 4 CPUs but the container only consumes 200m, the cloud provider bills you for 4 CPUs because the scheduler reserves the space. That gap is 'phantom spend'. Key Takeaway: Cloud bills scale with Kubernetes CPU/memory REQUESTS, not actual utilization. Over-provisioned requests force autoscalers to launch expensive unnecessary nodes. Right-size requests to p95 usage using VPA recommendation mode and enable Karpenter node consolidation.

How would you identify unused or over-provisioned cloud resources?

In every production cloud account over 1 year old, 15 to 25% of the monthly spend consists of 'zombie' or orphaned resources left behind by deleted clusters, test VMs, or failed CI pipelines. Key Takeaway: Implement continuous cloud hygiene: Audit for unattached EBS/managed disks, unassociated public IPs, and zero-target load balancers using AWS Compute Optimizer, Azure Advisor, and CLI query filters. Automate cleanup with Cloud Custodian policies.

47+ Production FinOps & System Design Interview Questions & Scenarios (2026 Edition)

Q: A developer pushes code that passes all unit tests and builds successfully in CI, but when it deploys to production, the application crashes immediately. Why might this happen?

\"In our engineering organization, DevOps culture meant aligning developer speed with site reliability. The interviewer is testing: Configuration Drift, 12-Factor App methodology.. I structure my answer around systematic triage first, root cause analysis second, and permanent remediation third.\" Key Takeaway: Pro-Tip: Configuration Mismatch: The production environment relies on environment variables, secrets, or database URLs that are either miss.

Staff / Principal SRE Cloud Architecture & Scalability

⚡ Scenario arch-ha-microservices-millions-requests

Design a High-Availability Cloud Infrastructure for Millions of Requests/Day

Full architectural blueprint for multi-AZ, production-grade microservices handling millions of daily requests: edge routing, Kubernetes compute with Karpenter, ...

#System Design #AWS #EKS

All FinOps & System Design Scenario Questions (47)