Q: Design a highly available, production-grade cloud infrastructure for a microservices application handling millions of requests per day. Explain your choices around networking, load balancing, autoscaling, databases, observability, security, and disaster recovery.

Multi-layered ingress and defense-in-depth perimeter:

• Edge Acceleration & Perimeter: Amazon CloudFront for global content delivery, SSL termination, and static caching, paired with AWS WAF (OWASP Top 10 rules, rate-limiting, bot control) and AWS Shield for DDoS protection.
• VPC Topology: Multi-AZ VPC spanning 3 Availability Zones (AZs) with three subnet tiers: Public Subnets (ALB, NAT Gateways), Private Subnets (EKS compute nodes), and Isolated Database Subnets (no internet route).
• Load Balancing Layer: Public Application Load Balancer (ALB) distributing traffic across EKS worker nodes, handing off to an in-cluster Ingress Controller (NGINX or AWS Load Balancer Controller) using IP target mode (bypasses kube-proxy hop directly to pod IPs via AWS VPC CNI).

High-density, cost-effective container orchestration:

• Managed Control Plane: Amazon EKS spanning 3 AZs for resilient API availability.
• Fast Node Provisioning (Karpenter): Replace slow Cluster Autoscaler with Karpenter for just-in-time EC2 provisioning (graviton/spot/on-demand mix) in <45 seconds based on pending pod requests.
• Multi-Tier Workload Autoscaling: Horizontal Pod Autoscaler (HPA) coupled with KEDA (Kubernetes Event-driven Autoscaling) to scale on custom business metrics (e.g. SQS queue backlog, Redis queue length, or HTTP RPS) before CPU saturates.
• HA Scheduling Safeguards: topologySpreadConstraints across AZs and PodDisruptionBudgets (PDBs) to guarantee minimum healthy replicas during rolling updates or node drains.

Decoupling hot reads, writes, and cache tiers:

• Relational Database: Amazon Aurora PostgreSQL (Multi-AZ) with a primary writer instance and auto-scaling Read Replicas across AZs. Aurora provides storage auto-replication across 6 storage nodes and sub-30s failover.
• Caching Tier: Amazon ElastiCache for Redis (Cluster Mode) with multi-AZ replication. Implements cache-aside pattern for hot user queries and session state, absorbing 80%+ read traffic from the database.
• NoSQL / Event Streaming: Amazon DynamoDB with on-demand capacity for ultra-low latency key-value lookups; Amazon MSK (Managed Kafka) or SQS for asynchronous event-driven inter-service messaging.

Hardening at rest, in transit, and across identities:

• Workload Identity: IAM Roles for Service Accounts (IRSA) — pods assume scoped AWS IAM roles without long-lived credentials.
• Secrets Management: AWS Secrets Manager integrated via External Secrets Operator (ESO) with automatic password rotation; etcd encryption-at-rest via AWS KMS.
• Network Segmentation: Calico/Cilium NetworkPolicies enforcing default-deny ingress/egress between microservice namespaces.
• Runtime Auditing: Falco runtime threat detection + AWS GuardDuty EKS Protection.

Unified telemetry and business continuity:

• Distributed Telemetry: OpenTelemetry collector agents forwarding metrics to Prometheus/Grafana, logs to Loki/OpenSearch, and traces to Tempo/Jaeger with W3C tracecontext headers.
• SLO/SLI Alerting: Multi-window burn rate alerts sent to PagerDuty based on error budget depletion.
• Disaster Recovery (RPO < 1m, RTO < 15m): Route 53 DNS failover with health checks. Aurora Global Databases replicating across a secondary AWS region with automated cross-region S3 backup replication.