⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All AWS & Cloud Architecture Interview Questions Scenario 118 of 179 in AWS & Cloud Architecture
Staff / Principal SRE AWS Cloud Architecture & Multi-Cloud Netflix-Scale Systems
🎯 Target Role / Context: Staff Cloud Architect / Enterprise Multi-Cloud Design

Q: Netflix runs multi-cloud. Describe your approach to cross-cloud routing, IAM, and secret syncing.

Engineering blueprint for designing an enterprise multi-cloud substrate across AWS and GCP covering private BGP interconnects, federated OIDC workload identities, and continuous secret synchronization.

#Multi-Cloud #AWS #GCP #BGP #Workload Identity #SPIFFE #Vault #System Design
🎙️ Candidate Opening & Architectural Context
"True enterprise multi-cloud is not about running the exact same Kubernetes YAML everywhere. It is about building a unified abstraction across three foundational pillars: low-latency private network routing without public internet traversal, cryptographic workload identity federation without static API keys, and centralized secret lifecycle reconciliation."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: True enterprise multi-cloud is not about running the exact same Kubernetes YAML everywhere. It is about building a unified abstraction across three foundational pillars: low-latency private network routing without public internet traversal, cryptographic workload identity federation without static API keys, and centralized secret lifecycle reconciliation.

1️⃣

Private Cross-Cloud Routing (AWS Direct Connect + GCP Cloud Interconnect)

Never route internal multi-cloud traffic over the public internet with IPsec tunnels due to MTU fragmentation and jitter:

  • Colocation Interconnect Fabric: Utilize Equinix Fabric or Megaport to cross-connect AWS Direct Connect (DX) and GCP Dedicated Interconnect with redundant 10Gbps cross-connects.
  • BGP Routing over Transit Gateway: Establish eBGP sessions between AWS Transit Gateway (TGW) and GCP Cloud Router with non-overlapping RFC 1918 CIDRs (e.g., AWS 10.100.0.0/16, GCP 10.200.0.0/16).
  • Bidirectional Forwarding Detection (BFD): Configure BFD with 300ms transmit/receive intervals to detect link failures and trigger sub-second route convergence.
2️⃣

Zero-Static-Key IAM Federation (SPIFFE/SPIRE & Workload Identity)

Eliminate long-lived cloud credentials across providers using OIDC Workload Identity Federation:

# AWS Trust Policy allowing GCP Service Account to assume AWS IAM Role:
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "Federated": "accounts.google.com"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "accounts.google.com:aud": "https://iam.googleapis.com/projects/12345/locations/global/workloadIdentityPools/gcp-pool/providers/gcp-provider"
      }
    }
  }]
}
  • A workload running on Google Cloud GKE acquires an ephemeral short-lived Google OIDC token, exchanges it at AWS STS for temporary IAM credentials, and accesses Amazon S3 without a single hardcoded secret.
Advertisement
3️⃣

Centralized Secret Syncing Engine (HashiCorp Vault + External Secrets Operator)

Treat secrets as a single control plane replicated securely across clouds:

  • Vault Primary-Replica Cluster: Multi-region Vault cluster with KMS auto-unseal (AWS KMS in us-east-1, GCP Cloud KMS in us-central1).
  • Kubernetes External Secrets Operator (ESO): Deployed in both AWS EKS and GCP GKE clusters, fetching credentials locally from Vault and instantiating native Kubernetes v1/Secret objects.
  • Automated Rotation: Database credentials (RDS and Cloud SQL) are dynamically generated with 1-hour TTLs via Vault database secrets engines.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Multi-cloud operational resilience requires eliminating static trust: private layer-3 BGP fabrics replace public tunnels, OIDC workload identity replaces static API keys, and declarative secret operators replace manual syncing."
⚡ 60-Second Elevator Pitch Talking Points
  • We build a private layer-3 backbone using AWS Direct Connect and GCP Cloud Interconnect over an Equinix Cloud Exchange fabric with redundant eBGP sessions and BFD for sub-second failover.
  • For IAM, we forbid static IAM access keys. We deploy SPIFFE/SPIRE and OIDC Workload Identity Federation so workloads exchange cryptographic JWTs across clouds to assume native temporary IAM roles.
  • For secrets, we maintain a federated HashiCorp Vault cluster with cloud-native KMS auto-unseal, synced via External Secrets Operator into local Kubernetes clusters.
  • All operational credentials feature automated rotation with 1-hour dynamic TTLs to minimize credential blast radius.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Purging compromised IAM user credentials immediately prior to querying CloudTrail forensic event logs when configuring Multi Cross Routing Deactivating a compromised key prevents abuse while preserving identity records for CloudTrail forensic queries to assess blast radius.
  2. Conflating local IAM Permissions Boundaries with organizational Service Control Policies (SCPs) when configuring Multi Cross Routing SCPs define maximum permission guards at the AWS Organization level across accounts, whereas permission boundaries limit delegate privileges within a single account.
  3. Deploying single-AZ cloud infrastructure for production workloads without multi-AZ failover resilience when configuring Multi Cross Routing Availability Zone hardware failures or localized network partitions cause complete service downtime when resources are not distributed across >=2 AZs.
  4. Jumping straight to SSH or instance restarts without reviewing CloudWatch metrics and AWS Systems Manager logs when configuring Multi Cross Routing Senior SREs triage control plane and hypervisor metrics via CloudWatch and SSM Session Manager before executing invasive host-level actions.
Advertisement
Want more AWS & Cloud Architecture scenarios?
Explore our complete collection of scenario-based AWS & Cloud Architecture interview runbooks.
Browse All AWS & Cloud Architecture Questions →