⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All AWS & Cloud Architecture Interview Questions Scenario 147 of 179 in AWS & Cloud Architecture
Staff Cloud Architect Azure & Cloud Database Architecture & SRE Distributed Databases

Q: Your global SaaS platform requires sub-10ms write latency for users in both the US and Europe with 99.999% availability. You need to enable Azure Cosmos DB Multi-Region Multi-Write. How do you design conflict resolution, select consistency levels, and configure client SDK connection policies to avoid stale reads and split-brain data loss?

Engineering a globally distributed active-active data tier using Azure Cosmos DB Multi-Region Multi-Write, conflict resolution policies (LWW vs Stored Procedures), and session consistency tuning.

#Azure #Cosmos DB #Multi-Region #Multi-Write #NoSQL #SRE
🎙️ Candidate Opening & Architectural Context
"Previously, our Cosmos DB instance ran single-region writes in East US with read replicas in West Europe. European users suffered 120ms write latency across the Atlantic. We migrated to Multi-Region Multi-Write with custom Last-Write-Wins conflict handling."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: Previously, our Cosmos DB instance ran single-region writes in East US with read replicas in West Europe. European users suffered 120ms write latency across the Atlantic. We migrated to Multi-Region Multi-Write with custom Last-Write-Wins conflict handling.

1️⃣

Enable Multi-Region Multi-Write on Cosmos DB Account

Reconfigure the account topology to permit write operations across all regional replicas:

  • CLI Provisioning: Executed az cosmosdb update --name cosmos-global --resource-group rg-data --enable-multiple-write-locations true --locations regionName=eastus failoverPriority=0 isZoneRedundant=true --locations regionName=westeurope failoverPriority=1 isZoneRedundant=true.
  • SLA Upgrade: Multi-region multi-write elevates Cosmos DB availability SLA to 99.999% for both reads and writes.
Pro Tip: Enabling multiple write locations allows write operations to complete against the local datacenter, reducing write latency from >100ms down to <8ms.
2️⃣

Configure Conflict Resolution: Last-Write-Wins (LWW) vs Custom

Establish deterministic data resolution rules when simultaneous concurrent writes update the same document in different regions:

  • LWW Policy: Configured conflictResolutionPolicy: { mode: 'LastWriteWins', conflictResolutionPath: '/_ts' } for standard document updates.
  • Custom Stored Procedure: For financial balance adjustments, deployed a Javascript Conflict Resolution Stored Procedure (SP) that executes vector-clock merge logic rather than discarding concurrent updates.
Pro Tip: Default Last-Write-Wins relies on the highest timestamp. If concurrent writes hit US and EU simultaneously, the earlier write is discarded unless custom merge logic is defined.
Advertisement
3️⃣

Select Consistency Level: Session vs Bounded Staleness

Balance CAP theorem trade-offs between write latency and read guarantees:

  • Session Consistency: Selected Session Consistency for user-facing workloads, ensuring monotonic reads and read-your-own-writes guarantees within the client session token.
  • Session Token Forwarding: Implemented HTTP response header passing of x-ms-session-token between distributed microservices to ensure read consistency across services.
Pro Tip: Strong consistency is not supported with Multi-Region Multi-Write because synchronous cross-region consensus violates low-latency physics.
4️⃣

Configure Azure Cosmos DB SDK Connection Policies

Ensure application SDK instances automatically route to their local geographic region:

  • Preferred Regions List: Configured SDK client builder: CosmosClientBuilder.endpoint(URI).preferredRegions(List.of('East US', 'West Europe')).buildClient().
  • Dynamic Fallback: Verified that during a simulated datacenter outage, SDK automatically retried writes against West Europe within 200ms.
Pro Tip: Without configuring preferredRegions in the SDK client, applications may inadvertently route writes across oceans to distant regions.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Cosmos DB Multi-Region Multi-Write achieves sub-10ms global writes and 99.999% SLA by combining local writes, Session consistency tokens, and well-defined conflict resolution policies."
⚡ 60-Second Elevator Pitch Talking Points
  • Enable multiple write locations on Cosmos DB account across geographic regions.
  • Implement Last-Write-Wins or custom Stored Procedure conflict resolution policies.
  • Use Session Consistency with session token propagation to guarantee monotonic reads.
  • Configure SDK preferredRegions list to ensure applications write to the closest datacenter.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Engineering bespoke in-house developer tools instead of adopting standardized platform Golden Paths when configuring Active Active Global Bespoke internal tools create long-term maintenance debt and fragmented workflows, whereas standardized Golden Paths accelerate delivery.
  2. Overlooking infrastructure cost visibility and FinOps allocation tagging across multi-tenant environments when configuring Active Active Global Without granular resource tagging, engineering organizations cannot attribute cloud infrastructure costs to specific teams or business units.
  3. Failing to define explicit Service Level Objectives (SLOs) and Error Budgets to balance feature velocity with reliability when configuring Active Active Global Without agreed SLOs, engineering and product teams lack an objective data-driven framework to prioritize technical debt over new features.
  4. Neglecting automated continuous drift detection between declarative configuration and deployed cloud state when configuring Active Active Global Unmanaged drift accumulates over time, causing future automated deployments to fail or overwrite emergency manual adjustments.
Advertisement
Want more AWS & Cloud Architecture scenarios?
Explore our complete collection of scenario-based AWS & Cloud Architecture interview runbooks.
Browse All AWS & Cloud Architecture Questions →