Q: Your global SaaS platform requires sub-10ms write latency for users in both the US and Europe with 99.999% availability. You need to enable Azure Cosmos DB Multi-Region Multi-Write. How do you design conflict resolution, select consistency levels, and configure client SDK connection policies to avoid stale reads and split-brain data loss?
Engineering a globally distributed active-active data tier using Azure Cosmos DB Multi-Region Multi-Write, conflict resolution policies (LWW vs Stored Procedures), and session consistency tuning.
Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: Previously, our Cosmos DB instance ran single-region writes in East US with read replicas in West Europe. European users suffered 120ms write latency across the Atlantic. We migrated to Multi-Region Multi-Write with custom Last-Write-Wins conflict handling.
Enable Multi-Region Multi-Write on Cosmos DB Account
Reconfigure the account topology to permit write operations across all regional replicas:
- CLI Provisioning: Executed
az cosmosdb update --name cosmos-global --resource-group rg-data --enable-multiple-write-locations true --locations regionName=eastus failoverPriority=0 isZoneRedundant=true --locations regionName=westeurope failoverPriority=1 isZoneRedundant=true. - SLA Upgrade: Multi-region multi-write elevates Cosmos DB availability SLA to 99.999% for both reads and writes.
Configure Conflict Resolution: Last-Write-Wins (LWW) vs Custom
Establish deterministic data resolution rules when simultaneous concurrent writes update the same document in different regions:
- LWW Policy: Configured
conflictResolutionPolicy: { mode: 'LastWriteWins', conflictResolutionPath: '/_ts' }for standard document updates. - Custom Stored Procedure: For financial balance adjustments, deployed a Javascript Conflict Resolution Stored Procedure (SP) that executes vector-clock merge logic rather than discarding concurrent updates.
Select Consistency Level: Session vs Bounded Staleness
Balance CAP theorem trade-offs between write latency and read guarantees:
- Session Consistency: Selected Session Consistency for user-facing workloads, ensuring monotonic reads and read-your-own-writes guarantees within the client session token.
- Session Token Forwarding: Implemented HTTP response header passing of
x-ms-session-tokenbetween distributed microservices to ensure read consistency across services.
Configure Azure Cosmos DB SDK Connection Policies
Ensure application SDK instances automatically route to their local geographic region:
- Preferred Regions List: Configured SDK client builder:
CosmosClientBuilder.endpoint(URI).preferredRegions(List.of('East US', 'West Europe')).buildClient(). - Dynamic Fallback: Verified that during a simulated datacenter outage, SDK automatically retried writes against West Europe within 200ms.
- Enable multiple write locations on Cosmos DB account across geographic regions.
- Implement Last-Write-Wins or custom Stored Procedure conflict resolution policies.
- Use Session Consistency with session token propagation to guarantee monotonic reads.
- Configure SDK preferredRegions list to ensure applications write to the closest datacenter.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Engineering bespoke in-house developer tools instead of adopting standardized platform Golden Paths when configuring Active Active Global Bespoke internal tools create long-term maintenance debt and fragmented workflows, whereas standardized Golden Paths accelerate delivery.
- Overlooking infrastructure cost visibility and FinOps allocation tagging across multi-tenant environments when configuring Active Active Global Without granular resource tagging, engineering organizations cannot attribute cloud infrastructure costs to specific teams or business units.
- Failing to define explicit Service Level Objectives (SLOs) and Error Budgets to balance feature velocity with reliability when configuring Active Active Global Without agreed SLOs, engineering and product teams lack an objective data-driven framework to prioritize technical debt over new features.
- Neglecting automated continuous drift detection between declarative configuration and deployed cloud state when configuring Active Active Global Unmanaged drift accumulates over time, causing future automated deployments to fail or overwrite emergency manual adjustments.