Q: A serverless microservice using DynamoDB is throttled with 'ProvisionedThroughputExceededException', even though total consumed capacity is only 25% of the provisioned table limits. Investigation reveals 95% of traffic queries a single partition key ('Status = PENDING'). Furthermore, an asynchronous Global Secondary Index (GSI) has reached write capacity limits and is backpressuring the main table. How do you re-architect the partition key scheme and mitigate GSI write throttling?
Solve DynamoDB ProvisionedThroughputExceededException throttling caused by asymmetric partition key distribution and asynchronous Global Secondary Index write backpressure.
#AWS #DynamoDB #Hot Partitions #GSI #Throttling #Partition Keys #NoSQL
🎙️ Candidate Opening & Architectural Context
"Amazon DynamoDB distributes data across storage partitions based on the hash of the Partition Key (PK). Each individual physical partition is capped at 1,000 Write Capacity Units (WCUs) and 3,000 Read Capacity Units (RCUs). When workload traffic is skewed toward a single partition key (a 'hot partition'), throttling occurs even if the overall table has unused provisioned capacity. Crucially, when an asynchronous Global Secondary Index (GSI) is throttled on writes, DynamoDB throttles writes on the base table to prevent the index from falling indefinitely behind."
Advertisement
⚡ Recommended Practice Lab
Want to master this scenario in a live sandbox? KodeKloud's PostgreSQL Database Administration & High Availability Course covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
⚡
Situation: Throttling Despite Low Global Table Utilization
⚡
Task: Eliminate Hot Partitions & Balance GSI Provisioning
Advertisement
⚡
Action: Partition Key Salting & Sparse Index Optimization
⚡
Result: 100% Throttling Elimination & 65% Cost Reduction
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"DynamoDB partitions are individually limited to 1,000 WCU / 3,000 RCU regardless of total table provisioning. Use key salting for hot keys, sparse GSIs to limit index write volume, and remember that throttled GSIs will backpressure base table writes."
⚡ 60-Second Elevator Pitch Talking Points
- DynamoDB partitions have a hard limit of 1,000 WCU and 3,000 RCU each.
- A single hot partition key or a throttled GSI will throttle writes to the entire base table.
- Apply write salting (partition sharding) and sparse GSIs to distribute throughput and eliminate backpressure.
Advertisement