Q: You have an SQS queue triggering a Lambda function to encode massive video files. Sometimes a video takes 8 minutes to encode. You randomly notice the exact same video being encoded simultaneously by two different Lambda functions. Why?
The SQS Visibility Timeout is misconfigured.
#AWS #Cost & Architecture #L2 #Cloud #Infrastructure
🎙️ Candidate Opening & Architectural Context
""In our AWS cloud environment, we managed high-traffic microservices where this exact scenario occurred. The interviewer is testing: SQS Visibility Timeout.. I structure my answer around systematic triage first, root cause analysis second, and permanent remediation third.""
Advertisement
🛠️ Production Runbook & Step-by-Step Resolution
1️⃣
Production Solution & Architecture
The SQS Visibility Timeout is misconfigured. When a Lambda polls SQS, the message doesn't delete immediately; it becomes "invisible" to other consumers for the duration of the Visibility Timeout (default 30 seconds). Because the video encode takes 8 minutes, the 30-second timeout expires violently mid-encode. SQS assumes the first Lambda quietly crashed, making the message instantly visible again. A second Lambda picks up the identical message and starts encoding it. *Fix:* You must increase the SQS Visibility Timeout to be strictly greater than the maximum theoretical runtime of the Lambda function (e.g., set it to 10 minutes, or 600 seconds).
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Pro-Tip: The SQS Visibility Timeout is misconfigured.."
⚡ 60-Second Elevator Pitch Talking Points
- Immediate Triage: The SQS Visibility Timeout is misconfigured.
- Run targeted verification commands before modifying configuration.
- Automate permanent guardrails (CI check, alerts, IaC policy) to prevent recurrence.
Advertisement