Q: What is the difference between Kinesis Data Streams and SQS?
Kinesis: ordered stream, multiple consumers can read same data, data retained 24h-365 days, good for analytics and fan-out.
Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: "In a previous role, our monitoring paged me for a similar incident across our AWS VPC infrastructure. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix."
Production Solution & Architecture
Kinesis: ordered stream, multiple consumers can read same data, data retained 24h-365 days, good for analytics and fan-out. SQS: queue, message deleted after consumed, at-least-once delivery, simpler programming model, good for decoupling services. Use Kinesis for streaming analytics, SQS for task queues.
- Immediate Triage: Kinesis: ordered stream, multiple consumers can read same data, data retained 24h-365 days, goo
- Run targeted verification commands before modifying configuration.
- Automate permanent guardrails (CI check, alerts, IaC policy) to prevent recurrence.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
-
Subscribing an Amazon SQS queue to an SNS topic without attaching an explicit queue access policy when configuring difference Kinesis Data
SNS lacks permission to write to SQS by default; without an IAM resource policy granting
sqs:SendMessageto the topic ARN, messages drop silently. -
Confusing standard SQS queues with FIFO queues regarding strict ordering and deduplication when configuring difference Kinesis Data
Standard SQS provides at-least-once delivery with best-effort ordering; strict FIFO ordering requires
MessageGroupIdand deduplication IDs. -
Using short polling instead of SQS long polling (
WaitTimeSeconds: 20) in high-volume consumers when configuring difference Kinesis Data Short polling queries single partition subsets and returns empty responses, dramatically inflating AWS API billing and consumer CPU usage. -
Failing to monitor
ApproximateNumberOfMessagesVisibleandApproximateAgeOfOldestMessagein CloudWatch when configuring difference Kinesis Data Queue depth and message age are primary SLI signals for consumer backlog saturation, far superior to consumer CPU utilization.