⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Networking & Cloud DNS Interview Questions Scenario 46 of 55 in Networking & Cloud DNS
Senior DevOps / SRE Networking Load Balancing & Traffic Diagnostics Barclays Classic

Q: Your application is healthy, but users experience intermittent timeouts. How would you identify whether the issue is in the application, network, infrastructure, or load balancer?

Production diagnostic framework to isolate intermittent timeout complaints across the application layer, network path, underlying compute infrastructure, and cloud load balancer using curl timing telemetry, connection metrics, and packet analysis.

#Networking #Load Balancer #Timeouts #Incident Triage #Barclays #SRE #Troubleshooting #Comparisons
🎙️ Candidate Opening & Architectural Context
"When users experience intermittent timeouts while target health checks remain green, the root cause is almost always an edge saturation condition—such as TCP connection queue overflow, ephemeral port exhaustion, upstream keep-alive mismatches, or tail-latency spikes that health probes miss. I isolate the fault layer-by-layer using client-side timing telemetry and load balancer access logs."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's Istio Service Mesh & Advanced Kubernetes Networking Course covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: When users experience intermittent timeouts while target health checks remain green, the root cause is almost always an edge saturation condition—such as TCP connection queue overflow, ephemeral port exhaustion, upstream keep-alive mismatches, or tail-latency spikes that health probes miss. I isolate the fault layer-by-layer using client-side timing telemetry and load balancer access logs.

1

Analyze Edge Timing Telemetry (Isolate Load Balancer vs. Backend)

Examine ALB/ELB access logs and CloudWatch metrics. Specifically compare request_processing_time, target_processing_time, and response_processing_time. If target_processing_time spikes to exactly the timeout threshold (e.g., 60.00s), the issue is downstream in the application or database. If target_processing_time is -1, the connection was dropped before reaching the target.

curl -w "@curl-format.txt" -o /dev/null -s -v https://api.production.internal/v1/health
# Sample curl-format.txt metrics:
# time_namelookup: %{time_namelookup}s
# time_connect:    %{time_connect}s
# time_appconnect: %{time_appconnect}s
# time_pretransfer:%{time_pretransfer}s
# time_starttransfer: %{time_starttransfer}s
# time_total:      %{time_total}s
2

Verify Infrastructure & Kernel Connection Queues

Log into host nodes or containers and inspect TCP socket allocation, listen backlog drops, and conntrack table saturation. If the application's listen queue is full (netstat -s | grep -i listen), requests sit in the OS backlog until the client or load balancer times out.

ss -lnt '( sport = :8080 )'
netstat -s | grep -E 'listen|overflow|drop'
cat /proc/sys/net/netfilter/nf_conntrack_count
cat /proc/sys/net/netfilter/nf_conntrack_max
Advertisement
3

Correlate Network Path & Packet Loss

Check NAT Gateway error metrics (ErrorPortAllocation, PacketsDropCount) and VPC flow logs. Verify whether AWS ALB idle timeout is shorter than the application's keep-alive timeout, which causes the load balancer to send requests down half-closed sockets resulting in 504 timeouts.

Pro Tip: Golden Rule: Always set application server keep-alive timeout higher than the load balancer's idle timeout (e.g., ALB idle timeout = 60s, App server keep-alive = 65s) to prevent race-condition connection resets.
4

Inspect Application Thread & Connection Pool Starvation

If network and load balancer metrics are clean, the application is suffering from thread pool starvation or database connection pool exhaustion where 99% of requests succeed fast, but 1% wait in a blocking queue until the HTTP client times out.

💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Intermittent timeouts with healthy probes indicate queue saturation or keep-alive race conditions. Break down latency into DNS, TCP handshake, TLS negotiation, and TTFB to immediately identify the guilty layer."
⚡ 60-Second Elevator Pitch Talking Points
  • Check ALB access logs: compare target_processing_time against request_processing_time to see if backend or LB is timing out.
  • Verify keep-alive parity: application server keepalive must be 5 seconds longer than the ALB idle timeout.
  • Inspect socket listen queues: check ss -lnt and netstat -s for listen queue overflows indicating app thread exhaustion.
  • Monitor NAT Gateway port allocation errors and conntrack table saturation for network-level drops.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Confusing ALB 502 Bad Gateway (target connection reset) with 504 Gateway Timeout (upstream response timeout) when configuring Application Healthy Users 502 indicates the backend target closed the TCP connection prematurely or returned malformed HTTP, whereas 504 means the target failed to respond before idle timeout.
  2. Setting target deregistration delay to zero instead of allowing connection draining when configuring Application Healthy Users Instant target deregistration severs in-flight HTTP requests during auto-scaling scale-in or rolling deployments, causing immediate user-facing drops.
  3. Failing to query CloudWatch ELB metrics (HTTPCode_ELB_5XX vs HTTPCode_Target_5XX) during load balancer outages when configuring Application Healthy Users Comparing load balancer generated errors against target generated errors immediately isolates whether the issue lies in the proxy tier or application runtime.
  4. Overlooking Security Group rules on backend EC2/ECS instances that block health check traffic from the ALB when configuring Application Healthy Users Target instances must explicitly permit inbound HTTP/HTTPS traffic from the ALB security group on the configured container/application port.
Advertisement
Want more Networking & Cloud DNS scenarios?
Explore our complete collection of scenario-based Networking & Cloud DNS interview runbooks.
Browse All Networking & Cloud DNS Questions →