Q: During a massive DDoS attack or viral traffic spike, you see thousands of `SYN_RECV` states in `netstat` and the kernel is dropping legitimate connections with a "syn flooding" message. How do you tune the kernel networking stack on the fly to survive this?
The server is dropping packets because the temporary queue that holds incomplete TCP connections (waiting for the final ACK) is full.
#Linux #Linux / SRE — Scenario-Based Interview Questions #L3 #SRE #Systems #Troubleshooting
🎙️ Candidate Opening & Architectural Context
""During an on-call shift, our alerts triggered when a critical Linux production server exhibited this behavior. The interviewer is testing: Tuning `sysctl`, kernel ring buffers, SYN backlog.. I structure my answer around systematic triage first, root cause analysis second, and permanent remediation third.""
Advertisement
🛠️ Production Runbook & Step-by-Step Resolution
1️⃣
Initial Diagnostics & Root Cause Analysis
The server is dropping packets because the temporary queue that holds incomplete TCP connections (waiting for the final ACK) is full.
sysctl -w net.ipv4.tcp_max_syn_backlog=4096(Increases the size of the incomplete connection queue).sysctl -w net.core.somaxconn=4096(Increases the absolute limit of socket listen queues).sysctl -w net.ipv4.tcp_syncookies=1(Enables SYN Cookies, allowing the kernel to stop saving state for incomplete SYNs entirely, bypassing the backlog limits cryptographically).
2️⃣
Remediation & Permanent Safeguards
I would dynamically increase the queue limits in the kernel via sysctl immediately: These command-line tweaks take effect immediately without a reboot, mitigating the drops live. I would then save them in /etc/sysctl.conf.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Pro-Tip: sysctl -w net.ipv4.tcp_max_syn_backlog=4096 (Increases the size of the incomplete connection queue).."
⚡ 60-Second Elevator Pitch Talking Points
- sysctl -w net.ipv4.tcp_max_syn_backlog=4096 (Increases the size of the incomplete connection queue).
- sysctl -w net.core.somaxconn=4096 (Increases the absolute limit of socket listen queues).
- sysctl -w net.ipv4.tcp_syncookies=1 (Enables SYN Cookies, allowing the kernel to stop saving stat...
Advertisement