⚡ ~/naveed Interview Prep
⚡ Portfolio Home ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Docker & Containers Interview Questions Scenario 139 of 158 in Docker & Containers
Staff Infrastructure Architect Docker Container Runtime & Systems Engineering Production Scenario

Q: Your microservices on a custom Docker bridge network experience intermittent 5,000ms latency spikes when making external API calls to AWS S3 and payment gateways. Application logs show sporadic `getaddrinfo EAI_AGAIN` or `dns lookup timed out` errors. Direct `curl` commands to external IP addresses respond in 15ms. You must trace the container's DNS resolution path through the embedded Docker DNS server at `127.0.0.11`, identify UDP packet drops and search domain amplification, and implement a resilient DNS configuration.

Diagnose intermittent 5-second DNS resolution timeouts inside Docker containers. Understand Docker's internal 127.0.0.11 resolver, `resolv.conf` `ndots:5` search path amplification, and UDP socket drops.

#Docker #Networking #DNS #Linux #Troubleshooting
🎙️ Candidate Opening & Architectural Context
"Diagnose intermittent 5-second DNS resolution timeouts inside Docker containers. Understand Docker's internal 127.0.0.11 resolver, `resolv.conf` `ndots:5` search path amplification, and UDP socket drops."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's Docker Certified Associate (DCA) Hands-On Lab Course covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Step 1

Understand Docker Embedded DNS Resolver Mechanics

On user-defined bridge networks, Docker injects an internal DNS server listening at `127.0.0.11:53`. This server intercepts DNS queries: container names are resolved via internal Docker daemon records, while external queries are forwarded to upstream DNS servers listed in the host's `/etc/resolv.conf`.

<!-- Docker DNS Path -->
Container App (getaddrinfo)
  └── Queries: 127.0.0.11:53 (Docker Embedded DNS)
        ├── Matches service name? -> Returns container IP (172.18.0.4)
        └── External domain?      -> Forwards UDP packet to Host DNS (e.g., 8.8.8.8)
Pro Tip: Understand Docker Embedded DNS Resolver Mechanics
Step 2

Diagnose the 5-Second Timeout and ndots Search Amplification

Standard Linux glibc resolvers append every domain listed in the `search` directive of `/etc/resolv.conf` if the query contains fewer dots than `ndots` (default is often 5 in container environments). Querying `api.stripe.com` generates redundant queries like `api.stripe.com.default.svc.cluster.local`, causing upstream rate limiting or dropped UDP packets that trigger a 5-second timeout.

# Inspect container resolv.conf
docker exec -it my-app cat /etc/resolv.conf
# Sample output:
# nameserver 127.0.0.11
# options ndots:5

# Trace DNS resolution using dig and tcpdump
docker exec -it my-app dig +trace api.stripe.com
Pro Tip: Diagnose the 5-Second Timeout and ndots Search Amplification
Advertisement
Step 3

Capture DNS Packets and Detect Conntrack Race Conditions

Use `tcpdump` inside the container or on the bridge interface. High-concurrency DNS lookups often trigger a known Linux kernel conntrack race condition where simultaneous A and AAAA DNS queries collide on the same UDP socket.

# Monitor DNS traffic on Docker bridge interface
sudo tcpdump -i br-1a2b3c4d -n "udp port 53" -vv
Pro Tip: Capture DNS Packets and Detect Conntrack Race Conditions
Step 4

Configure Explicit Upstream DNS and Single-Request Options

Fix the issue by specifying reliable upstream DNS servers and passing `single-request-reopen` via `--dns-opt` to force sequential UDP lookups, or lowering `ndots`.

# Docker run with hardened DNS options
docker run -d \
  --name resilient-api \
  --dns 1.1.1.1 \
  --dns 8.8.8.8 \
  --dns-opt ndots:2 \
  --dns-opt single-request-reopen \
  --dns-opt timeout:2 \
  my-api:v1.0

# Or globally in /etc/docker/daemon.json:
# {
#   "dns": ["1.1.1.1", "8.8.8.8"],
#   "dns-opts": ["single-request-reopen", "timeout:2"]
# }
Pro Tip: Configure Explicit Upstream DNS and Single-Request Options
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Docker containers on bridge networks resolve DNS via `127.0.0.11`. High-concurrency lookups suffer 5-second timeouts due to `ndots` search list amplification and concurrent A/AAAA UDP socket collisions in kernel conntrack. Setting `single-request-reopen` and configuring explicit DNS servers resolves the latency."
⚡ 60-Second Elevator Pitch Talking Points
  • W
  • e
  • d
  • i
  • a
  • g
  • n
  • o
  • s
  • e
  • d
  • i
  • n
  • t
  • e
  • r
  • m
  • i
  • t
  • t
  • e
  • n
  • t
  • 5
  • -
  • s
  • e
  • c
  • o
  • n
  • d
  • l
  • a
  • t
  • e
  • n
  • c
  • y
  • s
  • p
  • i
  • k
  • e
  • s
  • i
  • n
  • e
  • x
  • t
  • e
  • r
  • n
  • a
  • l
  • A
  • P
  • I
  • c
  • a
  • l
  • l
  • s
  • b
  • y
  • c
  • a
  • p
  • t
  • u
  • r
  • i
  • n
  • g
  • D
  • N
  • S
  • q
  • u
  • e
  • r
  • i
  • e
  • s
  • o
  • n
  • D
  • o
  • c
  • k
  • e
  • r
  • b
  • r
  • i
  • d
  • g
  • e
  • n
  • e
  • t
  • w
  • o
  • r
  • k
  • s
  • .
  • T
  • h
  • e
  • i
  • s
  • s
  • u
  • e
  • s
  • t
  • e
  • m
  • m
  • e
  • d
  • f
  • r
  • o
  • m
  • g
  • l
  • i
  • b
  • c
  • a
  • p
  • p
  • e
  • n
  • d
  • i
  • n
  • g
  • s
  • e
  • a
  • r
  • c
  • h
  • d
  • o
  • m
  • a
  • i
  • n
  • s
  • f
  • o
  • r
  • e
  • x
  • t
  • e
  • r
  • n
  • a
  • l
  • F
  • Q
  • D
  • N
  • s
  • a
  • n
  • d
  • c
  • o
  • n
  • c
  • u
  • r
  • r
  • e
  • n
  • t
  • A
  • /
  • A
  • A
  • A
  • A
  • s
  • o
  • c
  • k
  • e
  • t
  • c
  • o
  • l
  • l
  • i
  • s
  • i
  • o
  • n
  • s
  • i
  • n
  • L
  • i
  • n
  • u
  • x
  • c
  • o
  • n
  • n
  • t
  • r
  • a
  • c
  • k
  • .
  • C
  • o
  • n
  • f
  • i
  • g
  • u
  • r
  • i
  • n
  • g
  • `
  • s
  • i
  • n
  • g
  • l
  • e
  • -
  • r
  • e
  • q
  • u
  • e
  • s
  • t
  • -
  • r
  • e
  • o
  • p
  • e
  • n
  • `
  • a
  • n
  • d
  • t
  • u
  • n
  • i
  • n
  • g
  • `
  • n
  • d
  • o
  • t
  • s
  • :
  • 2
  • `
  • i
  • n
  • `
  • d
  • a
  • e
  • m
  • o
  • n
  • .
  • j
  • s
  • o
  • n
  • `
  • c
  • o
  • m
  • p
  • l
  • e
  • t
  • e
  • l
  • y
  • e
  • l
  • i
  • m
  • i
  • n
  • a
  • t
  • e
  • d
  • t
  • h
  • e
  • D
  • N
  • S
  • t
  • i
  • m
  • e
  • o
  • u
  • t
  • s
  • .
Advertisement
Want more Docker & Containers scenarios?
Explore our complete collection of scenario-based Docker & Containers interview runbooks.
Browse All Docker & Containers Questions →