⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Linux & SRE Interview Questions Scenario 3 of 135 in Linux & SRE
Senior DevOps / SRE Linux Performance & Processes Core Linux Skills
🎯 Target Role / Context: Linux System Internals & Performance Engineering

Q: Process is consuming high CPU — which commands would you use?

Full command toolkit for diagnosing high CPU processes: thread-level drilldown (top -H), syscall tracing (strace), kernel profiling (perf), and graceful mitigation.

#Linux #top #ps #strace #perf #jstack #CPU
🎙️ Candidate Opening & Architectural Context
"When a process consumes high CPU, my goal is to drill down from the system process level down to the exact thread, system call, or application function causing the burn."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? The Linux Foundation's Certified System Administrator (LFCS) & SRE Track covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: When a process consumes high CPU, my goal is to drill down from the system process level down to the exact thread, system call, or application function causing the burn.

1️⃣

Identify Process & System Context

Find the PID and understand its resource footprint:

  • top -c: Press P to sort by CPU. Shows full command-line paths.
  • ps aux --sort=-%cpu | head -10: Fast scriptable capture of top 10 CPU consumers.
  • pidstat 1 5 -u: Granular per-process CPU statistics over 5 seconds.
2️⃣

Thread-Level Inspection (top -H)

Multi-threaded runtimes (Java, Go, C++, Node worker threads) distribute load across threads:

  • top -H -p <PID>: Primary diagnostic command — displays individual threads inside the process as if they were processes. Identify the specific Thread ID (TID) pinned at 100%.
  • ps -T -p <PID>: List all threads and their CPU consumption.
  • For Java: Convert TID to hexadecimal (printf '%x\n' <TID>) and grep inside a jstack <PID> thread dump to find the exact line of Java code that is looping!
Advertisement
3️⃣

Trace Syscalls & Profile CPU (strace / perf)

Determine if CPU is burned in user code or kernel syscalls:

  • strace -p <PID> -c: Run for 10 seconds. Summarizes system calls, error rates, and time spent. If 90% time is in futex, process is suffering thread lock contention.
  • strace -p <PID> -t -e trace=all: View live system calls in real-time.
  • Kernel Profiling (perf): Run perf top -p <PID> or perf record -F 99 -p <PID> -g -- sleep 10 && perf report to generate a call graph / flamegraph showing exact function hotspots.
4️⃣

Mitigate and Control Priority

Managing the process safely:

  • Lower process priority: renice +10 -p <PID> gives other critical system processes CPU priority.
  • Graceful shutdown: kill -15 <PID> (SIGTERM). Only use kill -9 as a last resort.
  • Restrict with cgroups: Apply CPU bandwidth limits via systemd slice or Docker --cpus=1.5.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Use 'top -c' to find PID, 'top -H -p <PID>' to find the specific thread, and 'strace -c' or 'perf top' to pinpoint the looping function or lock contention."
⚡ 60-Second Elevator Pitch Talking Points
  • Find process: 'top -c' (sort by P) or 'ps aux --sort=-%cpu | head -10' or 'pidstat 1 5 -u'.
  • Inspect threads: 'top -H -p <PID>' to find the exact thread ID (TID) burning CPU.
  • Trace syscalls: 'strace -p <PID> -c' to see time spent in kernel syscalls vs user code.
  • Profile CPU: 'perf top -p <PID>' or capture thread dumps (jstack/pprof) before killing.
  • Control: 'renice +10 <PID>' to deprioritize, or 'kill -15 <PID>' for graceful termination.
  • Prevent: Set cgroup / systemd CPUQuota limits and container resource limits.
🎬 60-Second Video Walkthrough
Watch the Animated Breakdown & Practice Live

Mastered this procedure? Subscribe to @naveedkumbhar01 on YouTube for daily 60-second SRE incident runbooks, Kubernetes triage shortcuts, and high-impact interview elevator pitches.

🔔 Subscribe on YouTube (Free) ▶ Watch Short (60s) ↗
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Mistaking high Linux load average caused by uninterruptible I/O wait for compute CPU saturation Load average includes processes in uninterruptible disk sleep (D state); high load with low CPU indicates disk I/O bottlenecks or NFS hangs rather than compute saturation.
  2. Failing to check kernel ring buffer logs (dmesg -T) and journalctl during unexpected service crashes when addressing consuming Systemd service exits and kernel OOM reaper events are logged directly to the kernel ring buffer and systemd journal rather than application log files.
  3. Diagnosing infrastructure outages unsystematically without adhering to the USE or RED methodologies Senior Linux engineers systematically evaluate Utilization, Saturation, and Errors across compute, memory, and disk rather than making random configuration changes.
  4. Applying kernel runtime sysctl parameters on the fly without persisting them to /etc/sysctl.d/ Applying runtime sysctl adjustments via sysctl -w without configuration persistence causes critical tuning to vanish upon server reboot.
Advertisement
Want more Linux & SRE scenarios?
Explore our complete collection of scenario-based Linux & SRE interview runbooks.
Browse All Linux & SRE Questions →