Diagnose sleeping processes on Linux

A process in S state is usually waiting normally: for work, a signal, a lock, or another event. A large count alone does not establish a problem. Investigate sustained growth, exhausted worker capacity, memory pressure, or tasks stuck in D state.

Count states without confusing processes and threads

On a Linux system with procps tools:

ps -eo stat= | awk '{print substr($1,1,1)}' | sort | uniq -c
ps -eo stat=,comm= | awk '$1 ~ /^[SD]/ {print $2}' | sort | uniq -c | sort -rn

The first character is the main state; flags such as + or s add information. The ordinary process list does not enumerate every Java or application thread. Use thread-aware output when that is the suspected resource:

ps -eLo pid,tid,stat,comm

Compare several samples with the configured pool sizes and workload. Idle Apache or PHP-FPM workers commonly sleep. A pool near its configured maximum is not automatically healthy: determine whether workers are idle or requests are waiting for capacity.

for sample in 1 2 3; do
  ps -eo stat= | awk '$1 ~ /^S/ {n++} END {print n+0}'
  sleep 10
done

Inspect what a task is waiting for

Replace 1234 with the affected process ID:

cat /proc/1234/wchan
cat /proc/1234/stack
ls -1 /proc/1234/fd | wc -l
lsof -p 1234

Kernel configuration and permissions can restrict stack and wait-channel output. The file-descriptor count is a point-in-time approximation; compare it over time rather than treating one number as a leak.

Where authorized and operationally acceptable, syscall tracing can reveal waits:

strace -f -c -p 1234

Stop the trace with Ctrl+C to see the summary. Attaching can add overhead and expose sensitive arguments; it is an investigative step, not a harmless always-on check.

Interpret the pattern

D means uninterruptible sleep, often associated with I/O; it is not proof of a failed disk. Inspect dmesg or journalctl -k and the relevant mounts. Linux load averages include runnable and uninterruptible tasks, so a high load need not mean high CPU use.

Z is different: a zombie has exited but its parent has not collected its status. Fix the parent's process-management behavior rather than treating it as an idle worker.

Fix the wait, not the state label

Killing arbitrary sleeping workers can interrupt requests, discard work, or trigger restart loops. A task in uninterruptible sleep may not act on a pending signal until its wait completes. Diagnose the dependency or resource shortage first, then use the service's controlled recovery procedure.

These Linux interfaces apply beyond CentOS. Check the installed tools and use a supported operating-system release for ongoing fixes.

References: Linux process states and procps ps.