Diagnose Nginx “upstream timed out” errors

Read the phrase after upstream timed out. On Linux, error 110 is ETIMEDOUT, but the surrounding message identifies the operation that stalled. A connection timeout and an idle response timeout require different evidence.

A body read timeout does not by itself prove that the upstream crashed or closed the connection.

Test connection failures from the proxy's network

curl -v --connect-timeout 5 --max-time 15 http://192.0.2.10:8080/
# On the upstream Linux host:
ss -ltnp '( sport = :8080 )'

Use the real endpoint in place of the documentation address. Check security groups and firewalls, the interface on which the process listens, and whether a configured address is stale. In separate containers, localhost refers to the current container, not a sibling service. Test from Nginx's network namespace when possible.

A refused connection often fails immediately; dropped packets or an overloaded listen backlog can time out. Avoid treating every connect timeout as proof of one firewall rule.

Investigate slow response progress

For an HTTP proxy location, an intentional adjustment might be:

location /reports/ {
    proxy_pass http://127.0.0.1:8080;
    proxy_read_timeout 120s;
    proxy_send_timeout 120s;
}

Read and send timeouts apply between successive operations, not to the total response duration. FastCGI uses its own fastcgi_* directives. A longer Nginx timeout cannot override a shorter client, gateway, or application deadline.

Look for queue saturation, slow queries, dependency waits, runtime pauses, and network loss. Match the request's time window and upstream address to application logs or collected spans. Traces reveal only instrumented work; a request that never reached the application needs network and listener investigation first. Guance Nginx collection can help retain the proxy-side evidence.

FAQ

Is proxy_connect_timeout capped at exactly 75 seconds?

Nginx documentation notes that it usually cannot exceed 75 seconds; effective behavior depends on connection establishment and the underlying system. It is not a universal exact ceiling or a reason to avoid investigating the network.

Why is the problem intermittent?

Compare failures by backend, route, load, and time. Pool pressure, garbage collection, transient packet loss, or a single failing instance can all produce patterns. Add capacity only after establishing what is constrained.

Will Nginx always retry another upstream?

The default retry conditions include errors and timeouts, but retrying also depends on whether a response has reached the client, request replayability, available servers, and retry limits. Non-idempotent requests require particular care. If several backends fail, investigate shared dependencies and networking without assuming all failures have one cause.

Reference: Nginx proxy timeout and retry directives.