Number of keepalive connections to an upstream

[Maxim Dounin]

Hello!

On Mon, Mar 18, 2024 at 09:08:18PM +0100, Sébastien Rebecchi wrote:

> Hello,
> 
> What is the good rule of thumbs for setting the number of keepalive
> connections to an upstream group?
> 
> 1. https://www.nginx.com/blog/performance-tuning-tips-tricks/
> in this blog, the writer seems to recommend a constant value of 128, no
> real explanation why it would fit whatever the number of servers in the
> upstream
> 
> 2. https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
> the upstream module doc seems to recommend a rule like 16 times the number
> of servers in the upstream, as we have two examples with respectively
> keepalive 32 for 2 upstream servers and keepalive 16 for 1 upstream server
> 
> 3.
> https://www.nginx.com/blog/avoiding-top-10-nginx-configuration-mistakes/#no-keepalives
> in this blog, the writer recommends a rule of 2 times the number of servers
> in the upstream
> 
> I used to follow rule of item 3 as it comes with a somewhat good
> explanation, but it does not seem to be widely accepted.
> 
> What could explain such a divergence between several sources? What would
> you recommend please?

There is no strict rule, and actual numbers to use depend heavily 
on the particular configuration, expected load, the number of 
upstream servers, and the maximum number of connections each of 
the upstream servers can maintain.

The most notable limitation to keep in mind is that when you are 
configuring keepalive connections with upstream servers using 
process-per-connection model, it might be important to keep total 
number of cached connections below the maximum number of 
keepalive connections the upstream server can maintain, or you 
might end up with non-responsive upstream server.

For example, consider an upstream server, such as Apache with 
prefork MPM, which is configured to run 256 worker processes 
(MaxRequestWorkers 256, which is the default and will typically 
require more than 8G of memory).

As long as nginx is configured with "keepalive 32;" and 16 worker 
processes, this can easily consume all the Apache workers for 
keepalive connections, and you'll end up with occasional slow 
upstream responses simply because all the upstream worker 
processes are occupied by cached connections from other nginx 
worker processes, even without any real load.

That is, it is a good idea to keep (worker_processes * keepalive) 
below the maximum number of connections the upstream server can 
maintain.

Note well that keepalive cache size is configured for all servers, 
but it might end up keeping connection to just one upstream 
server.  As such, it might be important to choose the keepalive 
connections cache size based on the capacity of each upstream 
server individually, and not their total capacity.

On the other hand, configuring cache which cannot hold at least 1 
connection to each upstream server hardly makes sense, as it can 
easily end up in cached connections being closed after each 
request and never reused.

Hope this helps.

-- 
Maxim Dounin
http://mdounin.ru/