Number of keepalive connections to an upstream

[Sébastien Rebecchi]

Hi Maxim,

Thank you for that long explanation. It is quite clear and helps me a lot.

In my case upstream servers are vert.x servers, using event loop pattern so
i won't have the pb induced by thread-per-request servers like Apache you
mentionned.

I understood that all keepalive connections could end up being assigned to
the same server of the group occasionnally, no even distribution guarantee.
In my case I think that I can increase much the number of keepalive then. I
have 12 vertx servers in the group and implemented the *2 rule so it means
max 24 keepalive connections maintained to the same server, which is quite
low.

Thanks agin, have a great day.

Sébastien.


Le lun. 18 mars 2024, 23:50, Maxim Dounin <mdounin at mdounin.ru> a écrit :

> Hello!
>
> On Mon, Mar 18, 2024 at 09:08:18PM +0100, Sébastien Rebecchi wrote:
>
> > Hello,
> >
> > What is the good rule of thumbs for setting the number of keepalive
> > connections to an upstream group?
> >
> > 1. https://www.nginx.com/blog/performance-tuning-tips-tricks/
> > in this blog, the writer seems to recommend a constant value of 128, no
> > real explanation why it would fit whatever the number of servers in the
> > upstream
> >
> > 2.
> https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
> > the upstream module doc seems to recommend a rule like 16 times the
> number
> > of servers in the upstream, as we have two examples with respectively
> > keepalive 32 for 2 upstream servers and keepalive 16 for 1 upstream
> server
> >
> > 3.
> >
> https://www.nginx.com/blog/avoiding-top-10-nginx-configuration-mistakes/#no-keepalives
> > in this blog, the writer recommends a rule of 2 times the number of
> servers
> > in the upstream
> >
> > I used to follow rule of item 3 as it comes with a somewhat good
> > explanation, but it does not seem to be widely accepted.
> >
> > What could explain such a divergence between several sources? What would
> > you recommend please?
>
> There is no strict rule, and actual numbers to use depend heavily
> on the particular configuration, expected load, the number of
> upstream servers, and the maximum number of connections each of
> the upstream servers can maintain.
>
> The most notable limitation to keep in mind is that when you are
> configuring keepalive connections with upstream servers using
> process-per-connection model, it might be important to keep total
> number of cached connections below the maximum number of
> keepalive connections the upstream server can maintain, or you
> might end up with non-responsive upstream server.
>
> For example, consider an upstream server, such as Apache with
> prefork MPM, which is configured to run 256 worker processes
> (MaxRequestWorkers 256, which is the default and will typically
> require more than 8G of memory).
>
> As long as nginx is configured with "keepalive 32;" and 16 worker
> processes, this can easily consume all the Apache workers for
> keepalive connections, and you'll end up with occasional slow
> upstream responses simply because all the upstream worker
> processes are occupied by cached connections from other nginx
> worker processes, even without any real load.
>
> That is, it is a good idea to keep (worker_processes * keepalive)
> below the maximum number of connections the upstream server can
> maintain.
>
> Note well that keepalive cache size is configured for all servers,
> but it might end up keeping connection to just one upstream
> server.  As such, it might be important to choose the keepalive
> connections cache size based on the capacity of each upstream
> server individually, and not their total capacity.
>
> On the other hand, configuring cache which cannot hold at least 1
> connection to each upstream server hardly makes sense, as it can
> easily end up in cached connections being closed after each
> request and never reused.
>
> Hope this helps.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> --
> nginx mailing list
> nginx at freenginx.org
> https://freenginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://freenginx.org/pipermail/nginx/attachments/20240319/8141a7f3/attachment.htm>