Using 444

[Bernard Rosset]

>> You mentioned 250k requests/day, but you did not characterise the 
>> population spread.
> 
> This is also "whack-a-mole" -- you asked for "population spread" (my 
> comfort level in politics is low), but the spread is somewhat close to 
> world population - led by China, Pakistan, Vietnam, Brazil and 
> Microsoft.  Conspicuously absent are Russia and Google.  This is pure 
> math in my microcosm.

Maybe did I use the wrong word, but by "spread" I meant to talk about 
the diversity of IP addresses: are they always different or are the same 
coming back over and over?

In the former case, you are most probably hitting the LRU eviction on 
your zone, hence virtually "resetting" their rate-limit, and allow for 
more requests to pass than you would wish.
You could try and play with the zone memory size to see if that has an 
effect or not.

If that is not enough, are there recognisable patterns, such like 
relatively narrow CIDR ranges they would belong to (/10 or /11 are way 
too big), which could be linked to recurring organisations?

It's all guess-work after all.

At the moment, you are directly working in $binary_remote_address.
If you can regroup IP addresses in CIDR ranges, you could apply 
different rate limits.
To do that, you could stack the geo directive feeding a map one, in turn 
feeding limit_req_zone in the end.

Finally, on top of limiting requests, you could be also limiting 
connections of the worst offenders with limit_conn.
The effectiveness of that added layer will essentially depend on whether 
those requests are reusing connections or not.
-- 
Bernard Rosset
https://rosset.net/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4736 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://freenginx.org/pipermail/nginx/attachments/20250928/6b6d231a/attachment-0001.p7s>