<div dir="ltr">Hello Maxim,<br>Hope this helps.<br>We encounter disk failures fairly often and what the kernel will do most of the time is re-mount the disk as read-only.<br>What I did was add this check which checks if the disk is healthy before hand and executes the proxy_no_cache path if it is.<br>It doesn't cover all the possible disk failures, like sometimes you'll just get IO errors and still return 5XX to clients. But to cover all cases a much cleverer reshuffling of the code is needed. <br><br><br>diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c<br>index d7f427d50..839ed6c0d 100644<br>--- a/src/http/ngx_http_upstream.c<br>+++ b/src/http/ngx_http_upstream.c<br>@@ -8,6 +8,7 @@<br> #include <ngx_config.h><br> #include <ngx_core.h><br> #include <ngx_http.h><br>+#include <sys/statvfs.h><br> <br> #include <ngx_http_proxy_module.h><br> <br>@@ -3424,6 +3425,16 @@ ngx_http_upstream_send_response(ngx_http_request_t *r, ngx_http_upstream_t *u)<br> break;<br> <br> default: /* NGX_OK */<br>+ if (r->cache) {<br>+ struct statvfs fs;<br>+ if (statvfs((char *)r->cache->file_cache->path->name.data, &fs) == -1) {<br>+ return NGX_ERROR;<br>+ }<br>+ if ((fs.f_flag & ST_RDONLY) != 0) {<br>+ u->cacheable = 0;<br>+ break;<br>+ }<br>+ }<br> <br> if (u->cache_status == NGX_HTTP_CACHE_BYPASS) {<br><br></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Sun, Apr 7, 2024 at 4:56 PM Maxim Dounin <<a href="mailto:mdounin@mdounin.ru">mdounin@mdounin.ru</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello!<br>
<br>
On Sun, Apr 07, 2024 at 01:36:21PM +0200, Kirill A. Korinsky wrote:<br>
<br>
> Greetings,<br>
> <br>
> Let assume that I would like behavior on LB from the backend and force it to<br>
> cache only resposnes that have a X-No-Cache header with value NO.<br>
> <br>
> Nginx should cache a response with any code, if it has such headers.<br>
> <br>
> This works well until the backend is unavailable and nginx returns a<br>
> hardcoded 502 that doesn't have a control header, but such a response is<br>
> cached anyway.<br>
> <br>
> Here is the config that allows to reproduce the issue:<br>
> <br>
> http {<br>
> default_type application/octet-stream;<br>
> <br>
> proxy_cache_path /tmp/nginx_cache keys_zone=the_zone:1m;<br>
> proxy_cache the_zone;<br>
> proxy_cache_valid any 15m;<br>
> proxy_cache_methods GET HEAD POST;<br>
> <br>
> add_header X-Cache-Status $upstream_cache_status always;<br>
> <br>
> map $upstream_http_x_no_cache $no_cache {<br>
> default 1;<br>
> "NO" 0;<br>
> }<br>
> <br>
> proxy_no_cache $no_cache;<br>
> <br>
> upstream echo {<br>
> server <a href="http://127.127.127.127:80" rel="noreferrer" target="_blank">127.127.127.127:80</a>;<br>
> }<br>
> <br>
> server {<br>
> listen 1234;<br>
> server_name localhost;<br>
> <br>
> location / {<br>
> proxy_pass <a href="http://echo" rel="noreferrer" target="_blank">http://echo</a>;<br>
> }<br>
> }<br>
> }<br>
> <br>
> when I run:<br>
> <br>
> curl -D - <a href="http://127.0.0.1:1234/" rel="noreferrer" target="_blank">http://127.0.0.1:1234/</a><br>
> <br>
> it returns MISS on the first request, and HIT on the second one.<br>
> <br>
> Here I expect both requests to return MISS.<br>
<br>
Thanks for the report.<br>
<br>
Indeed, proxy_no_cache is only checked for proper upstream <br>
responses, but not when caching errors, including internally <br>
generated 502/504 in ngx_http_upstream_finalize_request(), and <br>
intercepted errors in ngx_http_upstream_intercept_errors().<br>
<br>
Quick look suggests there will be also issues with caching errors <br>
after proxy_cache_bypass (errors won't be cached even if they <br>
should), as well as issues with proxy_cache_max_range_offset after <br>
proxy_cache_bypass (it will be ignored).<br>
<br>
This needs cleanup / fixes, added to my TODO list.<br>
<br>
-- <br>
Maxim Dounin<br>
<a href="http://mdounin.ru/" rel="noreferrer" target="_blank">http://mdounin.ru/</a><br>
_______________________________________________<br>
nginx mailing list<br>
<a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
<a href="https://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" target="_blank">https://mailman.nginx.org/mailman/listinfo/nginx</a><br>
</blockquote></div>