[PATCH 1 of 3] Correctly calculate and set Age header
Maxim Dounin
mdounin at mdounin.ru
Thu Jul 18 20:49:13 UTC 2024
Hello!
On Thu, Jul 18, 2024 at 09:51:53PM +0900, Hiroaki Nakamura wrote:
> Hello,
>
> 2024年7月18日(木) 9:26 Maxim Dounin <mdounin at mdounin.ru>:
> >
> > Hello!
> >
> > On Mon, Jul 01, 2024 at 10:45:59PM +0900, Hiroaki Nakamura wrote:
> >
> > > Hello
> > >
> > > 2024年6月27日(木) 9:32 Maxim Dounin <mdounin at mdounin.ru>:
> [...]
> >
> > Thanks for yet another version of the patch. In its current form
> > it starts to look reasonable.
> >
> > I've spent some time looking into this, including looking through
> > previous attempts to implement some forms of Age support, notably:
> [...]
>
> Thank you for taking your time to investigate and implement Age support!
>
> >
> > Overall, there seems to be two major parts of what is expected to
> > be done here:
> >
> > 1. Adjusting cache validity times based on the Age header in the
> > upstream server response.
> >
> > 2. Returning cached responses with updated/added Age header.
> >
> > These two parts probably can be implemented separately, with
> > minimal interdependencies.
>
> I agree.
>
> >
> > Further, I tend to think that, while (1) probably can be the
> > default, it might not be a good idea to do (2) by default, as in
> > most setups [free]nginx is used as an origin server, even if
> > caching is enabled. In such configurations administrators often
> > use "Cache-Control: max-age=N" for client-side caching, and
> > control caching in [free]nginx separately, such as with
> > proxy_cache_valid. Adding the Age header in such configurations
> > might result in unexpectedly non-working client-side caching.
> >
> > In particular, (2) can be implemented as a variable with relevant
> > information to be used in the "add_header" directive if one needs
> > to implement Age updating. Alternative approach would be to add
> > per-module directives to control (2), such as "proxy_cache_age on|off",
> > and add the Age header in the upstream module, though a variable
> > looks easier.
>
> I agree the latter approach is better.
>
> >
> > Below are two patches that implement (1) adjusting cache validity
> > base on the Age header and (2) a variable to set the Age header,
> > and yet another patch to implement tests. Please take a look.
> [...]
>
> I have looked at the two patches and the test.
> Yes, it achieves what you described and it is a lot simpler and clearer
> than my patches.
> I also have verified that it covers both cases of orders of
> cache-control and age headers, that is cache-control before age and
> cache-control after age.
>
> I have run the test with freenginx with the two patches applied and
> verified it passes.
>
> I wrote tests using HTTP/2 and HTTP/3 based on your test and
> verified these tests pass too.
> That means we don't need to modify http_v2_module and http_v3_module.
>
> In summary, your patches and tests seem perfect to me.
Thanks for the review, committed. And thanks for prodding this,
appreciated.
Just in case, below is a preliminary documentation patch.
# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1721325059 -10800
# Thu Jul 18 20:50:59 2024 +0300
# Node ID 653583426bf2a93ee6bb6796881a4633cfca5503
# Parent 6cab598f14a1f4bbf2e8e4f374f6d15c18c54659
Documented Age header handling.
diff --git a/xml/en/docs/http/ngx_http_fastcgi_module.xml b/xml/en/docs/http/ngx_http_fastcgi_module.xml
--- a/xml/en/docs/http/ngx_http_fastcgi_module.xml
+++ b/xml/en/docs/http/ngx_http_fastcgi_module.xml
@@ -10,7 +10,7 @@
<module name="Module ngx_http_fastcgi_module"
link="/en/docs/http/ngx_http_fastcgi_module.html"
lang="en"
- rev="54">
+ rev="55">
<section id="summary">
@@ -649,6 +649,12 @@ with another value, such a response will
taking into account the corresponding request header fields (1.7.7).
</listitem>
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
</list>
Processing of one or more of these response header fields can be disabled
using the <link id="fastcgi_ignore_headers"/> directive.
@@ -766,7 +772,7 @@ The following fields can be ignored: <he
<header>X-Accel-Buffering</header> (1.1.6),
<header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
</para>
<para>
@@ -777,7 +783,7 @@ effect:
<listitem>
<header>X-Accel-Expires</header>, <header>Expires</header>,
<header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
set the parameters of response <link id="fastcgi_cache_valid">caching</link>;
</listitem>
diff --git a/xml/en/docs/http/ngx_http_proxy_module.xml b/xml/en/docs/http/ngx_http_proxy_module.xml
--- a/xml/en/docs/http/ngx_http_proxy_module.xml
+++ b/xml/en/docs/http/ngx_http_proxy_module.xml
@@ -10,7 +10,7 @@
<module name="Module ngx_http_proxy_module"
link="/en/docs/http/ngx_http_proxy_module.html"
lang="en"
- rev="77">
+ rev="78">
<section id="summary">
@@ -667,6 +667,12 @@ with another value, such a response will
taking into account the corresponding request header fields (1.7.7).
</listitem>
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
</list>
Processing of one or more of these response header fields can be disabled
using the <link id="proxy_ignore_headers"/> directive.
@@ -1011,7 +1017,7 @@ The following fields can be ignored: <he
<header>X-Accel-Buffering</header> (1.1.6),
<header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
</para>
<para>
@@ -1022,7 +1028,7 @@ effect:
<listitem>
<header>X-Accel-Expires</header>, <header>Expires</header>,
<header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
set the parameters of response <link id="proxy_cache_valid">caching</link>;
</listitem>
diff --git a/xml/en/docs/http/ngx_http_scgi_module.xml b/xml/en/docs/http/ngx_http_scgi_module.xml
--- a/xml/en/docs/http/ngx_http_scgi_module.xml
+++ b/xml/en/docs/http/ngx_http_scgi_module.xml
@@ -10,7 +10,7 @@
<module name="Module ngx_http_scgi_module"
link="/en/docs/http/ngx_http_scgi_module.html"
lang="en"
- rev="42">
+ rev="43">
<section id="summary">
@@ -640,6 +640,12 @@ with another value, such a response will
taking into account the corresponding request header fields (1.7.7).
</listitem>
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
</list>
Processing of one or more of these response header fields can be disabled
using the <link id="scgi_ignore_headers"/> directive.
@@ -731,7 +737,7 @@ The following fields can be ignored: <he
<header>X-Accel-Buffering</header> (1.1.6),
<header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
</para>
<para>
@@ -742,7 +748,7 @@ effect:
<listitem>
<header>X-Accel-Expires</header>, <header>Expires</header>,
<header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
set the parameters of response <link id="scgi_cache_valid">caching</link>;
</listitem>
diff --git a/xml/en/docs/http/ngx_http_upstream_module.xml b/xml/en/docs/http/ngx_http_upstream_module.xml
--- a/xml/en/docs/http/ngx_http_upstream_module.xml
+++ b/xml/en/docs/http/ngx_http_upstream_module.xml
@@ -10,7 +10,7 @@
<module name="Module ngx_http_upstream_module"
link="/en/docs/http/ngx_http_upstream_module.html"
lang="en"
- rev="90">
+ rev="91">
<section id="summary">
@@ -574,6 +574,12 @@ are separated by commas and colons like
<link id="var_upstream_addr">$upstream_addr</link> variable.
</tag-desc>
+<tag-name id="var_upstream_cache_age"><var>$upstream_cache_age</var>
+</tag-name>
+<tag-desc>
+age of the cache item (1.27.3).
+</tag-desc>
+
<tag-name id="var_upstream_cache_key"><var>$upstream_cache_key</var>
</tag-name>
<tag-desc>
diff --git a/xml/en/docs/http/ngx_http_uwsgi_module.xml b/xml/en/docs/http/ngx_http_uwsgi_module.xml
--- a/xml/en/docs/http/ngx_http_uwsgi_module.xml
+++ b/xml/en/docs/http/ngx_http_uwsgi_module.xml
@@ -10,7 +10,7 @@
<module name="Module ngx_http_uwsgi_module"
link="/en/docs/http/ngx_http_uwsgi_module.html"
lang="en"
- rev="51">
+ rev="52">
<section id="summary">
@@ -640,6 +640,12 @@ with another value, such a response will
taking into account the corresponding request header fields (1.7.7).
</listitem>
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
</list>
Processing of one or more of these response header fields can be disabled
using the <link id="uwsgi_ignore_headers"/> directive.
@@ -731,7 +737,7 @@ The following fields can be ignored: <he
<header>X-Accel-Buffering</header> (1.1.6),
<header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
</para>
<para>
@@ -742,7 +748,7 @@ effect:
<listitem>
<header>X-Accel-Expires</header>, <header>Expires</header>,
<header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
set the parameters of response <link id="uwsgi_cache_valid">caching</link>;
</listitem>
diff --git a/xml/ru/docs/http/ngx_http_fastcgi_module.xml b/xml/ru/docs/http/ngx_http_fastcgi_module.xml
--- a/xml/ru/docs/http/ngx_http_fastcgi_module.xml
+++ b/xml/ru/docs/http/ngx_http_fastcgi_module.xml
@@ -10,7 +10,7 @@
<module name="Модуль ngx_http_fastcgi_module"
link="/ru/docs/http/ngx_http_fastcgi_module.html"
lang="ru"
- rev="54">
+ rev="55">
<section id="summary">
@@ -649,6 +649,12 @@ fastcgi_cache_valid any 1m;
с учётом соответствующих полей заголовка запроса (1.7.7).
</listitem>
+<listitem>
+Если в заголовке ответа есть поле <header>Age</header>
+и время кэширования определяется по полю <header>Cache-Control</header>,
+время кэширования будет соответственно уменьшено (1.27.3).
+</listitem>
+
</list>
Обработка одного или более из этих полей заголовка может быть отключена
при помощи директивы <link id="fastcgi_ignore_headers"/>.
@@ -764,8 +770,8 @@ nginx не передаёт клиенту поля заголовка <header>Status</header> и
<header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
<header>X-Accel-Buffering</header> (1.1.6),
<header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-и <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) и <header>Age</header> (1.27.3).
</para>
<para>
@@ -774,8 +780,8 @@ nginx не передаёт клиенту поля заголовка <header>Status</header> и
<listitem>
<header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-и <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> и <header>Age</header>
задают параметры <link id="fastcgi_cache_valid">кэширования</link> ответа;
</listitem>
diff --git a/xml/ru/docs/http/ngx_http_proxy_module.xml b/xml/ru/docs/http/ngx_http_proxy_module.xml
--- a/xml/ru/docs/http/ngx_http_proxy_module.xml
+++ b/xml/ru/docs/http/ngx_http_proxy_module.xml
@@ -10,7 +10,7 @@
<module name="Модуль ngx_http_proxy_module"
link="/ru/docs/http/ngx_http_proxy_module.html"
lang="ru"
- rev="77">
+ rev="78">
<section id="summary">
@@ -669,6 +669,12 @@ proxy_cache_valid any 1m;
с учётом соответствующих полей заголовка запроса (1.7.7).
</listitem>
+<listitem>
+Если в заголовке ответа есть поле <header>Age</header>
+и время кэширования определяется по полю <header>Cache-Control</header>,
+время кэширования будет соответственно уменьшено (1.27.3).
+</listitem>
+
</list>
Обработка одного или более из этих полей заголовка может быть отключена
при помощи директивы <link id="proxy_ignore_headers"/>.
@@ -1010,8 +1016,8 @@ NTLM</link> рекомендуется версия 1.1.
<header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
<header>X-Accel-Buffering</header> (1.1.6),
<header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-и <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) и <header>Age</header> (1.27.3).
</para>
<para>
@@ -1020,8 +1026,8 @@ NTLM</link> рекомендуется версия 1.1.
<listitem>
<header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-и <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> и <header>Age</header>
задают параметры <link id="proxy_cache_valid">кэширования</link> ответа;
</listitem>
diff --git a/xml/ru/docs/http/ngx_http_scgi_module.xml b/xml/ru/docs/http/ngx_http_scgi_module.xml
--- a/xml/ru/docs/http/ngx_http_scgi_module.xml
+++ b/xml/ru/docs/http/ngx_http_scgi_module.xml
@@ -10,7 +10,7 @@
<module name="Модуль ngx_http_scgi_module"
link="/ru/docs/http/ngx_http_scgi_module.html"
lang="ru"
- rev="42">
+ rev="43">
<section id="summary">
@@ -640,6 +640,12 @@ scgi_cache_valid any 1m;
с учётом соответствующих полей заголовка запроса (1.7.7).
</listitem>
+<listitem>
+Если в заголовке ответа есть поле <header>Age</header>
+и время кэширования определяется по полю <header>Cache-Control</header>,
+время кэширования будет соответственно уменьшено (1.27.3).
+</listitem>
+
</list>
Обработка одного или более из этих полей заголовка может быть отключена
при помощи директивы <link id="scgi_ignore_headers"/>.
@@ -729,8 +735,8 @@ nginx не передаёт клиенту поля заголовка <header>Status</header> и
<header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
<header>X-Accel-Buffering</header> (1.1.6),
<header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-и <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) и <header>Age</header> (1.27.3).
</para>
<para>
@@ -739,8 +745,8 @@ nginx не передаёт клиенту поля заголовка <header>Status</header> и
<listitem>
<header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-и <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> и <header>Age</header>
задают параметры <link id="scgi_cache_valid">кэширования</link> ответа;
</listitem>
diff --git a/xml/ru/docs/http/ngx_http_upstream_module.xml b/xml/ru/docs/http/ngx_http_upstream_module.xml
--- a/xml/ru/docs/http/ngx_http_upstream_module.xml
+++ b/xml/ru/docs/http/ngx_http_upstream_module.xml
@@ -10,7 +10,7 @@
<module name="Модуль ngx_http_upstream_module"
link="/ru/docs/http/ngx_http_upstream_module.html"
lang="ru"
- rev="90">
+ rev="91">
<section id="summary">
@@ -581,6 +581,12 @@ server {
<link id="var_upstream_addr">$upstream_addr</link>.
</tag-desc>
+<tag-name id="var_upstream_cache_age"><var>$upstream_cache_age</var>
+</tag-name>
+<tag-desc>
+возраст элемента кэша (1.27.3).
+</tag-desc>
+
<tag-name id="var_upstream_cache_key"><var>$upstream_cache_key</var>
</tag-name>
<tag-desc>
diff --git a/xml/ru/docs/http/ngx_http_uwsgi_module.xml b/xml/ru/docs/http/ngx_http_uwsgi_module.xml
--- a/xml/ru/docs/http/ngx_http_uwsgi_module.xml
+++ b/xml/ru/docs/http/ngx_http_uwsgi_module.xml
@@ -10,7 +10,7 @@
<module name="Модуль ngx_http_uwsgi_module"
link="/ru/docs/http/ngx_http_uwsgi_module.html"
lang="ru"
- rev="51">
+ rev="52">
<section id="summary">
@@ -639,6 +639,12 @@ uwsgi_cache_valid any 1m;
с учётом соответствующих полей заголовка запроса (1.7.7).
</listitem>
+<listitem>
+Если в заголовке ответа есть поле <header>Age</header>
+и время кэширования определяется по полю <header>Cache-Control</header>,
+время кэширования будет соответственно уменьшено (1.27.3).
+</listitem>
+
</list>
Обработка одного или более из этих полей заголовка может быть отключена
при помощи директивы <link id="uwsgi_ignore_headers"/>.
@@ -728,8 +734,8 @@ nginx не передаёт клиенту поля заголовка <header>Status</header> и
<header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
<header>X-Accel-Buffering</header> (1.1.6),
<header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-и <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) и <header>Age</header> (1.27.3).
</para>
<para>
@@ -738,8 +744,8 @@ nginx не передаёт клиенту поля заголовка <header>Status</header> и
<listitem>
<header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-и <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> и <header>Age</header>
задают параметры <link id="uwsgi_cache_valid">кэширования</link> ответа;
</listitem>
--
Maxim Dounin
http://mdounin.ru/
More information about the nginx-devel
mailing list