From hnakamur at gmail.com  Mon Jul  1 13:45:59 2024
From: hnakamur at gmail.com (Hiroaki Nakamura)
Date: Mon, 1 Jul 2024 22:45:59 +0900
Subject: [PATCH 1 of 3] Correctly calculate and set Age header
In-Reply-To: <Znywy92srHcefS69@mdounin.ru>
References: <CAN-DUMQwyKd07fkbpa5osUnAH0sJvdwT=QgO6aGtCpqmn2iBNw@mail.gmail.com>
 <Zniy3EFG005hIsXl@mdounin.ru>
 <CAN-DUMQzHifqXq4Uc+gRQqNG4pKuNcz=Zwz_+K+Srx-QKWW34Q@mail.gmail.com>
 <Znywy92srHcefS69@mdounin.ru>
Message-ID: <CAN-DUMRpSuOnYWe3O2_X7x-P_BMwQ4=h7RYm2W_rNqTP8KReaQ@mail.gmail.com>

Hello

2024?6?27?(?) 9:32 Maxim Dounin <mdounin at mdounin.ru>:
> [...]
> The commit message certainly needs more work.  Still, you may want
> to postpone it, as well as code modifications, before various
> conceptual questions are cleaned up.

I have updated the commit log again.

> > > > @@ -75,6 +77,8 @@
> > > >      time_t                           error_sec;
> > > >      time_t                           last_modified;
> > > >      time_t                           date;
> > > > +    time_t                           response_time;
> [...]
> The "date" field is set when the cache item is created, and
> can be updated on cache item revalidation (with
> "proxy_cache_revalidate on").  It is not updated when sending a
> cached response.

Thank you, I had misunderstood this.
I take this into consideration and I found that we can keep
ngx_http_file_cache_header_t as is if we ignore
corrected_initial_age concept and just use the Age header from the
upstream as is (see below).

> [...]
> Note that "too many debug logging" comment still applies.

I have deleted debug logs.

> [...]
>
> > > > diff -r 73929a4f3447 -r c81df54e3d03 src/http/ngx_http_upstream.c
> > > > --- a/src/http/ngx_http_upstream.c      Thu Jun 20 20:26:24 2024 +0900
> > > > +++ b/src/http/ngx_http_upstream.c      Thu Jun 20 20:26:41 2024 +0900
> > > > @@ -1549,6 +1559,7 @@
> > > >      ngx_memzero(u->state, sizeof(ngx_http_upstream_state_t));
> > > >
> > > >      u->start_time = ngx_current_msec;
> > > > +    u->request_time = ngx_time();
> > >
> > > Adding a duplicate start time shouldn't be needed.
> >
> > It is not a duplicate since u->request_time is real clock time as opposed to
> > u->start_time is monotonic clock.
> >
> > We need a real clock time of response_time since we calculate
> > the difference from the upstream date value.
> >
> > Also we use the calculation response_time - request_time
> > when the upstream date is not present or not well synchronized (see below),
> > so we also need request_time to be a real clock time.
>
> Actually, monotonic clock is much better suited for time
> difference calculation, and that's exactly the reason why it is
> used for u->start_time.  And, since at any particular moment you
> know u->start_time, current monotonic time and current wall clock
> time, it is trivial to calculate wall clock start time if needed
> (but I doubt it's actually needed, see below).
>
> [...]
>
> > > > @@ -2648,7 +2663,12 @@
> > > >              valid = ngx_http_file_cache_valid(u->conf->cache_valid,
> > > >                                                u->headers_in.status_n);
> > > >              if (valid) {
> > > > -                valid = now + valid;
> > > > +                ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> > > > +                               "adjust cache valid_sec:%O, "
> > > > +                               "valid:%O, init_age:%d for 304",
> > > > +                               now + valid - r->cache->corrected_initial_age,
> > > > +                               valid, r->cache->corrected_initial_age);
> > > > +                valid = now + valid - r->cache->corrected_initial_age;
> > >
> > > Not sure if Age calculations should apply to caching time as
> > > calculated with proxy_cache_valid directives, but if it does, it
> > > should apply consistently, in all cases where proxy_cache_valid
> > > times are used.  At least ngx_http_upstream_intercept_errors()
> > > case seems to be ignored in the patch.
> >
> > I have updated ngx_http_upstream_intercept_errors as well.
> >
> > > Also, if Age applies, there should be a way to ignore it,
> > > similarly to how Cache-Control can be ignored with the
> > > proxy_ignore_headers directive.
> >
> > I do not know how to do this.
> > Maybe add
> >     { ngx_string("Age"), NGX_HTTP_UPSTREAM_IGN_AGE },
> > to ngx_http_upstream_ignore_headers_masks?
>
> If at all, yes, this probably should be implemented like this.
>
> Still, for now I'm no convinced that Age needs to be taken into
> account for proxy_cache_valid.
>
> For example, consider a setup with two caching servers: an edge
> cache, a large central cache, and an origin server.  Assuming a
> response without any Cache-Control headers, something like
>
>     proxy_cache_valid 200 1h;
>
> on the edge cache, and
>
>     proxy_cache_valid 200 1d;
>
> on the large cache.
>
> With the suggested patch there will be Age header generated for
> responses returned by the large cache, effectively preventing
> caching on the edge cache.
>
> While such (obviously suboptimal and not desired) behaviour can be
> prevented by using "proxy_ignore_headers Age;" (assuming it is
> implemented), this would require extra otherwise unneeded
> configuration.  Further, this will prevent proper use of the Age
> header for Cache-Control-based caching for other requests.
>
> A better approach might be to limit the Age header to
> Cache-Control-based caching only.  Not sure though, and it
> certainly needs more analysis.

Thank you for your explanation.

I think now I understand an important difference between
proxy_cache_valid and Cache-Control.
The start time of the cache period of proxy_cache_valid is when nginx
receives a response while the start time of Cache-Control
max-age and s-maxage is when sending a response from the
origin server.

I have modified the patch to limit adjusting valid_sec by decreasing
the value of Age header to Cache-Control-based caching only.

> [...]
> This looks slightly more reasonable.  At least, this won't result
> in obviously bogus age values if clocks are not synchronized.
>
> Still, a simpler and mostly equivalent solution might be to avoid
> the whole "corrected_initial_age" concept, and use just Age as is,
> combined with the time passed since the cache item was
> created/revalidated.  The time in question is when the response
> headers were received, and usually a good approximation of the
> time when the response was actually generated.

Alright, I have changed the patch to ignore apparent_age and
response_delay and just use the upstream age.

When sending a cached response, the Age value is set to
the initial Age value plus resident time.

In this implementation, the modification of ngx_http_file_cache_header_t
is not needed since we can just use the value of the date field and
the Age header value stored in the cached response.

>
> > > > +#if (NGX_HTTP_CACHE)
> > > > +    if (r->cache) {
> > > > +        r->cache->response_time = response_time;
> > > > +        r->cache->corrected_initial_age = corrected_initial_age;
> > > > +        if (u->headers_in.adjusting_valid_sec) {
> > > > +            r->cache->valid_sec -= corrected_initial_age;
> > >
> > > The "adjusting_valid_sec" logic looks unclear (probably at least
> > > needs a better name) and fragile.  If I'm reading the code
> > > correctly, it will do the wrong thing at least if the upstream
> > > server returns something like:
> > >
> > > Cache-Control: max-age=60
> > > X-Accel-Expires: @<time>
> > >
> > > since u->headers_in.adjusting_valid_sec won't be cleared.
> >
> > I found that we must ignore Expires when Cache-Control with max-age or s-maxage
> > is set according to https://www.rfc-editor.org/rfc/rfc9111#name-expires:
> >
> > "If a response includes a Cache-Control header field with the max-age directive
> > (Section 5.2.2.1), a recipient MUST ignore the Expires header field. Likewise,
> > if a response includes the s-maxage directive (Section 5.2.2.10), a shared cache
> > recipient MUST ignore the Expires header field."
>
> The Expires header and X-Accel-Expires header are different
> headers.  The X-Accel-Expires header is a non-standard header
> which takes precedence over both Expires and Cache-Control.

I have fixed the patch to reset the relative_freshness flag for the
X-Accel-Expires header.

>
> [...]
>
> > > > +            ngx_log_debug2(NGX_LOG_DEBUG_HTTP, u->peer.connection->log, 0,
> > > > +                           "http upstream adjusted cache "
> > > > +                           "valid_sec:%O, init_age:%O",
> > > > +                           r->cache->valid_sec, corrected_initial_age);
> > > > +        }
> > > > +    }
> > > > +#endif
> > >
> > > If I'm reading the code correctly, this will add the Age
> > > header to responses which are not cached.  This somewhat
> > > contradicts to what RFC 9111 says in the description of the Age
> > > header (https://www.rfc-editor.org/rfc/rfc9111.html#section-5.1-6):
> > >
> > > : The presence of an Age header field implies that the response was
> > > : not generated or validated by the origin server for this request.
> > >
> > > Please also note that in many cases [free]nginx is expected to be
> > > seen as the origin server, even if it caches some upstream server
> > > responses according to the configuration (in many cases, ignoring
> > > Cache-Control or Expires headers).  In this case using the
> > > Age header even on cached responses might not be desired.  It
> > > would be great to dig further into possible use cases and make
> > > sure all are properly covered.
> >
> > I have changed to not add Age if the calculated value is zero and also
> > updated tests.
>
> Note that this somewhat contradicts special handling of "Age: 0"
> in HTTP/3 part of your patch.
>
> Also, if I'm reading the patch correctly, as currently implemented
> the code might result in non-zero Age even for directly returned
> non-cached responses.

I have changed the modification of the Age header as below:

* pass through the Age header from the upstream when receiving
  a response from the upstream.
* set the Age header (may be 0) when sending a cached response.

>
> Also, "Age: 0" might legitimately appear for cached responses
> during the first second after the response was cached.  Not sure
> if it needs to be returned, but given the about RFC quote it might
> be usable for some.

Yes, I think "Age: 0" is valid in this case.

* https://www.rfc-editor.org/rfc/rfc9111#name-age
  Age = delta-seconds
  delta-seconds = 1*DIGIT
* https://www.rfc-editor.org/rfc/rfc9204.html#static-table
  The value of 0 for the Age header can be omitted.


>
> Note well that the "would be great to dig further" comment still
> applies.
>
> [...]
>
> Hope this helps.
>
> --
> Maxim Dounin
> http://mdounin.ru/

Best regards,

--
Hiroaki Nakamura

From hnakamur at gmail.com  Mon Jul  1 13:46:54 2024
From: hnakamur at gmail.com (Hiroaki Nakamura)
Date: Mon, 1 Jul 2024 22:46:54 +0900
Subject: [PATCH 1 of 2] Cache: added calculation of the Age header
In-Reply-To: <CAN-DUMRpSuOnYWe3O2_X7x-P_BMwQ4=h7RYm2W_rNqTP8KReaQ@mail.gmail.com>
References: <CAN-DUMQwyKd07fkbpa5osUnAH0sJvdwT=QgO6aGtCpqmn2iBNw@mail.gmail.com>
 <Zniy3EFG005hIsXl@mdounin.ru>
 <CAN-DUMQzHifqXq4Uc+gRQqNG4pKuNcz=Zwz_+K+Srx-QKWW34Q@mail.gmail.com>
 <Znywy92srHcefS69@mdounin.ru>
 <CAN-DUMRpSuOnYWe3O2_X7x-P_BMwQ4=h7RYm2W_rNqTP8KReaQ@mail.gmail.com>
Message-ID: <CAN-DUMQLOvcNWcWYRDxY-w7V_pUoVh9cbkBYjo-doaTmwVf7OA@mail.gmail.com>

# HG changeset patch
# User Hiroaki Nakamura <hnakamur at gmail.com>
# Date 1719839232 -32400
#      Mon Jul 01 22:07:12 2024 +0900
# Node ID 685f47b210263988886db6fe15a95e177616de1f
# Parent  cef4919b830f6e149d7811fa22130da4135f4f90
Cache: added calculation of the Age header.
* Implement the modified calculation of the Age header as specified in
  RFC 9111 "HTTP Caching", https://www.rfc-editor.org/rfc/rfc9111.html

  Calculation initial age as specified in RFC 9111:

  apparent_age = max(0, response_time - date_value);

  response_delay = response_time - request_time;
  corrected_age_value = age_value + response_delay;

  corrected_initial_age = max(apparent_age, corrected_age_value);

  However, we ignore apparent_age and response_delay and just use:

  corrected_initial_age = age_value;

  We adjust valid_sec of ngx_http_cache_t by decreasing by the
  age value received from the upstream only when Cache-Control
  max-age or s-maxage is used.
  (For proxy_cache_valid directive, Expires, and X-Accel-Expires
   we do not adjust valid_sec).

  When sending a cached response, we calculate the Age
  as below:

  resident_time = now - response_time;
  current_age = initial_age + resident_time;

  where response_time is cache_creation_time.

* Also ignore Expires when Cache-Control max-age
  or s-maxage is set as specified in
  https://www.rfc-editor.org/rfc/rfc9111#name-expires:

  "If a response includes a Cache-Control header field with the
  max-age directive (Section 5.2.2.1), a recipient MUST ignore
  the Expires header field. Likewise, if a response includes the
  s-maxage directive (Section 5.2.2.10), a shared cache recipient
  MUST ignore the Expires header field."

  Note X-Accel-Expires has higher precedence over Cache-Control
  so the modification of the Age header is not applied.

diff -r cef4919b830f -r 685f47b21026 src/http/ngx_http_header_filter_module.c
--- a/src/http/ngx_http_header_filter_module.c    Mon Jul 01 22:07:03 2024 +0900
+++ b/src/http/ngx_http_header_filter_module.c    Mon Jul 01 22:07:12 2024 +0900
@@ -322,6 +322,10 @@ ngx_http_header_filter(ngx_http_request_
         len += sizeof("Last-Modified: Mon, 28 Sep 1970 06:00:00 GMT" CRLF) - 1;
     }

+    if (r->headers_out.age_n != -1) {
+        len += sizeof("Age: ") - 1 + NGX_TIME_T_LEN + 2;
+    }
+
     c = r->connection;

     if (r->headers_out.location
@@ -518,6 +522,10 @@ ngx_http_header_filter(ngx_http_request_
         *b->last++ = CR; *b->last++ = LF;
     }

+    if (r->headers_out.age_n != -1) {
+        b->last = ngx_sprintf(b->last, "Age: %T" CRLF, r->headers_out.age_n);
+    }
+
     if (host.data) {

         p = b->last + sizeof("Location: ") - 1;
diff -r cef4919b830f -r 685f47b21026 src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c  Mon Jul 01 22:07:03 2024 +0900
+++ b/src/http/ngx_http_request.c  Mon Jul 01 22:07:12 2024 +0900
@@ -646,6 +646,7 @@ ngx_http_alloc_request(ngx_connection_t
     r->headers_in.keep_alive_n = -1;
     r->headers_out.content_length_n = -1;
     r->headers_out.last_modified_time = -1;
+    r->headers_out.age_n = -1;

     r->uri_changes = NGX_HTTP_MAX_URI_CHANGES + 1;
     r->subrequests = NGX_HTTP_MAX_SUBREQUESTS + 1;
diff -r cef4919b830f -r 685f47b21026 src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h  Mon Jul 01 22:07:03 2024 +0900
+++ b/src/http/ngx_http_request.h  Mon Jul 01 22:07:12 2024 +0900
@@ -291,6 +291,7 @@ typedef struct {
     off_t                             content_offset;
     time_t                            date_time;
     time_t                            last_modified_time;
+    time_t                            age_n;
 } ngx_http_headers_out_t;


diff -r cef4919b830f -r 685f47b21026 src/http/ngx_http_special_response.c
--- a/src/http/ngx_http_special_response.c   Mon Jul 01 22:07:03 2024 +0900
+++ b/src/http/ngx_http_special_response.c   Mon Jul 01 22:07:12 2024 +0900
@@ -581,6 +581,7 @@ ngx_http_clean_header(ngx_http_request_t

     r->headers_out.content_length_n = -1;
     r->headers_out.last_modified_time = -1;
+    r->headers_out.age_n = -1;
 }


diff -r cef4919b830f -r 685f47b21026 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c Mon Jul 01 22:07:03 2024 +0900
+++ b/src/http/ngx_http_upstream.c Mon Jul 01 22:07:12 2024 +0900
@@ -136,6 +136,8 @@ static ngx_int_t
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_upstream_process_vary(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
+static ngx_int_t ngx_http_upstream_process_age(ngx_http_request_t *r,
+    ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_upstream_copy_header_line(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t
@@ -323,6 +325,10 @@ static ngx_http_upstream_header_t  ngx_h
                  ngx_http_upstream_copy_header_line,
                  offsetof(ngx_http_headers_out_t, content_encoding), 0 },

+    { ngx_string("Age"),
+                 ngx_http_upstream_process_age, 0,
+                 ngx_http_upstream_ignore_header_line, 0, 0 },
+
     { ngx_null_string, NULL, 0, NULL, 0, 0 }
 };

@@ -507,6 +513,7 @@ ngx_http_upstream_create(ngx_http_reques

     u->headers_in.content_length_n = -1;
     u->headers_in.last_modified_time = -1;
+    u->headers_in.age_n = -1;

     return NGX_OK;
 }
@@ -961,6 +968,7 @@ ngx_http_upstream_cache(ngx_http_request
         c->updating_sec = 0;
         c->error_sec = 0;

+        u->headers_in.relative_freshness = 0;
         u->buffer.start = NULL;
         u->cache_status = NGX_HTTP_CACHE_EXPIRED;

@@ -1057,6 +1065,7 @@ ngx_http_upstream_cache_get(ngx_http_req
 static ngx_int_t
 ngx_http_upstream_cache_send(ngx_http_request_t *r, ngx_http_upstream_t *u)
 {
+    time_t             age_when_revalidated;
     ngx_int_t          rc;
     ngx_http_cache_t  *c;

@@ -1068,6 +1077,8 @@ ngx_http_upstream_cache_send(ngx_http_re
         return ngx_http_cache_send(r);
     }

+    age_when_revalidated = u->headers_in.age_n;
+
     /* TODO: cache stack */

     u->buffer = *c->buf;
@@ -1076,6 +1087,7 @@ ngx_http_upstream_cache_send(ngx_http_re
     ngx_memzero(&u->headers_in, sizeof(ngx_http_upstream_headers_in_t));
     u->headers_in.content_length_n = -1;
     u->headers_in.last_modified_time = -1;
+    u->headers_in.age_n = -1;

     if (ngx_list_init(&u->headers_in.headers, r->pool, 8,
                       sizeof(ngx_table_elt_t))
@@ -1094,6 +1106,10 @@ ngx_http_upstream_cache_send(ngx_http_re
     rc = u->process_header(r);

     if (rc == NGX_OK) {
+        if (u->cache_status == NGX_HTTP_CACHE_REVALIDATED) {
+            /* use age from upstream when revalidated */
+            u->headers_in.age_n = age_when_revalidated;
+        }

         rc = ngx_http_upstream_process_headers(r, u);

@@ -2016,6 +2032,7 @@ ngx_http_upstream_reinit(ngx_http_reques
     ngx_memzero(&u->headers_in, sizeof(ngx_http_upstream_headers_in_t));
     u->headers_in.content_length_n = -1;
     u->headers_in.last_modified_time = -1;
+    u->headers_in.age_n = -1;

     if (ngx_list_init(&u->headers_in.headers, r->pool, 8,
                       sizeof(ngx_table_elt_t))
@@ -2977,6 +2994,42 @@ ngx_http_upstream_process_headers(ngx_ht
     r->headers_out.status_line = u->headers_in.status_line;

     r->headers_out.content_length_n = u->headers_in.content_length_n;
+    r->headers_out.age_n = u->headers_in.age_n;
+
+#if (NGX_HTTP_CACHE)
+    if (r->cache) {
+        ngx_http_cache_t  *c;
+
+        c = r->cache;
+
+        if (u->cache_status == NGX_HTTP_CACHE_HIT
+            || u->cache_status == NGX_HTTP_CACHE_STALE)
+        {
+            time_t  resident_time, age_value;
+
+            /*
+             * Update age response header.
+             * https://www.rfc-editor.org/rfc/rfc9111.html#name-calculating-age
+             *
+             * resident_time = now - response_time;
+             * current_age = corrected_initial_age + resident_time;
+             */
+            age_value = u->headers_in.age_n != -1 ? u->headers_in.age_n : 0;
+            resident_time = ngx_time() - c->date;
+            r->headers_out.age_n = age_value + resident_time;
+
+        } else if (u->cache_status == NGX_HTTP_CACHE_MISS
+                   || u->cache_status == NGX_HTTP_CACHE_REVALIDATED)
+        {
+            if (u->headers_in.relative_freshness) {
+                time_t  age_value;
+                /* we treat non existent age header as age: 0 */
+                age_value = u->headers_in.age_n != -1 ?
u->headers_in.age_n : 0;
+                c->valid_sec -= age_value;
+            }
+        }
+    }
+#endif

     r->disable_not_modified = !u->cacheable;

@@ -4945,6 +4998,7 @@ ngx_http_upstream_process_cache_control(
         }

         r->cache->valid_sec = ngx_time() + n;
+        u->headers_in.relative_freshness = 1;
         u->headers_in.expired = 0;
     }

@@ -5047,7 +5101,18 @@ ngx_http_upstream_process_expires(ngx_ht
         return NGX_OK;
     }

-    r->cache->valid_sec = expires;
+    /*
+     * https://www.rfc-editor.org/rfc/rfc9111#name-expires
+     *
+     * If a response includes a Cache-Control header field with the max-age
+     * directive (Section 5.2.2.1), a recipient MUST ignore the Expires
+     * header field. Likewise, if a response includes the s-maxage directive
+     * (Section 5.2.2.10), a shared cache recipient MUST ignore the Expires
+     * header field.
+     */
+    if (!u->headers_in.relative_freshness) {
+        r->cache->valid_sec = expires;
+    }
     }
 #endif

@@ -5119,7 +5184,8 @@ ngx_http_upstream_process_accel_expires(
     n = ngx_atoi(p, len);

     if (n != NGX_ERROR) {
-        r->cache->valid_sec = n;
+        /* X-Accel-Expires has higher precedence over Cache-Control. */
+        u->headers_in.relative_freshness = 0;
         u->headers_in.no_cache = 0;
         u->headers_in.expired = 0;
     }
@@ -5373,6 +5439,38 @@ ngx_http_upstream_process_vary(ngx_http_


 static ngx_int_t
+ngx_http_upstream_process_age(ngx_http_request_t *r,
+    ngx_table_elt_t *h, ngx_uint_t offset)
+{
+    ngx_http_upstream_t  *u;
+
+    u = r->upstream;
+
+    if (u->headers_in.age) {
+        ngx_log_error(NGX_LOG_WARN, r->connection->log, 0,
+                      "ignore duplicate age header from upstream: \"%V: %V\", "
+                      "previous value: \"%V: %V\"",
+                      &h->key, &h->value,
+                      &u->headers_in.age->key,
+                      &u->headers_in.age->value);
+        return NGX_OK;
+    }
+
+    h->next = NULL;
+    u->headers_in.age = h;
+    u->headers_in.age_n = ngx_atotm(h->value.data, h->value.len);
+
+    if (u->headers_in.age_n == NGX_ERROR) {
+        ngx_log_error(NGX_LOG_WARN, r->connection->log, 0,
+                      "ignore invalid \"Age\" header from upstream: "
+                      "\"%V: %V\"", &h->key, &h->value);
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_http_upstream_copy_header_line(ngx_http_request_t *r, ngx_table_elt_t *h,
     ngx_uint_t offset)
 {
diff -r cef4919b830f -r 685f47b21026 src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h Mon Jul 01 22:07:03 2024 +0900
+++ b/src/http/ngx_http_upstream.h Mon Jul 01 22:07:12 2024 +0900
@@ -287,14 +287,17 @@ typedef struct {

     ngx_table_elt_t                 *cache_control;
     ngx_table_elt_t                 *set_cookie;
+    ngx_table_elt_t                 *age;

     off_t                            content_length_n;
     time_t                           last_modified_time;
+    time_t                           age_n;

     unsigned                         connection_close:1;
     unsigned                         chunked:1;
     unsigned                         no_cache:1;
     unsigned                         expired:1;
+    unsigned                         relative_freshness:1;
 } ngx_http_upstream_headers_in_t;


diff -r cef4919b830f -r 685f47b21026 src/http/v2/ngx_http_v2.h
--- a/src/http/v2/ngx_http_v2.h    Mon Jul 01 22:07:03 2024 +0900
+++ b/src/http/v2/ngx_http_v2.h    Mon Jul 01 22:07:12 2024 +0900
@@ -398,6 +398,7 @@ ngx_int_t ngx_http_v2_table_size(ngx_htt
 #define NGX_HTTP_V2_STATUS_404_INDEX      13
 #define NGX_HTTP_V2_STATUS_500_INDEX      14

+#define NGX_HTTP_V2_AGE_INDEX             21
 #define NGX_HTTP_V2_CONTENT_LENGTH_INDEX  28
 #define NGX_HTTP_V2_CONTENT_TYPE_INDEX    31
 #define NGX_HTTP_V2_DATE_INDEX            33
diff -r cef4919b830f -r 685f47b21026 src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c     Mon Jul 01 22:07:03 2024 +0900
+++ b/src/http/v2/ngx_http_v2_filter_module.c     Mon Jul 01 22:07:12 2024 +0900
@@ -258,6 +258,10 @@ ngx_http_v2_header_filter(ngx_http_reque
         len += 1 + ngx_http_v2_literal_size("Wed, 31 Dec 1986 18:00:00 GMT");
     }

+    if (r->headers_out.age_n != -1) {
+        len += 1 + ngx_http_v2_integer_octets(NGX_TIME_T_LEN) + NGX_TIME_T_LEN;
+    }
+
     if (r->headers_out.location && r->headers_out.location->value.len) {

         if (r->headers_out.location->value.data[0] == '/'
@@ -552,6 +556,18 @@ ngx_http_v2_header_filter(ngx_http_reque
         pos = ngx_http_v2_write_value(pos, pos, len, tmp);
     }

+    if (r->headers_out.age_n != -1) {
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 output header: \"age: %T\"",
+                       r->headers_out.age_n);
+
+        *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_AGE_INDEX);
+
+        p = pos;
+        pos = ngx_sprintf(pos + 1, "%T", r->headers_out.age_n);
+        *p = NGX_HTTP_V2_ENCODE_RAW | (u_char) (pos - p - 1);
+    }
+
     if (r->headers_out.location && r->headers_out.location->value.len) {
         ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
                        "http2 output header: \"location: %V\"",
diff -r cef4919b830f -r 685f47b21026 src/http/v3/ngx_http_v3_filter_module.c
--- a/src/http/v3/ngx_http_v3_filter_module.c     Mon Jul 01 22:07:03 2024 +0900
+++ b/src/http/v3/ngx_http_v3_filter_module.c     Mon Jul 01 22:07:12 2024 +0900
@@ -13,6 +13,7 @@
 /* static table indices */
 #define NGX_HTTP_V3_HEADER_AUTHORITY                 0
 #define NGX_HTTP_V3_HEADER_PATH_ROOT                 1
+#define NGX_HTTP_V3_HEADER_AGE_ZERO                  2
 #define NGX_HTTP_V3_HEADER_CONTENT_LENGTH_ZERO       4
 #define NGX_HTTP_V3_HEADER_DATE                      6
 #define NGX_HTTP_V3_HEADER_LAST_MODIFIED             10
@@ -213,6 +214,15 @@ ngx_http_v3_header_filter(ngx_http_reque
                                   sizeof("Mon, 28 Sep 1970 06:00:00 GMT") - 1);
     }

+    if (r->headers_out.age_n > 0) {
+        len += ngx_http_v3_encode_field_lri(NULL, 0,
+                                            NGX_HTTP_V3_HEADER_AGE_ZERO,
+                                            NULL, NGX_TIME_T_LEN);
+    } else if (r->headers_out.age_n == 0) {
+        len += ngx_http_v3_encode_field_ri(NULL, 0,
+                                           NGX_HTTP_V3_HEADER_AGE_ZERO);
+    }
+
     if (r->headers_out.location && r->headers_out.location->value.len) {

         if (r->headers_out.location->value.data[0] == '/'
@@ -452,6 +462,27 @@ ngx_http_v3_header_filter(ngx_http_reque
                                               p, n);
     }

+    if (r->headers_out.age_n != -1) {
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0,
+                       "http3 output header: \"age: %T\"",
+                       r->headers_out.age_n);
+
+        if (r->headers_out.age_n > 0) {
+            p = ngx_sprintf(b->last, "%T", r->headers_out.age_n);
+            n = p - b->last;
+
+            b->last = (u_char *) ngx_http_v3_encode_field_lri(b->last, 0,
+                                                   NGX_HTTP_V3_HEADER_AGE_ZERO,
+                                                   NULL, n);
+
+            b->last = ngx_sprintf(b->last, "%T", r->headers_out.age_n);
+
+        } else {
+            b->last = (u_char *) ngx_http_v3_encode_field_ri(b->last, 0,
+                                                  NGX_HTTP_V3_HEADER_AGE_ZERO);
+        }
+    }
+
     if (r->headers_out.location && r->headers_out.location->value.len) {
         ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0,
                        "http3 output header: \"location: %V\"",

From hnakamur at gmail.com  Mon Jul  1 13:47:39 2024
From: hnakamur at gmail.com (Hiroaki Nakamura)
Date: Mon, 1 Jul 2024 22:47:39 +0900
Subject: [PATCH 2 of 2] Tests: Update and add tests for Age header
In-Reply-To: <CAN-DUMRpSuOnYWe3O2_X7x-P_BMwQ4=h7RYm2W_rNqTP8KReaQ@mail.gmail.com>
References: <CAN-DUMQwyKd07fkbpa5osUnAH0sJvdwT=QgO6aGtCpqmn2iBNw@mail.gmail.com>
 <Zniy3EFG005hIsXl@mdounin.ru>
 <CAN-DUMQzHifqXq4Uc+gRQqNG4pKuNcz=Zwz_+K+Srx-QKWW34Q@mail.gmail.com>
 <Znywy92srHcefS69@mdounin.ru>
 <CAN-DUMRpSuOnYWe3O2_X7x-P_BMwQ4=h7RYm2W_rNqTP8KReaQ@mail.gmail.com>
Message-ID: <CAN-DUMQUacaLyNnyykj2VN1oLJWdj9-REW1LTzpfZ4tkNz7BLA@mail.gmail.com>

# HG changeset patch
# User Hiroaki Nakamura <hnakamur at gmail.com>
# Date 1719839239 -32400
#      Mon Jul 01 22:07:19 2024 +0900
# Node ID dcb24ed2502869338b8a1f591cbe586edfce53c1
# Parent  685f47b210263988886db6fe15a95e177616de1f
Tests: Update and add tests for Age header.

diff -r 685f47b21026 -r dcb24ed25028 h2_proxy_cache_age.t
--- /dev/null  Thu Jan 01 00:00:00 1970 +0000
+++ b/h2_proxy_cache_age.t    Mon Jul 01 22:07:19 2024 +0900
@@ -0,0 +1,228 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+# (C) Hiroaki Nakamura
+
+# Tests for age in HTTP/2 proxy cache.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::HTTP2;
+
+use POSIX qw/ ceil /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http http_v2 proxy cache/)->plan(10)
+    ->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    proxy_cache_path %%TESTDIR%%/cache    keys_zone=NAME:1m;
+    proxy_cache_path %%TESTDIR%%/cache2   keys_zone=NAME2:1m;
+
+    map $arg_slow $rate {
+        default 8k;
+        1       90;
+    }
+
+    server {
+        listen       127.0.0.1:8080 http2;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8080.log debug;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8081;
+            proxy_cache            NAME;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8081.log debug;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8082;
+            proxy_cache            NAME2;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8082;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8082.log debug;
+
+        location / {
+            add_header Cache-Control s-maxage=$arg_ttl;
+            limit_rate $rate;
+        }
+    }
+}
+
+EOF
+
+$t->write_file('t.html', 'SEE-THIS');
+$t->write_file('t2.html', 'SEE-THIS');
+$t->write_file('t3.html', 'SEE-THIS');
+
+# suppress deprecation warning
+
+open OLDERR, ">&", \*STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", \*OLDERR;
+
+###############################################################################
+
+my $s = Test::Nginx::HTTP2->new();
+
+my ($path, $sid, $frames, $frame, $t1, $age);
+
+# normal origin
+
+wait_until_next_second();
+
+$path = '/t.html?ttl=2';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'age first');
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age hit');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 2, 'age hit');
+
+select undef, undef, undef, 1.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'age updated');
+
+SKIP: {
+skip 'no exec on win32', 3 if $^O eq 'MSWin32';
+
+# slow origin
+
+wait_until_next_second();
+
+$path = '/t2.html?ttl=10&slow=1';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'slow origin first');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 2, 'slow origin hit');
+
+select undef, undef, undef, 9.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'slow origin updated');
+$t1 = time();
+
+$t->stop();
+
+select undef, undef, undef, 1.0;
+
+open OLDERR, ">&", \*STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", \*OLDERR;
+
+$age = time() - $t1;
+
+$s = Test::Nginx::HTTP2->new();
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, $age, 'age after restart');
+
+}
+
+# update age after restart
+
+wait_until_next_second();
+
+$path = '/t3.html?ttl=20';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'age before restart');
+$t1 = time();
+
+$t->stop();
+
+select undef, undef, undef, 1.0;
+
+open OLDERR, ">&", \*STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", \*OLDERR;
+
+$age = time() - $t1;
+
+$s = Test::Nginx::HTTP2->new();
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, $age, 'age after restart');
+
+$t->stop();
+
+###############################################################################
+
+# Wait until the next second boundary.
+# Calling this before sending a request increases the likelihood that the
+# timestamp value does not cross into the next second while sending a request
+# and receiving a response.
+sub wait_until_next_second {
+    my $now = time();
+    my $next_second = ceil($now);
+    my $sleep = $next_second - $now;
+    select undef, undef, undef, $sleep;
+}
+
+###############################################################################
diff -r 685f47b21026 -r dcb24ed25028 h2_ssl_proxy_cache_age.t
--- /dev/null  Thu Jan 01 00:00:00 1970 +0000
+++ b/h2_ssl_proxy_cache_age.t     Mon Jul 01 22:07:19 2024 +0900
@@ -0,0 +1,274 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+# (C) Hiroaki Nakamura
+
+# Tests for age in HTTP/2 ssl proxy cache.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::HTTP2;
+
+use POSIX qw/ ceil /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()
+    ->has(qw/http http_ssl http_v2 proxy cache socket_ssl/)->plan(13)
+    ->has_daemon('openssl');
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    proxy_cache_path %%TESTDIR%%/cache   keys_zone=NAME:1m;
+    proxy_cache_path %%TESTDIR%%/cache2  keys_zone=NAME2:1m;
+
+    map $arg_slow $rate {
+        default 8k;
+        1       90;
+    }
+
+    server {
+        listen       127.0.0.1:8080 http2 ssl;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8080.log debug;
+
+        ssl_certificate_key localhost.key;
+        ssl_certificate localhost.crt;
+
+        location / {
+            proxy_pass   http://127.0.0.1:8081;
+            proxy_cache  NAME;
+            proxy_http_version 1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8081.log debug;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8082;
+            proxy_cache            NAME2;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8082;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8082.log debug;
+
+        location / {
+            add_header Cache-Control s-maxage=$arg_ttl;
+            limit_rate $rate;
+        }
+    }
+}
+
+EOF
+
+$t->write_file('openssl.conf', <<EOF);
+[ req ]
+default_bits = 2048
+encrypt_key = no
+distinguished_name = req_distinguished_name
+[ req_distinguished_name ]
+EOF
+
+my $d = $t->testdir();
+
+foreach my $name ('localhost') {
+    system('openssl req -x509 -new '
+         . "-config $d/openssl.conf -subj /CN=$name/ "
+         . "-out $d/$name.crt -keyout $d/$name.key "
+         . ">>$d/openssl.out 2>&1") == 0
+         or die "Can't create certificate for $name: $!\n";
+}
+
+$t->write_file('t.html', 'SEE-THIS');
+$t->write_file('t2.html', 'SEE-THIS');
+$t->write_file('t3.html', 'SEE-THIS');
+
+open OLDERR, ">&", \*STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", \*OLDERR;
+
+###############################################################################
+
+my $s = getconn(port(8080));
+ok($s, 'ssl connection');
+
+my ($path, $sid, $frames, $frame, $t1, $age);
+
+# normal origin
+
+wait_until_next_second();
+
+$path = '/t.html?ttl=2';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'age first');
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age hit');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 2, 'age hit');
+
+select undef, undef, undef, 1.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'age updated');
+
+# slow origin
+
+SKIP: {
+skip 'no exec on win32', 3 if $^O eq 'MSWin32';
+
+wait_until_next_second();
+
+$path = '/t2.html?ttl=10&slow=1';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'slow origin first');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 2, 'slow origin hit');
+
+select undef, undef, undef, 9.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'slow origin updated');
+$t1 = time();
+
+$t->stop();
+
+select undef, undef, undef, 1.0;
+
+open OLDERR, ">&", \*STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", \*OLDERR;
+
+$age = time() - $t1;
+
+$s = getconn(port(8080));
+ok($s, 'ssl connection');
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, $age, 'age after restart');
+
+}
+
+# update age after restart
+
+$path = '/t3.html?ttl=20';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'age before restart');
+$t1 = time();
+
+$t->stop();
+
+select undef, undef, undef, 1.0;
+
+open OLDERR, ">&", \*STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", \*OLDERR;
+
+$age = time() - $t1;
+
+$s = getconn(port(8080));
+ok($s, 'ssl connection');
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, $age, 'age after restart');
+
+$t->stop();
+
+###############################################################################
+
+sub getconn {
+    my ($port) = @_;
+    my $s;
+
+    eval {
+         my $sock = Test::Nginx::HTTP2::new_socket($port, SSL => 1,
+              alpn => 'h2');
+         $s = Test::Nginx::HTTP2->new($port, socket => $sock)
+              if $sock->alpn_selected();
+    };
+
+    return $s if defined $s;
+
+    eval {
+         my $sock = Test::Nginx::HTTP2::new_socket($port, SSL => 1,
+              npn => 'h2');
+         $s = Test::Nginx::HTTP2->new($port, socket => $sock)
+              if $sock->next_proto_negotiated();
+    };
+
+    return $s;
+}
+
+# Wait until the next second boundary.
+# Calling this before sending a request increases the likelihood that the
+# timestamp value does not cross into the next second while sending a request
+# and receiving a response.
+sub wait_until_next_second {
+    my $now = time();
+    my $next_second = ceil($now);
+    my $sleep = $next_second - $now;
+    select undef, undef, undef, $sleep;
+}
+
+###############################################################################
diff -r 685f47b21026 -r dcb24ed25028 h3_proxy_cache_age.t
--- /dev/null  Thu Jan 01 00:00:00 1970 +0000
+++ b/h3_proxy_cache_age.t    Mon Jul 01 22:07:19 2024 +0900
@@ -0,0 +1,222 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+# (C) Hiroaki Nakamura
+
+# Tests for age in HTTP/3 proxy cache.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::HTTP3;
+
+use POSIX qw/ ceil /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http http_v3 proxy cryptx/)
+    ->has_daemon('openssl')->plan(9)
+    ->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    ssl_certificate_key localhost.key;
+    ssl_certificate localhost.crt;
+
+    log_format test $uri:$status:$request_completion;
+
+    proxy_cache_path %%TESTDIR%%/cache    keys_zone=NAME:1m;
+
+    map $arg_slow $rate {
+        default 8k;
+        1       90;
+    }
+
+    server {
+        listen       127.0.0.1:%%PORT_8980_UDP%% quic;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8080.log debug;
+
+        location / {
+            proxy_pass http://127.0.0.1:8081/;
+            proxy_cache NAME;
+            proxy_http_version 1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8081.log debug;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8082;
+            proxy_cache            NAME;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8082;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8082.log debug;
+
+        location / {
+            add_header Cache-Control s-maxage=$arg_ttl;
+            limit_rate $rate;
+        }
+    }
+}
+
+EOF
+
+$t->write_file('openssl.conf', <<EOF);
+[ req ]
+default_bits = 2048
+encrypt_key = no
+distinguished_name = req_distinguished_name
+[ req_distinguished_name ]
+EOF
+
+my $d = $t->testdir();
+
+foreach my $name ('localhost') {
+    system('openssl req -x509 -new '
+         . "-config $d/openssl.conf -subj /CN=$name/ "
+         . "-out $d/$name.crt -keyout $d/$name.key "
+         . ">>$d/openssl.out 2>&1") == 0
+         or die "Can't create certificate for $name: $!\n";
+}
+
+my $content = 'SEE-THIS';
+$t->write_file('t.html', $content);
+$t->write_file('t2.html', $content);
+$t->write_file('t3.html', $content);
+$t->run();
+
+###############################################################################
+
+my $s = Test::Nginx::HTTP3->new();
+
+my ($path, $sid, $frames, $frame, $t1, $age);
+
+# normal origin
+
+wait_until_next_second();
+
+$path = '/t.html?ttl=2';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'age first');
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age hit');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 2, 'age hit');
+
+select undef, undef, undef, 1.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'age updated');
+
+# slow origin
+
+wait_until_next_second();
+
+$path = '/t2.html?ttl=4&slow=1';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'slow origin first');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 2, 'slow origin hit');
+
+select undef, undef, undef, 3.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'slow origin updated');
+
+# update age after restart
+
+$path = '/t3.html?ttl=20';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, undef, 'age before restart');
+$t1 = time();
+
+$t->stop();
+
+select undef, undef, undef, 1.0;
+
+open OLDERR, ">&", \*STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", \*OLDERR;
+
+$age = time() - $t1;
+
+$s = Test::Nginx::HTTP3->new();
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, $age, 'age after restart');
+
+$t->stop();
+
+###############################################################################
+
+# Wait until the next second boundary.
+# Calling this before sending a request increases the likelihood that the
+# timestamp value does not cross into the next second while sending a request
+# and receiving a response.
+sub wait_until_next_second {
+    my $now = time();
+    my $next_second = ceil($now);
+    my $sleep = $next_second - $now;
+    select undef, undef, undef, $sleep;
+}
+
+###############################################################################
diff -r 685f47b21026 -r dcb24ed25028 proxy_cache_age.t
--- /dev/null  Thu Jan 01 00:00:00 1970 +0000
+++ b/proxy_cache_age.t  Mon Jul 01 22:07:19 2024 +0900
@@ -0,0 +1,195 @@
+#!/usr/bin/perl
+
+# (C) Maxim Dounin
+# (C) Hiroaki Nakamura
+
+# Tests for age in http proxy cache.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+use Socket qw/ CRLF /;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+
+use POSIX qw/ ceil /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http proxy cache/)->plan(10)
+    ->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    proxy_cache_path   %%TESTDIR%%/cache  levels=1:2
+                       keys_zone=NAME:1m;
+    proxy_cache_path   %%TESTDIR%%/cache2  levels=1:2
+                       keys_zone=NAME2:1m;
+
+    map $arg_slow $rate {
+        default 8k;
+        1       100;
+    }
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8080.log debug;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8081;
+            proxy_cache            NAME;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+            add_header parent_date $upstream_http_date;
+            add_header child_msec $msec;
+            add_header child_cache $upstream_cache_status;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8081.log debug;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8082;
+            proxy_cache            NAME2;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+            add_header origin_date $upstream_http_date;
+            add_header parent_msec $msec;
+            add_header parent_cache $upstream_cache_status;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8082;
+        server_name  localhost;
+        error_log    %%TESTDIR%%/error8082.log debug;
+
+        location / {
+            add_header Cache-Control $http_x_cache_control;
+            limit_rate $rate;
+            add_header origin_msec $msec;
+        }
+    }
+}
+
+EOF
+
+$t->write_file('t.html', 'SEE-THIS');
+$t->write_file('t2.html', 'SEE-THIS');
+$t->write_file('t3.html', 'SEE-THIS');
+
+$t->run();
+
+###############################################################################
+
+# normal origin
+
+wait_until_next_second();
+
+unlike(get('/t.html', 's-maxage=2'), qr/\r\nAge: /, 'age first');
+like(get('/t.html', 's-maxage=2'), qr/\r\nAge: 0\r\n/, 'age hit');
+
+sleep 2;
+
+like(get('/t.html', 's-maxage=2'), qr/\r\nAge: 2\r\n/, 'age hit');
+
+sleep 1;
+
+unlike(http_get('/t.html'), qr/\r\nAge: /, 'age updated');
+
+# slow origin
+
+SKIP: {
+skip 'no exec on win32', 3 if $^O eq 'MSWin32';
+
+wait_until_next_second();
+
+unlike(get('/t2.html?slow=1', 's-maxage=4'), qr/\r\nAge: /,
+    'slow origin first');
+
+sleep 2;
+
+like(http_get('/t2.html?slow=1'), qr/\r\nAge: 2\r\n/, 'slow origin hit');
+
+sleep 3;
+
+unlike(http_get('/t2.html?slow=1'), qr/\r\nAge: /, 'slow origin updated');
+my $t1 = time();
+
+$t->stop();
+
+sleep 1;
+
+$t->run();
+
+my $age = time() - $t1;
+like(http_get('/t2.html?slow=1'), qr/\r\nAge: $age\r\n/,
+    'age after restart');
+
+}
+
+# update age after restart
+
+wait_until_next_second();
+
+unlike(get('/t3.html', 's-maxage=20'), qr/\r\nAge: /, 'age before restart');
+my $t1 = time();
+
+$t->stop();
+
+sleep 1;
+
+$t->run();
+
+my $age = time() - $t1;
+like(http_get('/t3.html'), qr/\r\nAge: $age\r\n/,
+    'age after restart');
+
+$t->stop();
+
+###############################################################################
+
+sub get {
+    my ($url, $extra, %extra) = @_;
+    return http(<<EOF, %extra);
+GET $url HTTP/1.1
+Host: localhost
+Connection: close
+X-Cache-Control: $extra
+
+EOF
+}
+
+# Wait until the next second boundary.
+# Calling this before sending a request increases the likelihood that the
+# timestamp value does not cross into the next second while sending a request
+# and receiving a response.
+sub wait_until_next_second {
+    my $now = time();
+    my $next_second = ceil($now);
+    my $sleep = $next_second - $now;
+    select undef, undef, undef, $sleep;
+}
+
+###############################################################################
diff -r 685f47b21026 -r dcb24ed25028 proxy_cache_use_stale.t
--- a/proxy_cache_use_stale.t Mon Jul 01 22:07:12 2024 +0900
+++ b/proxy_cache_use_stale.t Mon Jul 01 22:07:19 2024 +0900
@@ -206,7 +206,8 @@ like(http_get('/regexp.html'), qr/HIT/,

 $t->write_file('t6.html', 'SEE-THAT');

-my $s = get('/t6.html', 'max-age=1, stale-while-revalidate=2', start => 1);
+# max-age must be 5 here since response delay is 4 seconds.
+my $s = get('/t6.html', 'max-age=5, stale-while-revalidate=2', start => 1);
 select undef, undef, undef, 0.2;
 like(http_get('/t6.html'), qr/UPDATING.*SEE-THIS/s, 's-w-r - updating');
 like(http_end($s), qr/STALE.*SEE-THIS/s, 's-w-r - updating stale');

From mdounin at mdounin.ru  Thu Jul  4 14:49:46 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Thu, 04 Jul 2024 17:49:46 +0300
Subject: [nginx] SSI: fixed incorrect or duplicate stub output.
Message-ID: <hg.5be23505292b.1720104586.7221895432598085720@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx/rev/5be23505292b
branches:  
changeset: 9300:5be23505292b
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Jul 04 17:41:28 2024 +0300
description:
SSI: fixed incorrect or duplicate stub output.

Following 3518:eb3aaf8bd2a9 (0.8.37), r->request_output is only set
if there are data in the first buffer sent in the subrequest.  As a
result, following the change mentioned this flag cannot be used to
prevent duplicate ngx_http_ssi_stub_output() calls, since it is not
set if there was already some output, but the first buffer was empty.

Still, when there are multiple subrequests, even an empty subrequest
response might be delayed by the postpone filter, leading to a second
call of ngx_http_ssi_stub_output() during finalization from
ngx_http_writer() the subreqest buffers are released by the postpone
filter.  Since r->request_output is not set after the first call, this
resulted in duplicate stub output.

Additionally, checking only the first buffer might be wrong in some
unusual cases.  For example, the first buffer might be empty if
$r->flush() is called before printing any data in the embedded Perl
module.

Depending on the postpone_output value and corresponding sizes, this
issue can result in either duplicate or unexpected stub output, or
"zero size buf in writer" alerts.

Following 8124:f5515e727656 (1.23.4), it became slightly easier to
reproduce the issue, as empty static files and empty cache items now
result in a response with an empty buffer.  Before the change, an empty
proxied response can be used to reproduce the issue.

Fix is check all buffers and set r->request_output if any non-empty
buffers are sent.  This ensures that all unusual cases of non-empty
responses are covered, and also that r->request_output will be set
after the first stub output, preventing duplicate output.

Reported by Jan Gassen.

diffstat:

 src/http/ngx_http_copy_filter_module.c |  10 ++++++++--
 1 files changed, 8 insertions(+), 2 deletions(-)

diffs (28 lines):

diff --git a/src/http/ngx_http_copy_filter_module.c b/src/http/ngx_http_copy_filter_module.c
--- a/src/http/ngx_http_copy_filter_module.c
+++ b/src/http/ngx_http_copy_filter_module.c
@@ -83,6 +83,7 @@ static ngx_int_t
 ngx_http_copy_filter(ngx_http_request_t *r, ngx_chain_t *in)
 {
     ngx_int_t                     rc;
+    ngx_chain_t                  *cl;
     ngx_connection_t             *c;
     ngx_output_chain_ctx_t       *ctx;
     ngx_http_core_loc_conf_t     *clcf;
@@ -132,9 +133,14 @@ ngx_http_copy_filter(ngx_http_request_t 
             ctx->thread_handler = ngx_http_copy_thread_handler;
         }
 #endif
+    }
 
-        if (in && in->buf && ngx_buf_size(in->buf)) {
-            r->request_output = 1;
+    if (!r->request_output) {
+        for (cl = in; cl; cl = cl->next) {
+            if (ngx_buf_size(cl->buf)) {
+                r->request_output = 1;
+                break;
+            }
         }
     }
 

From mdounin at mdounin.ru  Thu Jul  4 14:50:03 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Thu, 04 Jul 2024 17:50:03 +0300
Subject: [nginx-tests] Tests: SSI include stub tests.
Message-ID: <hg.85d88cd5091c.1720104603.6442806439585662546@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx-tests/rev/85d88cd5091c
branches:  
changeset: 1990:85d88cd5091c
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Jul 04 17:42:55 2024 +0300
description:
Tests: SSI include stub tests.

diffstat:

 ssi_stub.t |  108 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 108 insertions(+), 0 deletions(-)

diffs (113 lines):

diff --git a/ssi_stub.t b/ssi_stub.t
new file mode 100644
--- /dev/null
+++ b/ssi_stub.t
@@ -0,0 +1,108 @@
+#!/usr/bin/perl
+
+# (C) Maxim Dounin
+
+# Tests for nginx ssi module, stub output.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http proxy ssi/)->plan(4)
+	->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+
+        location / {
+            ssi on;
+        }
+
+        location = /empty {
+            # static
+        }
+
+        location = /error404 {
+            # static 404
+        }
+
+        location = /proxy404 {
+            proxy_pass http://127.0.0.1:8081;
+            proxy_intercept_errors on;
+            error_page 404 /error404;
+        }
+
+        location = /not_empty {
+            proxy_pass http://127.0.0.1:8081;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+    }
+}
+
+EOF
+
+$t->write_file('empty.html',
+	'<!--#block name="fallback" -->fallback<!--#endblock -->' .
+	':<!--#include virtual="/empty" stub="fallback" -->:');
+
+$t->write_file('error.html',
+	'<!--#block name="fallback" -->fallback<!--#endblock -->' .
+	':<!--#include virtual="/error404" stub="fallback" -->:');
+
+$t->write_file('proxy_error.html',
+	'<!--#block name="fallback" -->fallback<!--#endblock -->' .
+	':<!--#include virtual="/proxy404" stub="fallback" -->:');
+
+$t->write_file('postponed.html',
+	'<!--#block name="fallback" -->fallback<!--#endblock -->' .
+	':<!--#include virtual="/not_empty" -->' .
+	':<!--#include virtual="/empty" stub="fallback" -->:');
+
+$t->write_file('empty', '');
+$t->write_file('not_empty', 'not empty');
+
+$t->run();
+
+###############################################################################
+
+like(http_get('/empty.html'), qr/:fallback:/, 'ssi stub empty');
+like(http_get('/error.html'), qr/:fallback:/, 'ssi stub error');
+like(http_get('/proxy_error.html'), qr/:fallback:/, 'ssi stub proxied error');
+
+TODO: {
+local $TODO = 'not yet' unless $t->has_version('1.27.2');
+
+like(http_get('/postponed.html'), qr/:not empty:fallback:/s,
+	'ssi stub postponed');
+
+}
+
+###############################################################################

From mdounin at mdounin.ru  Tue Jul  9 00:34:04 2024
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 9 Jul 2024 03:34:04 +0300
Subject: freenginx-1.27.2 changes draft
Message-ID: <ZoyELEvFoMutgM9K@mdounin.ru>

Hello!

Below are changes draft for freenginx 1.27.2.
Comments are welcome.


Changes with freenginx 1.27.2                                    09 Jul 2024

    *) Feature: now maximum error logging rate can be limited with the
       "rate" parameter of the "error_log" directive; by default the limit
       is 1000 messages per second.

    *) Bugfix: in resolver.
       Thanks to Kirill A. Korinsky.

    *) Bugfix: the "proxy_no_cache" directive was not checked when caching
       errors.

    *) Bugfix: a block used in the "stub" parameter of the "include" SSI
       directive might be sent twice.


????????? ? freenginx 1.27.2                                      09.07.2024

    *) ??????????: ?????? ???????????? ??????? ????????????? ?????? ?????
       ???? ?????????? ? ??????? ????????? rate ????????? error_log; ??
       ????????? ???????????? ??????????? ? 1000 ????????? ? ???????.

    *) ???????????: ? resolver'?.
       ??????? ??????? ??????????.

    *) ???????????: ????????? proxy_no_cache ?? ??????????? ??? ???????????
       ??????.

    *) ???????????: ????, ???????????? ? ????????? stub ? SSI-?????????
       include, ??? ???? ????????? ??????.


-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Tue Jul  9 14:54:50 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Tue, 09 Jul 2024 17:54:50 +0300
Subject: [nginx] Updated OpenSSL used for win32 builds.
Message-ID: <hg.88105160c077.1720536890.8423735094122401078@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx/rev/88105160c077
branches:  
changeset: 9301:88105160c077
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Jul 09 03:52:37 2024 +0300
description:
Updated OpenSSL used for win32 builds.

diffstat:

 misc/GNUmakefile |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff --git a/misc/GNUmakefile b/misc/GNUmakefile
--- a/misc/GNUmakefile
+++ b/misc/GNUmakefile
@@ -6,7 +6,7 @@ TEMP =		tmp
 
 CC =		cl
 OBJS =		objs.msvc8
-OPENSSL =	openssl-3.0.13
+OPENSSL =	openssl-3.0.14
 ZLIB =		zlib-1.3.1
 PCRE =		pcre2-10.39
 

From mdounin at mdounin.ru  Tue Jul  9 14:54:50 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Tue, 09 Jul 2024 17:54:50 +0300
Subject: [nginx] freenginx-1.27.2-RELEASE
Message-ID: <hg.91d77cc29d33.1720536890.8423735094122401078@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx/rev/91d77cc29d33
branches:  
changeset: 9302:91d77cc29d33
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Jul 09 17:44:13 2024 +0300
description:
freenginx-1.27.2-RELEASE

diffstat:

 docs/xml/nginx/changes.xml |  49 ++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 49 insertions(+), 0 deletions(-)

diffs (59 lines):

diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml
+++ b/docs/xml/nginx/changes.xml
@@ -7,6 +7,55 @@
 <section title="freenginx">
 
 
+<changes ver="1.27.2" date="2024-07-09">
+
+<change type="feature">
+<para lang="ru">
+?????? ???????????? ??????? ????????????? ?????? ????? ???? ??????????
+? ??????? ????????? rate ????????? error_log;
+?? ????????? ???????????? ??????????? ? 1000 ????????? ? ???????.
+</para>
+<para lang="en">
+now maximum error logging rate can be limited
+with the "rate" parameter of the "error_log" directive;
+by default the limit is 1000 messages per second.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? resolver'?.<br/>
+??????? ??????? ??????????.
+</para>
+<para lang="en">
+in resolver.<br/>
+Thanks to Kirill A. Korinsky.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+????????? proxy_no_cache ?? ??????????? ??? ??????????? ??????.
+</para>
+<para lang="en">
+the "proxy_no_cache" directive was not checked when caching errors.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+????, ???????????? ? ????????? stub ? SSI-????????? include,
+??? ???? ????????? ??????.
+</para>
+<para lang="en">
+a block used in the "stub" parameter of the "include" SSI directive
+might be sent twice.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.27.1" date="2024-06-04">
 
 <change type="feature">

From mdounin at mdounin.ru  Tue Jul  9 14:54:51 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Tue, 09 Jul 2024 17:54:51 +0300
Subject: [nginx] release-1.27.2 tag
Message-ID: <hg.2a847df38232.1720536891.8423735094122401078@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx/rev/2a847df38232
branches:  
changeset: 9303:2a847df38232
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Jul 09 17:44:14 2024 +0300
description:
release-1.27.2 tag

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -480,3 +480,4 @@ ab948bfa042d7a7b20c3e730d7e9675cc172324f
 2956b59565c91baa79d13d6411f2404614c0134e release-1.25.5
 8c4e2b7de093d357b2f462399d6d395b899ffe76 release-1.27.0
 ee3eb2b9705f0c913a1bf4b9fe74def31411e8bf release-1.27.1
+91d77cc29d3336b1df25fe42d6c5ad4ca24b72b9 release-1.27.2

From mdounin at mdounin.ru  Tue Jul  9 14:55:13 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Tue, 09 Jul 2024 17:55:13 +0300
Subject: [nginx-site] freenginx-1.27.2
Message-ID: <hg.6cab598f14a1.1720536913.2846049246072879395@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx-site/rev/6cab598f14a1
branches:  
changeset: 3091:6cab598f14a1
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Jul 09 17:51:47 2024 +0300
description:
freenginx-1.27.2

diffstat:

 text/en/CHANGES    |  16 ++++++++++++++++
 text/ru/CHANGES.ru |  16 ++++++++++++++++
 xml/index.xml      |  10 ++++++++++
 xml/versions.xml   |   1 +
 4 files changed, 43 insertions(+), 0 deletions(-)

diffs (79 lines):

diff --git a/text/en/CHANGES b/text/en/CHANGES
--- a/text/en/CHANGES
+++ b/text/en/CHANGES
@@ -1,4 +1,20 @@
 
+Changes with freenginx 1.27.2                                    09 Jul 2024
+
+    *) Feature: now maximum error logging rate can be limited with the
+       "rate" parameter of the "error_log" directive; by default the limit
+       is 1000 messages per second.
+
+    *) Bugfix: in resolver.
+       Thanks to Kirill A. Korinsky.
+
+    *) Bugfix: the "proxy_no_cache" directive was not checked when caching
+       errors.
+
+    *) Bugfix: a block used in the "stub" parameter of the "include" SSI
+       directive might be sent twice.
+
+
 Changes with freenginx 1.27.1                                    04 Jun 2024
 
     *) Feature: the "max_headers" directive.
diff --git a/text/ru/CHANGES.ru b/text/ru/CHANGES.ru
--- a/text/ru/CHANGES.ru
+++ b/text/ru/CHANGES.ru
@@ -1,4 +1,20 @@
 
+????????? ? freenginx 1.27.2                                      09.07.2024
+
+    *) ??????????: ?????? ???????????? ??????? ????????????? ?????? ?????
+       ???? ?????????? ? ??????? ????????? rate ????????? error_log; ??
+       ????????? ???????????? ??????????? ? 1000 ????????? ? ???????.
+
+    *) ???????????: ? resolver'?.
+       ??????? ??????? ??????????.
+
+    *) ???????????: ????????? proxy_no_cache ?? ??????????? ??? ???????????
+       ??????.
+
+    *) ???????????: ????, ???????????? ? ????????? stub ? SSI-?????????
+       include, ??? ???? ????????? ??????.
+
+
 ????????? ? freenginx 1.27.1                                      04.06.2024
 
     *) ??????????: ????????? max_headers.
diff --git a/xml/index.xml b/xml/index.xml
--- a/xml/index.xml
+++ b/xml/index.xml
@@ -8,6 +8,16 @@
 
 <news name="freenginx news" link="/" lang="en">
 
+<event date="2024-07-09">
+<para>
+<link doc="en/download.xml">freenginx-1.27.2</link>
+mainline version has been released,
+featuring
+<link doc="en/docs/ngx_core_module.xml" id="error_log">rate limiting</link>
+for error logs.
+</para>
+</event>
+
 <event date="2024-06-04">
 <para>
 <link doc="en/download.xml">freenginx-1.27.1</link>
diff --git a/xml/versions.xml b/xml/versions.xml
--- a/xml/versions.xml
+++ b/xml/versions.xml
@@ -9,6 +9,7 @@
 
 <download tag="mainline" changes="">
 
+<item ver="1.27.2" name="freenginx" />
 <item ver="1.27.1" name="freenginx" />
 <item ver="1.27.0" name="freenginx" />
 

From mdounin at mdounin.ru  Wed Jul 10 15:25:17 2024
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Wed, 10 Jul 2024 18:25:17 +0300
Subject: [PATCH] Keepalive connections now respect lingering_timeout on
 shutdown
Message-ID: <3e36eed5bdaf1eec3d06.1720625117@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1720570370 -10800
#      Wed Jul 10 03:12:50 2024 +0300
# Node ID 3e36eed5bdaf1eec3d069705f197d04ef942b9d4
# Parent  2a847df382320a54c32abd9e64412ac611fcff0c
Keepalive connections now respect lingering_timeout on shutdown.

During graceful shutdown keepalive connections are now closed only after
at least lingering_timeout of inactivity.  To do so, c->idle is only
set on keepalive connections after lingering_timeout expires.

This ensures that the connection close race will less likely result in
connections being reset when a client sends a request at the same time
when the connection is closed by ngx_close_idle_connections() during
graceful shutdown.

diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c
+++ b/src/http/ngx_http_request.c
@@ -3099,6 +3099,7 @@ ngx_http_set_keepalive(ngx_http_request_
 {
     int                        tcp_nodelay;
     ngx_buf_t                 *b, *f;
+    ngx_msec_t                 timer;
     ngx_chain_t               *cl, *ln;
     ngx_event_t               *rev, *wev;
     ngx_connection_t          *c;
@@ -3299,10 +3300,19 @@ ngx_http_set_keepalive(ngx_http_request_
     r->http_state = NGX_HTTP_KEEPALIVE_STATE;
 #endif
 
-    c->idle = 1;
     ngx_reusable_connection(c, 1);
 
-    ngx_add_timer(rev, clcf->keepalive_timeout);
+    if (clcf->lingering_close
+        && clcf->lingering_timeout > 0)
+    {
+        timer = ngx_min(clcf->keepalive_timeout, clcf->lingering_timeout);
+        hc->keepalive_timeout = clcf->keepalive_timeout - timer;
+        ngx_add_timer(rev, timer);
+
+    } else {
+        c->idle = 1;
+        ngx_add_timer(rev, clcf->keepalive_timeout);
+    }
 
     if (rev->ready) {
         ngx_post_event(rev, &ngx_posted_events);
@@ -3313,16 +3323,32 @@ ngx_http_set_keepalive(ngx_http_request_
 static void
 ngx_http_keepalive_handler(ngx_event_t *rev)
 {
-    size_t             size;
-    ssize_t            n;
-    ngx_buf_t         *b;
-    ngx_connection_t  *c;
+    size_t                  size;
+    ssize_t                 n;
+    ngx_buf_t              *b;
+    ngx_connection_t       *c;
+    ngx_http_connection_t  *hc;
 
     c = rev->data;
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http keepalive handler");
 
-    if (rev->timedout || c->close) {
+    if (rev->timedout) {
+        if (c->idle || ngx_exiting) {
+            ngx_http_close_connection(c);
+            return;
+        }
+
+        hc = c->data;
+
+        c->idle = 1;
+        rev->timedout = 0;
+        ngx_add_timer(rev, hc->keepalive_timeout);
+
+        return;
+    }
+
+    if (c->close) {
         ngx_http_close_connection(c);
         return;
     }
diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -331,6 +331,8 @@ typedef struct {
 
     ngx_chain_t                      *free;
 
+    ngx_msec_t                        keepalive_timeout;
+
     unsigned                          ssl:1;
     unsigned                          proxy_protocol:1;
 } ngx_http_connection_t;


From mdounin at mdounin.ru  Thu Jul 18 00:26:16 2024
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 18 Jul 2024 03:26:16 +0300
Subject: [PATCH 1 of 3] Correctly calculate and set Age header
In-Reply-To: <CAN-DUMRpSuOnYWe3O2_X7x-P_BMwQ4=h7RYm2W_rNqTP8KReaQ@mail.gmail.com>
References: <CAN-DUMQwyKd07fkbpa5osUnAH0sJvdwT=QgO6aGtCpqmn2iBNw@mail.gmail.com>
 <Zniy3EFG005hIsXl@mdounin.ru>
 <CAN-DUMQzHifqXq4Uc+gRQqNG4pKuNcz=Zwz_+K+Srx-QKWW34Q@mail.gmail.com>
 <Znywy92srHcefS69@mdounin.ru>
 <CAN-DUMRpSuOnYWe3O2_X7x-P_BMwQ4=h7RYm2W_rNqTP8KReaQ@mail.gmail.com>
Message-ID: <ZphhKKqxCoOKM5jT@mdounin.ru>

Hello!

On Mon, Jul 01, 2024 at 10:45:59PM +0900, Hiroaki Nakamura wrote:

> Hello
> 
> 2024?6?27?(?) 9:32 Maxim Dounin <mdounin at mdounin.ru>:
> > [...]
> > The commit message certainly needs more work.  Still, you may want
> > to postpone it, as well as code modifications, before various
> > conceptual questions are cleaned up.
> 
> I have updated the commit log again.
> 
> > > > > @@ -75,6 +77,8 @@
> > > > >      time_t                           error_sec;
> > > > >      time_t                           last_modified;
> > > > >      time_t                           date;
> > > > > +    time_t                           response_time;
> > [...]
> > The "date" field is set when the cache item is created, and
> > can be updated on cache item revalidation (with
> > "proxy_cache_revalidate on").  It is not updated when sending a
> > cached response.
> 
> Thank you, I had misunderstood this.
> I take this into consideration and I found that we can keep
> ngx_http_file_cache_header_t as is if we ignore
> corrected_initial_age concept and just use the Age header from the
> upstream as is (see below).

[...]

> > Note well that the "would be great to dig further" comment still
> > applies.

Thanks for yet another version of the patch.  In its current form 
it starts to look reasonable.

I've spent some time looking into this, including looking through 
previous attempts to implement some forms of Age support, notably:

Florent Le Coz in 2013, cache validity based on Age
https://mailman.nginx.org/pipermail/nginx-devel/2013-November/004532.html

Piotr Sikora in 2014, $upstream_cache_age variable
https://mailman.nginx.org/pipermail/nginx-devel/2014-October/006025.html

Damien Tournoud in 2015, $upstream_cache_age variable
https://mailman.nginx.org/pipermail/nginx-devel/2015-June/007002.html

Overall, there seems to be two major parts of what is expected to 
be done here:

1. Adjusting cache validity times based on the Age header in the 
upstream server response.

2. Returning cached responses with updated/added Age header.

These two parts probably can be implemented separately, with 
minimal interdependencies.

Further, I tend to think that, while (1) probably can be the 
default, it might not be a good idea to do (2) by default, as in 
most setups [free]nginx is used as an origin server, even if 
caching is enabled.  In such configurations administrators often 
use "Cache-Control: max-age=N" for client-side caching, and 
control caching in [free]nginx separately, such as with 
proxy_cache_valid.  Adding the Age header in such configurations 
might result in unexpectedly non-working client-side caching.

In particular, (2) can be implemented as a variable with relevant 
information to be used in the "add_header" directive if one needs 
to implement Age updating.  Alternative approach would be to add 
per-module directives to control (2), such as "proxy_cache_age on|off", 
and add the Age header in the upstream module, though a variable 
looks easier.

Below are two patches that implement (1) adjusting cache validity 
base on the Age header and (2) a variable to set the Age header, 
and yet another patch to implement tests.  Please take a look.


# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1721259797 -10800
#      Thu Jul 18 02:43:17 2024 +0300
# Node ID 0843408f7af487d59800edcd773eddab33eade05
# Parent  2a847df382320a54c32abd9e64412ac611fcff0c
Upstream: using the "Age" header when caching responses.

As long as the "Age" header is present and not ignored, it is now
respected when caching responses per "Cache-Control: max-age", reducing
cache validity time.

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -136,6 +136,8 @@ static ngx_int_t
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_upstream_process_vary(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
+static ngx_int_t ngx_http_upstream_process_age(ngx_http_request_t *r,
+    ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_upstream_copy_header_line(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t
@@ -293,6 +295,10 @@ static ngx_http_upstream_header_t  ngx_h
                  ngx_http_upstream_copy_multi_header_lines,
                  offsetof(ngx_http_headers_out_t, link), 0 },
 
+    { ngx_string("Age"),
+                 ngx_http_upstream_process_age, 0,
+                 ngx_http_upstream_copy_header_line, 0, 0 },
+
     { ngx_string("X-Accel-Expires"),
                  ngx_http_upstream_process_accel_expires, 0,
                  ngx_http_upstream_copy_header_line, 0, 0 },
@@ -475,6 +481,7 @@ ngx_conf_bitmask_t  ngx_http_upstream_ig
     { ngx_string("Cache-Control"), NGX_HTTP_UPSTREAM_IGN_CACHE_CONTROL },
     { ngx_string("Set-Cookie"), NGX_HTTP_UPSTREAM_IGN_SET_COOKIE },
     { ngx_string("Vary"), NGX_HTTP_UPSTREAM_IGN_VARY },
+    { ngx_string("Age"), NGX_HTTP_UPSTREAM_IGN_AGE },
     { ngx_null_string, 0 }
 };
 
@@ -4939,13 +4946,14 @@ ngx_http_upstream_process_cache_control(
             return NGX_OK;
         }
 
-        if (n == 0) {
+        if (n <= u->headers_in.age_n) {
             u->headers_in.no_cache = 1;
             return NGX_OK;
         }
 
-        r->cache->valid_sec = ngx_time() + n;
+        r->cache->valid_sec = ngx_time() + n - u->headers_in.age_n;
         u->headers_in.expired = 0;
+        u->headers_in.max_age = 1;
     }
 
 extensions:
@@ -5109,6 +5117,7 @@ ngx_http_upstream_process_accel_expires(
             r->cache->valid_sec = ngx_time() + n;
             u->headers_in.no_cache = 0;
             u->headers_in.expired = 0;
+            u->headers_in.max_age = 0;
             return NGX_OK;
         }
     }
@@ -5122,6 +5131,7 @@ ngx_http_upstream_process_accel_expires(
         r->cache->valid_sec = n;
         u->headers_in.no_cache = 0;
         u->headers_in.expired = 0;
+        u->headers_in.max_age = 0;
     }
     }
 #endif
@@ -5373,6 +5383,61 @@ ngx_http_upstream_process_vary(ngx_http_
 
 
 static ngx_int_t
+ngx_http_upstream_process_age(ngx_http_request_t *r,
+    ngx_table_elt_t *h, ngx_uint_t offset)
+{
+    ngx_http_upstream_t  *u;
+
+    u = r->upstream;
+
+    if (u->headers_in.age) {
+        ngx_log_error(NGX_LOG_WARN, r->connection->log, 0,
+                      "upstream sent duplicate header line: \"%V: %V\", "
+                      "previous value: \"%V: %V\", ignored",
+                      &h->key, &h->value,
+                      &u->headers_in.age->key,
+                      &u->headers_in.age->value);
+        h->hash = 0;
+        return NGX_OK;
+    }
+
+    u->headers_in.age = h;
+    h->next = NULL;
+
+#if (NGX_HTTP_CACHE)
+    {
+    ngx_int_t  n;
+
+    if (u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_AGE) {
+        return NGX_OK;
+    }
+
+    if (r->cache == NULL || !u->cacheable) {
+        return NGX_OK;
+    }
+
+    n = ngx_atoi(h->value.data, h->value.len);
+
+    if (n != NGX_ERROR) {
+        u->headers_in.age_n = n;
+
+        if (u->headers_in.max_age) {
+            if (n >= r->cache->valid_sec - ngx_time()) {
+                u->headers_in.no_cache = 1;
+                return NGX_OK;
+            }
+
+            r->cache->valid_sec -= n;
+        }
+    }
+    }
+#endif
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_http_upstream_copy_header_line(ngx_http_request_t *r, ngx_table_elt_t *h,
     ngx_uint_t offset)
 {
diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h
+++ b/src/http/ngx_http_upstream.h
@@ -54,6 +54,7 @@
 #define NGX_HTTP_UPSTREAM_IGN_XA_BUFFERING   0x00000080
 #define NGX_HTTP_UPSTREAM_IGN_XA_CHARSET     0x00000100
 #define NGX_HTTP_UPSTREAM_IGN_VARY           0x00000200
+#define NGX_HTTP_UPSTREAM_IGN_AGE            0x00000400
 
 
 typedef struct {
@@ -287,14 +288,17 @@ typedef struct {
 
     ngx_table_elt_t                 *cache_control;
     ngx_table_elt_t                 *set_cookie;
+    ngx_table_elt_t                 *age;
 
     off_t                            content_length_n;
     time_t                           last_modified_time;
+    time_t                           age_n;
 
     unsigned                         connection_close:1;
     unsigned                         chunked:1;
     unsigned                         no_cache:1;
     unsigned                         expired:1;
+    unsigned                         max_age:1;
 } ngx_http_upstream_headers_in_t;
 
 
# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1721260119 -10800
#      Thu Jul 18 02:48:39 2024 +0300
# Node ID ea9e8bb70744f245e4d381f13766ffb4470515b3
# Parent  0843408f7af487d59800edcd773eddab33eade05
Upstream: $upstream_cache_age variable.

The variable reflects response age, including the time spent in the
cache and the upstream response age as obtained from the "Age" header.
If the response wasn't cached, the variable reflects the "Age" header
of the upstream response.

If the intended use case is to cache responses as per HTTP/1.1 caching
model, the $upstream_cache_age variable can be used to provide the "Age"
header with the "add_header" directive, such as:

    add_header Age $upstream_cache_age;

This now removes the "Age" header if it was present.

Further, the "expires" directives now removes the "Age" header if it
was present in the response, as the "expires" directive assumes zero
age when it adds "Expires" and "Cache-Control" headers.

diff --git a/src/http/modules/ngx_http_headers_filter_module.c b/src/http/modules/ngx_http_headers_filter_module.c
--- a/src/http/modules/ngx_http_headers_filter_module.c
+++ b/src/http/modules/ngx_http_headers_filter_module.c
@@ -93,6 +93,10 @@ static ngx_http_set_header_t  ngx_http_s
                  offsetof(ngx_http_headers_out_t, etag),
                  ngx_http_set_response_header },
 
+    { ngx_string("Age"),
+                 offsetof(ngx_http_headers_out_t, age),
+                 ngx_http_set_response_header },
+
     { ngx_null_string, 0, NULL }
 };
 
@@ -352,6 +356,11 @@ ngx_http_set_expires(ngx_http_request_t 
         }
     }
 
+    if (r->headers_out.age) {
+        r->headers_out.age->hash = 0;
+        r->headers_out.age = NULL;
+    }
+
     e = r->headers_out.expires;
 
     if (e == NULL) {
diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -278,6 +278,7 @@ typedef struct {
 
     ngx_table_elt_t                  *cache_control;
     ngx_table_elt_t                  *link;
+    ngx_table_elt_t                  *age;
 
     ngx_str_t                        *override_charset;
 
diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -31,6 +31,8 @@ static ngx_int_t ngx_http_upstream_cache
     ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_upstream_cache_etag(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_http_upstream_cache_age(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data);
 #endif
 
 static void ngx_http_upstream_init_request(ngx_http_request_t *r);
@@ -297,7 +299,8 @@ static ngx_http_upstream_header_t  ngx_h
 
     { ngx_string("Age"),
                  ngx_http_upstream_process_age, 0,
-                 ngx_http_upstream_copy_header_line, 0, 0 },
+                 ngx_http_upstream_copy_header_line,
+                 offsetof(ngx_http_headers_out_t, age), 0 },
 
     { ngx_string("X-Accel-Expires"),
                  ngx_http_upstream_process_accel_expires, 0,
@@ -436,6 +439,10 @@ static ngx_http_variable_t  ngx_http_ups
       ngx_http_upstream_cache_etag, 0,
       NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 },
 
+    { ngx_string("upstream_cache_age"), NULL,
+      ngx_http_upstream_cache_age, 0,
+      NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 },
+
 #endif
 
     { ngx_string("upstream_http_"), NULL, ngx_http_upstream_header_variable,
@@ -6211,6 +6218,60 @@ ngx_http_upstream_cache_etag(ngx_http_re
     return NGX_OK;
 }
 
+
+static ngx_int_t
+ngx_http_upstream_cache_age(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data)
+{
+    u_char  *p;
+    time_t   now, age;
+
+    if (r->upstream == NULL) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    if (!r->cached
+        || r->cache == NULL
+        || r->upstream->cache_status == NGX_HTTP_CACHE_REVALIDATED)
+    {
+        if (r->upstream->headers_in.age == NULL) {
+            v->not_found = 1;
+            return NGX_OK;
+        }
+
+        v->valid = 1;
+        v->no_cacheable = 0;
+        v->not_found = 0;
+        v->len = r->upstream->headers_in.age->value.len;
+        v->data = r->upstream->headers_in.age->value.data;
+
+        return NGX_OK;
+    }
+
+    p = ngx_pnalloc(r->pool, NGX_TIME_T_LEN);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    now = ngx_time();
+    age = now - r->cache->date;
+
+    if (r->cache->date > now) {
+        age = 0;
+    }
+
+    age += r->upstream->headers_in.age_n;
+
+    v->len = ngx_sprintf(p, "%T", age) - p;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->data = p;
+
+    return NGX_OK;
+}
+
 #endif
 
 

Tests:

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1721260388 -10800
#      Thu Jul 18 02:53:08 2024 +0300
# Node ID bae8b56d95f6810e146fb6e681229189526973f5
# Parent  85d88cd5091c6691d58de81dec544c901f33b6d7
Tests: proxy cache Age header handling tests.

diff --git a/proxy_cache_age.t b/proxy_cache_age.t
new file mode 100644
--- /dev/null
+++ b/proxy_cache_age.t
@@ -0,0 +1,145 @@
+#!/usr/bin/perl
+
+# (C) Maxim Dounin
+
+# Tests for proxy cache Age header handling.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http proxy cache/)->plan(13);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    proxy_cache_path %%TESTDIR%%/cache keys_zone=NAME:1m;
+    proxy_cache_key $request_uri;
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+
+        add_header  X-Cache-Status  $upstream_cache_status;
+        add_header  Age  $upstream_cache_age;
+
+        location / {
+            proxy_pass  http://127.0.0.1:8081;
+            proxy_cache NAME;
+        }
+
+        location /revalidate {
+            proxy_pass  http://127.0.0.1:8081;
+            proxy_cache NAME;
+            proxy_cache_revalidate on;
+        }
+
+        location /ignore/ {
+            proxy_pass  http://127.0.0.1:8081/;
+            proxy_cache NAME;
+
+            proxy_ignore_headers Age;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+
+        location /fresh {
+            add_header Cache-Control max-age=100;
+            add_header Age 90;
+        }
+
+        location /stale {
+            add_header Cache-Control max-age=100;
+            add_header Age 110;
+        }
+
+        location /before {
+            add_header Age 110;
+            add_header Cache-Control max-age=100;
+        }
+
+        location /noage {
+            add_header Cache-Control max-age=100;
+        }
+
+        location /revalidate {
+            add_header Cache-Control max-age=1;
+        }
+    }
+}
+
+EOF
+
+$t->write_file('fresh', 'SEE-THIS');
+$t->write_file('stale', 'SEE-THIS');
+$t->write_file('before', 'SEE-THIS');
+$t->write_file('noage', 'SEE-THIS');
+$t->write_file('revalidate', 'SEE-THIS');
+
+$t->try_run('no age support');
+
+###############################################################################
+
+# responses with Age header cached
+
+like(get('/fresh'), qr/HIT/, 'fresh cached');
+like(get('/stale'), qr/MISS/, 'stale not cached');
+like(get('/before'), qr/MISS/, 'stale age first not cached');
+like(get('/noage'), qr/HIT/, 'noage cached');
+like(get('/revalidate'), qr/HIT/, 'revalidate cached');
+
+# the same with the Age header ignored
+
+like(get('/ignore/fresh'), qr/HIT/, 'fresh ignore');
+like(get('/ignore/stale'), qr/HIT/, 'stale ignore');
+like(get('/ignore/before'), qr/HIT/, 'stale age first ignore');
+like(get('/ignore/noage'), qr/HIT/, 'noage ignore');
+
+# age header updated on cached responses
+
+sleep(2);
+
+like(http_get('/fresh'), qr/(?<!Age:.{,200})Age: 9[1-5](?!.*Age:)/ms,
+	'cached age updated');
+like(http_get('/stale'), qr/(?<!Age:.{,200})Age: 110(?!.*Age:)/ms,
+	'not cached age preserved');
+like(http_get('/noage'), qr/(?<!Age:.{,200})Age: [1-5](?!.*Age:)/ms,
+	'noage age added');
+
+like(http_get('/revalidate'), qr/REVALIDATED(?!.*Age:)/ms,
+	'revalidate age not added');
+
+###############################################################################
+
+sub get {
+	my ($uri) = @_;
+	http_get($uri);
+	http_get($uri);
+}
+
+###############################################################################

-- 
Maxim Dounin
http://mdounin.ru/

From hnakamur at gmail.com  Thu Jul 18 12:51:53 2024
From: hnakamur at gmail.com (Hiroaki Nakamura)
Date: Thu, 18 Jul 2024 21:51:53 +0900
Subject: [PATCH 1 of 3] Correctly calculate and set Age header
In-Reply-To: <ZphhKKqxCoOKM5jT@mdounin.ru>
References: <CAN-DUMQwyKd07fkbpa5osUnAH0sJvdwT=QgO6aGtCpqmn2iBNw@mail.gmail.com>
 <Zniy3EFG005hIsXl@mdounin.ru>
 <CAN-DUMQzHifqXq4Uc+gRQqNG4pKuNcz=Zwz_+K+Srx-QKWW34Q@mail.gmail.com>
 <Znywy92srHcefS69@mdounin.ru>
 <CAN-DUMRpSuOnYWe3O2_X7x-P_BMwQ4=h7RYm2W_rNqTP8KReaQ@mail.gmail.com>
 <ZphhKKqxCoOKM5jT@mdounin.ru>
Message-ID: <CAN-DUMT61s=ZUOHqGiRaTu6M2qm+sSRFFF5KW+cNBJkwSVYYrA@mail.gmail.com>

Hello,

2024?7?18?(?) 9:26 Maxim Dounin <mdounin at mdounin.ru>:
>
> Hello!
>
> On Mon, Jul 01, 2024 at 10:45:59PM +0900, Hiroaki Nakamura wrote:
>
> > Hello
> >
> > 2024?6?27?(?) 9:32 Maxim Dounin <mdounin at mdounin.ru>:
[...]
>
> Thanks for yet another version of the patch.  In its current form
> it starts to look reasonable.
>
> I've spent some time looking into this, including looking through
> previous attempts to implement some forms of Age support, notably:
[...]

Thank you for taking your time to investigate and implement Age support!

>
> Overall, there seems to be two major parts of what is expected to
> be done here:
>
> 1. Adjusting cache validity times based on the Age header in the
> upstream server response.
>
> 2. Returning cached responses with updated/added Age header.
>
> These two parts probably can be implemented separately, with
> minimal interdependencies.

I agree.

>
> Further, I tend to think that, while (1) probably can be the
> default, it might not be a good idea to do (2) by default, as in
> most setups [free]nginx is used as an origin server, even if
> caching is enabled.  In such configurations administrators often
> use "Cache-Control: max-age=N" for client-side caching, and
> control caching in [free]nginx separately, such as with
> proxy_cache_valid.  Adding the Age header in such configurations
> might result in unexpectedly non-working client-side caching.
>
> In particular, (2) can be implemented as a variable with relevant
> information to be used in the "add_header" directive if one needs
> to implement Age updating.  Alternative approach would be to add
> per-module directives to control (2), such as "proxy_cache_age on|off",
> and add the Age header in the upstream module, though a variable
> looks easier.

I agree the latter approach is better.

>
> Below are two patches that implement (1) adjusting cache validity
> base on the Age header and (2) a variable to set the Age header,
> and yet another patch to implement tests.  Please take a look.
[...]

I have looked at the two patches and the test.
Yes, it achieves what you described and it is a lot simpler and clearer
than my patches.
I also have verified that it covers both cases of orders of
cache-control and age headers, that is cache-control before age and
cache-control after age.

I have run the test with freenginx with the two patches applied and
verified it passes.

I wrote tests using HTTP/2 and HTTP/3 based on your test and
verified these tests pass too.
That means we don't need to modify http_v2_module and http_v3_module.

In summary, your patches and tests seem perfect to me.

Thank you,

From mdounin at mdounin.ru  Thu Jul 18 20:43:59 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Thu, 18 Jul 2024 23:43:59 +0300
Subject: [nginx] Version bump.
Message-ID: <hg.f209bf590e84.1721335439.-3142137246386495026@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx/rev/f209bf590e84
branches:  
changeset: 9304:f209bf590e84
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Jul 18 19:39:29 2024 +0300
description:
Version bump.

diffstat:

 src/core/nginx.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff --git a/src/core/nginx.h b/src/core/nginx.h
--- a/src/core/nginx.h
+++ b/src/core/nginx.h
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1027002
-#define NGINX_VERSION      "1.27.2"
+#define nginx_version      1027003
+#define NGINX_VERSION      "1.27.3"
 
 #define NGINX_NAME         "freenginx"
 #define NGINX_VER          NGINX_NAME "/" NGINX_VERSION

From mdounin at mdounin.ru  Thu Jul 18 20:43:59 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Thu, 18 Jul 2024 23:43:59 +0300
Subject: [nginx] Upstream: using the "Age" header when caching responses.
Message-ID: <hg.8cdab3d89c44.1721335439.-3142137246386495026@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx/rev/8cdab3d89c44
branches:  
changeset: 9305:8cdab3d89c44
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Jul 18 19:39:39 2024 +0300
description:
Upstream: using the "Age" header when caching responses.

As long as the "Age" header is present and not ignored, it is now
respected when caching responses per "Cache-Control: max-age", reducing
cache validity time.

diffstat:

 src/http/ngx_http_upstream.c |  69 ++++++++++++++++++++++++++++++++++++++++++-
 src/http/ngx_http_upstream.h |   4 ++
 2 files changed, 71 insertions(+), 2 deletions(-)

diffs (155 lines):

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -136,6 +136,8 @@ static ngx_int_t
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_upstream_process_vary(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
+static ngx_int_t ngx_http_upstream_process_age(ngx_http_request_t *r,
+    ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_upstream_copy_header_line(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t
@@ -293,6 +295,10 @@ static ngx_http_upstream_header_t  ngx_h
                  ngx_http_upstream_copy_multi_header_lines,
                  offsetof(ngx_http_headers_out_t, link), 0 },
 
+    { ngx_string("Age"),
+                 ngx_http_upstream_process_age, 0,
+                 ngx_http_upstream_copy_header_line, 0, 0 },
+
     { ngx_string("X-Accel-Expires"),
                  ngx_http_upstream_process_accel_expires, 0,
                  ngx_http_upstream_copy_header_line, 0, 0 },
@@ -475,6 +481,7 @@ ngx_conf_bitmask_t  ngx_http_upstream_ig
     { ngx_string("Cache-Control"), NGX_HTTP_UPSTREAM_IGN_CACHE_CONTROL },
     { ngx_string("Set-Cookie"), NGX_HTTP_UPSTREAM_IGN_SET_COOKIE },
     { ngx_string("Vary"), NGX_HTTP_UPSTREAM_IGN_VARY },
+    { ngx_string("Age"), NGX_HTTP_UPSTREAM_IGN_AGE },
     { ngx_null_string, 0 }
 };
 
@@ -4939,13 +4946,14 @@ ngx_http_upstream_process_cache_control(
             return NGX_OK;
         }
 
-        if (n == 0) {
+        if (n <= u->headers_in.age_n) {
             u->headers_in.no_cache = 1;
             return NGX_OK;
         }
 
-        r->cache->valid_sec = ngx_time() + n;
+        r->cache->valid_sec = ngx_time() + n - u->headers_in.age_n;
         u->headers_in.expired = 0;
+        u->headers_in.max_age = 1;
     }
 
 extensions:
@@ -5109,6 +5117,7 @@ ngx_http_upstream_process_accel_expires(
             r->cache->valid_sec = ngx_time() + n;
             u->headers_in.no_cache = 0;
             u->headers_in.expired = 0;
+            u->headers_in.max_age = 0;
             return NGX_OK;
         }
     }
@@ -5122,6 +5131,7 @@ ngx_http_upstream_process_accel_expires(
         r->cache->valid_sec = n;
         u->headers_in.no_cache = 0;
         u->headers_in.expired = 0;
+        u->headers_in.max_age = 0;
     }
     }
 #endif
@@ -5373,6 +5383,61 @@ ngx_http_upstream_process_vary(ngx_http_
 
 
 static ngx_int_t
+ngx_http_upstream_process_age(ngx_http_request_t *r,
+    ngx_table_elt_t *h, ngx_uint_t offset)
+{
+    ngx_http_upstream_t  *u;
+
+    u = r->upstream;
+
+    if (u->headers_in.age) {
+        ngx_log_error(NGX_LOG_WARN, r->connection->log, 0,
+                      "upstream sent duplicate header line: \"%V: %V\", "
+                      "previous value: \"%V: %V\", ignored",
+                      &h->key, &h->value,
+                      &u->headers_in.age->key,
+                      &u->headers_in.age->value);
+        h->hash = 0;
+        return NGX_OK;
+    }
+
+    u->headers_in.age = h;
+    h->next = NULL;
+
+#if (NGX_HTTP_CACHE)
+    {
+    ngx_int_t  n;
+
+    if (u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_AGE) {
+        return NGX_OK;
+    }
+
+    if (r->cache == NULL || !u->cacheable) {
+        return NGX_OK;
+    }
+
+    n = ngx_atoi(h->value.data, h->value.len);
+
+    if (n != NGX_ERROR) {
+        u->headers_in.age_n = n;
+
+        if (u->headers_in.max_age) {
+            if (n >= r->cache->valid_sec - ngx_time()) {
+                u->headers_in.no_cache = 1;
+                return NGX_OK;
+            }
+
+            r->cache->valid_sec -= n;
+        }
+    }
+    }
+#endif
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_http_upstream_copy_header_line(ngx_http_request_t *r, ngx_table_elt_t *h,
     ngx_uint_t offset)
 {
diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h
+++ b/src/http/ngx_http_upstream.h
@@ -54,6 +54,7 @@
 #define NGX_HTTP_UPSTREAM_IGN_XA_BUFFERING   0x00000080
 #define NGX_HTTP_UPSTREAM_IGN_XA_CHARSET     0x00000100
 #define NGX_HTTP_UPSTREAM_IGN_VARY           0x00000200
+#define NGX_HTTP_UPSTREAM_IGN_AGE            0x00000400
 
 
 typedef struct {
@@ -287,14 +288,17 @@ typedef struct {
 
     ngx_table_elt_t                 *cache_control;
     ngx_table_elt_t                 *set_cookie;
+    ngx_table_elt_t                 *age;
 
     off_t                            content_length_n;
     time_t                           last_modified_time;
+    time_t                           age_n;
 
     unsigned                         connection_close:1;
     unsigned                         chunked:1;
     unsigned                         no_cache:1;
     unsigned                         expired:1;
+    unsigned                         max_age:1;
 } ngx_http_upstream_headers_in_t;
 
 

From mdounin at mdounin.ru  Thu Jul 18 20:44:00 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Thu, 18 Jul 2024 23:44:00 +0300
Subject: [nginx] Upstream: $upstream_cache_age variable.
Message-ID: <hg.e46e1ea89ccd.1721335440.-3142137246386495026@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx/rev/e46e1ea89ccd
branches:  
changeset: 9306:e46e1ea89ccd
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Jul 18 19:39:45 2024 +0300
description:
Upstream: $upstream_cache_age variable.

The variable reflects response age, including the time spent in the
cache and the upstream response age as obtained from the "Age" header.
If the response wasn't cached, the variable reflects the "Age" header
of the upstream response.

If the intended use case is to cache responses as per HTTP/1.1 caching
model, the $upstream_cache_age variable can be used to provide the "Age"
header with the "add_header" directive, such as:

    add_header Age $upstream_cache_age;

This now removes the "Age" header if it was present.

Further, the "expires" directives now removes the "Age" header if it
was present in the response, as the "expires" directive assumes zero
age when it adds "Expires" and "Cache-Control" headers.

diffstat:

 src/http/modules/ngx_http_headers_filter_module.c |   9 +++
 src/http/ngx_http_request.h                       |   1 +
 src/http/ngx_http_upstream.c                      |  63 ++++++++++++++++++++++-
 3 files changed, 72 insertions(+), 1 deletions(-)

diffs (131 lines):

diff --git a/src/http/modules/ngx_http_headers_filter_module.c b/src/http/modules/ngx_http_headers_filter_module.c
--- a/src/http/modules/ngx_http_headers_filter_module.c
+++ b/src/http/modules/ngx_http_headers_filter_module.c
@@ -93,6 +93,10 @@ static ngx_http_set_header_t  ngx_http_s
                  offsetof(ngx_http_headers_out_t, etag),
                  ngx_http_set_response_header },
 
+    { ngx_string("Age"),
+                 offsetof(ngx_http_headers_out_t, age),
+                 ngx_http_set_response_header },
+
     { ngx_null_string, 0, NULL }
 };
 
@@ -352,6 +356,11 @@ ngx_http_set_expires(ngx_http_request_t 
         }
     }
 
+    if (r->headers_out.age) {
+        r->headers_out.age->hash = 0;
+        r->headers_out.age = NULL;
+    }
+
     e = r->headers_out.expires;
 
     if (e == NULL) {
diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -278,6 +278,7 @@ typedef struct {
 
     ngx_table_elt_t                  *cache_control;
     ngx_table_elt_t                  *link;
+    ngx_table_elt_t                  *age;
 
     ngx_str_t                        *override_charset;
 
diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -31,6 +31,8 @@ static ngx_int_t ngx_http_upstream_cache
     ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_upstream_cache_etag(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_http_upstream_cache_age(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data);
 #endif
 
 static void ngx_http_upstream_init_request(ngx_http_request_t *r);
@@ -297,7 +299,8 @@ static ngx_http_upstream_header_t  ngx_h
 
     { ngx_string("Age"),
                  ngx_http_upstream_process_age, 0,
-                 ngx_http_upstream_copy_header_line, 0, 0 },
+                 ngx_http_upstream_copy_header_line,
+                 offsetof(ngx_http_headers_out_t, age), 0 },
 
     { ngx_string("X-Accel-Expires"),
                  ngx_http_upstream_process_accel_expires, 0,
@@ -436,6 +439,10 @@ static ngx_http_variable_t  ngx_http_ups
       ngx_http_upstream_cache_etag, 0,
       NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 },
 
+    { ngx_string("upstream_cache_age"), NULL,
+      ngx_http_upstream_cache_age, 0,
+      NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 },
+
 #endif
 
     { ngx_string("upstream_http_"), NULL, ngx_http_upstream_header_variable,
@@ -6211,6 +6218,60 @@ ngx_http_upstream_cache_etag(ngx_http_re
     return NGX_OK;
 }
 
+
+static ngx_int_t
+ngx_http_upstream_cache_age(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data)
+{
+    u_char  *p;
+    time_t   now, age;
+
+    if (r->upstream == NULL) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    if (!r->cached
+        || r->cache == NULL
+        || r->upstream->cache_status == NGX_HTTP_CACHE_REVALIDATED)
+    {
+        if (r->upstream->headers_in.age == NULL) {
+            v->not_found = 1;
+            return NGX_OK;
+        }
+
+        v->valid = 1;
+        v->no_cacheable = 0;
+        v->not_found = 0;
+        v->len = r->upstream->headers_in.age->value.len;
+        v->data = r->upstream->headers_in.age->value.data;
+
+        return NGX_OK;
+    }
+
+    p = ngx_pnalloc(r->pool, NGX_TIME_T_LEN);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    now = ngx_time();
+    age = now - r->cache->date;
+
+    if (r->cache->date > now) {
+        age = 0;
+    }
+
+    age += r->upstream->headers_in.age_n;
+
+    v->len = ngx_sprintf(p, "%T", age) - p;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->data = p;
+
+    return NGX_OK;
+}
+
 #endif
 
 

From mdounin at mdounin.ru  Thu Jul 18 20:44:18 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Thu, 18 Jul 2024 23:44:18 +0300
Subject: [nginx-tests] Tests: proxy cache Age header handling tests.
Message-ID: <hg.a8c4d48a4073.1721335458.426132123594550804@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx-tests/rev/a8c4d48a4073
branches:  
changeset: 1991:a8c4d48a4073
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Jul 18 19:40:19 2024 +0300
description:
Tests: proxy cache Age header handling tests.

diffstat:

 proxy_cache_age.t |  144 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 144 insertions(+), 0 deletions(-)

diffs (149 lines):

diff --git a/proxy_cache_age.t b/proxy_cache_age.t
new file mode 100644
--- /dev/null
+++ b/proxy_cache_age.t
@@ -0,0 +1,144 @@
+#!/usr/bin/perl
+
+# (C) Maxim Dounin
+
+# Tests for proxy cache Age header handling.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http proxy cache/)
+	->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    proxy_cache_path %%TESTDIR%%/cache keys_zone=NAME:1m;
+    proxy_cache_key $request_uri;
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+
+        add_header  X-Cache-Status  $upstream_cache_status;
+        add_header  Age  $upstream_cache_age;
+
+        location / {
+            proxy_pass  http://127.0.0.1:8081;
+            proxy_cache NAME;
+        }
+
+        location /revalidate {
+            proxy_pass  http://127.0.0.1:8081;
+            proxy_cache NAME;
+            proxy_cache_revalidate on;
+        }
+
+        location /ignore/ {
+            proxy_pass  http://127.0.0.1:8081/;
+            proxy_cache NAME;
+
+            proxy_ignore_headers Age;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+
+        location /fresh {
+            add_header Cache-Control max-age=100;
+            add_header Age 90;
+        }
+
+        location /stale {
+            add_header Cache-Control max-age=100;
+            add_header Age 110;
+        }
+
+        location /before {
+            add_header Age 110;
+            add_header Cache-Control max-age=100;
+        }
+
+        location /noage {
+            add_header Cache-Control max-age=100;
+        }
+
+        location /revalidate {
+            add_header Cache-Control max-age=1;
+        }
+    }
+}
+
+EOF
+
+$t->write_file('fresh', 'SEE-THIS');
+$t->write_file('stale', 'SEE-THIS');
+$t->write_file('before', 'SEE-THIS');
+$t->write_file('noage', 'SEE-THIS');
+$t->write_file('revalidate', 'SEE-THIS');
+
+$t->try_run('no age support')->plan(13);
+
+###############################################################################
+
+# responses with Age header cached
+
+like(get('/fresh'), qr/HIT/, 'fresh cached');
+like(get('/stale'), qr/MISS/, 'stale not cached');
+like(get('/before'), qr/MISS/, 'stale age first not cached');
+like(get('/noage'), qr/HIT/, 'noage cached');
+like(get('/revalidate'), qr/HIT/, 'revalidate cached');
+
+# the same with the Age header ignored
+
+like(get('/ignore/fresh'), qr/HIT/, 'fresh ignore');
+like(get('/ignore/stale'), qr/HIT/, 'stale ignore');
+like(get('/ignore/before'), qr/HIT/, 'stale age first ignore');
+like(get('/ignore/noage'), qr/HIT/, 'noage ignore');
+
+# age header updated on cached responses
+
+sleep(2);
+
+like(http_get('/fresh'), qr/(?<!Age:.{,200})Age: 9[1-5](?!.*Age:)/ms,
+	'cached age updated');
+like(http_get('/stale'), qr/(?<!Age:.{,200})Age: 110(?!.*Age:)/ms,
+	'not cached age preserved');
+like(http_get('/noage'), qr/(?<!Age:.{,200})Age: [1-5](?!.*Age:)/ms,
+	'noage age added');
+
+like(http_get('/revalidate'), qr/REVALIDATED(?!.*Age:)/ms,
+	'revalidate age not added');
+
+###############################################################################
+
+sub get {
+	my ($uri) = @_;
+	http_get($uri);
+	http_get($uri);
+}
+
+###############################################################################

From mdounin at mdounin.ru  Thu Jul 18 20:49:13 2024
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 18 Jul 2024 23:49:13 +0300
Subject: [PATCH 1 of 3] Correctly calculate and set Age header
In-Reply-To: <CAN-DUMT61s=ZUOHqGiRaTu6M2qm+sSRFFF5KW+cNBJkwSVYYrA@mail.gmail.com>
References: <CAN-DUMQwyKd07fkbpa5osUnAH0sJvdwT=QgO6aGtCpqmn2iBNw@mail.gmail.com>
 <Zniy3EFG005hIsXl@mdounin.ru>
 <CAN-DUMQzHifqXq4Uc+gRQqNG4pKuNcz=Zwz_+K+Srx-QKWW34Q@mail.gmail.com>
 <Znywy92srHcefS69@mdounin.ru>
 <CAN-DUMRpSuOnYWe3O2_X7x-P_BMwQ4=h7RYm2W_rNqTP8KReaQ@mail.gmail.com>
 <ZphhKKqxCoOKM5jT@mdounin.ru>
 <CAN-DUMT61s=ZUOHqGiRaTu6M2qm+sSRFFF5KW+cNBJkwSVYYrA@mail.gmail.com>
Message-ID: <Zpl_yebQTsot6q_-@mdounin.ru>

Hello!

On Thu, Jul 18, 2024 at 09:51:53PM +0900, Hiroaki Nakamura wrote:

> Hello,
> 
> 2024?7?18?(?) 9:26 Maxim Dounin <mdounin at mdounin.ru>:
> >
> > Hello!
> >
> > On Mon, Jul 01, 2024 at 10:45:59PM +0900, Hiroaki Nakamura wrote:
> >
> > > Hello
> > >
> > > 2024?6?27?(?) 9:32 Maxim Dounin <mdounin at mdounin.ru>:
> [...]
> >
> > Thanks for yet another version of the patch.  In its current form
> > it starts to look reasonable.
> >
> > I've spent some time looking into this, including looking through
> > previous attempts to implement some forms of Age support, notably:
> [...]
> 
> Thank you for taking your time to investigate and implement Age support!
> 
> >
> > Overall, there seems to be two major parts of what is expected to
> > be done here:
> >
> > 1. Adjusting cache validity times based on the Age header in the
> > upstream server response.
> >
> > 2. Returning cached responses with updated/added Age header.
> >
> > These two parts probably can be implemented separately, with
> > minimal interdependencies.
> 
> I agree.
> 
> >
> > Further, I tend to think that, while (1) probably can be the
> > default, it might not be a good idea to do (2) by default, as in
> > most setups [free]nginx is used as an origin server, even if
> > caching is enabled.  In such configurations administrators often
> > use "Cache-Control: max-age=N" for client-side caching, and
> > control caching in [free]nginx separately, such as with
> > proxy_cache_valid.  Adding the Age header in such configurations
> > might result in unexpectedly non-working client-side caching.
> >
> > In particular, (2) can be implemented as a variable with relevant
> > information to be used in the "add_header" directive if one needs
> > to implement Age updating.  Alternative approach would be to add
> > per-module directives to control (2), such as "proxy_cache_age on|off",
> > and add the Age header in the upstream module, though a variable
> > looks easier.
> 
> I agree the latter approach is better.
> 
> >
> > Below are two patches that implement (1) adjusting cache validity
> > base on the Age header and (2) a variable to set the Age header,
> > and yet another patch to implement tests.  Please take a look.
> [...]
> 
> I have looked at the two patches and the test.
> Yes, it achieves what you described and it is a lot simpler and clearer
> than my patches.
> I also have verified that it covers both cases of orders of
> cache-control and age headers, that is cache-control before age and
> cache-control after age.
> 
> I have run the test with freenginx with the two patches applied and
> verified it passes.
> 
> I wrote tests using HTTP/2 and HTTP/3 based on your test and
> verified these tests pass too.
> That means we don't need to modify http_v2_module and http_v3_module.
> 
> In summary, your patches and tests seem perfect to me.

Thanks for the review, committed.  And thanks for prodding this, 
appreciated.

Just in case, below is a preliminary documentation patch.

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1721325059 -10800
#      Thu Jul 18 20:50:59 2024 +0300
# Node ID 653583426bf2a93ee6bb6796881a4633cfca5503
# Parent  6cab598f14a1f4bbf2e8e4f374f6d15c18c54659
Documented Age header handling.

diff --git a/xml/en/docs/http/ngx_http_fastcgi_module.xml b/xml/en/docs/http/ngx_http_fastcgi_module.xml
--- a/xml/en/docs/http/ngx_http_fastcgi_module.xml
+++ b/xml/en/docs/http/ngx_http_fastcgi_module.xml
@@ -10,7 +10,7 @@
 <module name="Module ngx_http_fastcgi_module"
         link="/en/docs/http/ngx_http_fastcgi_module.html"
         lang="en"
-        rev="54">
+        rev="55">
 
 <section id="summary">
 
@@ -649,6 +649,12 @@ with another value, such a response will
 taking into account the corresponding request header fields (1.7.7).
 </listitem>
 
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
 </list>
 Processing of one or more of these response header fields can be disabled
 using the <link id="fastcgi_ignore_headers"/> directive.
@@ -766,7 +772,7 @@ The following fields can be ignored: <he
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -777,7 +783,7 @@ effect:
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
 set the parameters of response <link id="fastcgi_cache_valid">caching</link>;
 </listitem>
 
diff --git a/xml/en/docs/http/ngx_http_proxy_module.xml b/xml/en/docs/http/ngx_http_proxy_module.xml
--- a/xml/en/docs/http/ngx_http_proxy_module.xml
+++ b/xml/en/docs/http/ngx_http_proxy_module.xml
@@ -10,7 +10,7 @@
 <module name="Module ngx_http_proxy_module"
         link="/en/docs/http/ngx_http_proxy_module.html"
         lang="en"
-        rev="77">
+        rev="78">
 
 <section id="summary">
 
@@ -667,6 +667,12 @@ with another value, such a response will
 taking into account the corresponding request header fields (1.7.7).
 </listitem>
 
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
 </list>
 Processing of one or more of these response header fields can be disabled
 using the <link id="proxy_ignore_headers"/> directive.
@@ -1011,7 +1017,7 @@ The following fields can be ignored: <he
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -1022,7 +1028,7 @@ effect:
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
 set the parameters of response <link id="proxy_cache_valid">caching</link>;
 </listitem>
 
diff --git a/xml/en/docs/http/ngx_http_scgi_module.xml b/xml/en/docs/http/ngx_http_scgi_module.xml
--- a/xml/en/docs/http/ngx_http_scgi_module.xml
+++ b/xml/en/docs/http/ngx_http_scgi_module.xml
@@ -10,7 +10,7 @@
 <module name="Module ngx_http_scgi_module"
         link="/en/docs/http/ngx_http_scgi_module.html"
         lang="en"
-        rev="42">
+        rev="43">
 
 <section id="summary">
 
@@ -640,6 +640,12 @@ with another value, such a response will
 taking into account the corresponding request header fields (1.7.7).
 </listitem>
 
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
 </list>
 Processing of one or more of these response header fields can be disabled
 using the <link id="scgi_ignore_headers"/> directive.
@@ -731,7 +737,7 @@ The following fields can be ignored: <he
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -742,7 +748,7 @@ effect:
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
 set the parameters of response <link id="scgi_cache_valid">caching</link>;
 </listitem>
 
diff --git a/xml/en/docs/http/ngx_http_upstream_module.xml b/xml/en/docs/http/ngx_http_upstream_module.xml
--- a/xml/en/docs/http/ngx_http_upstream_module.xml
+++ b/xml/en/docs/http/ngx_http_upstream_module.xml
@@ -10,7 +10,7 @@
 <module name="Module ngx_http_upstream_module"
         link="/en/docs/http/ngx_http_upstream_module.html"
         lang="en"
-        rev="90">
+        rev="91">
 
 <section id="summary">
 
@@ -574,6 +574,12 @@ are separated by commas and colons like 
 <link id="var_upstream_addr">$upstream_addr</link> variable.
 </tag-desc>
 
+<tag-name id="var_upstream_cache_age"><var>$upstream_cache_age</var>
+</tag-name>
+<tag-desc>
+age of the cache item (1.27.3).
+</tag-desc>
+
 <tag-name id="var_upstream_cache_key"><var>$upstream_cache_key</var>
 </tag-name>
 <tag-desc>
diff --git a/xml/en/docs/http/ngx_http_uwsgi_module.xml b/xml/en/docs/http/ngx_http_uwsgi_module.xml
--- a/xml/en/docs/http/ngx_http_uwsgi_module.xml
+++ b/xml/en/docs/http/ngx_http_uwsgi_module.xml
@@ -10,7 +10,7 @@
 <module name="Module ngx_http_uwsgi_module"
         link="/en/docs/http/ngx_http_uwsgi_module.html"
         lang="en"
-        rev="51">
+        rev="52">
 
 <section id="summary">
 
@@ -640,6 +640,12 @@ with another value, such a response will
 taking into account the corresponding request header fields (1.7.7).
 </listitem>
 
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
 </list>
 Processing of one or more of these response header fields can be disabled
 using the <link id="uwsgi_ignore_headers"/> directive.
@@ -731,7 +737,7 @@ The following fields can be ignored: <he
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -742,7 +748,7 @@ effect:
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
 set the parameters of response <link id="uwsgi_cache_valid">caching</link>;
 </listitem>
 
diff --git a/xml/ru/docs/http/ngx_http_fastcgi_module.xml b/xml/ru/docs/http/ngx_http_fastcgi_module.xml
--- a/xml/ru/docs/http/ngx_http_fastcgi_module.xml
+++ b/xml/ru/docs/http/ngx_http_fastcgi_module.xml
@@ -10,7 +10,7 @@
 <module name="?????? ngx_http_fastcgi_module"
         link="/ru/docs/http/ngx_http_fastcgi_module.html"
         lang="ru"
-        rev="54">
+        rev="55">
 
 <section id="summary">
 
@@ -649,6 +649,12 @@ fastcgi_cache_valid any      1m;
 ? ?????? ??????????????? ????? ????????? ??????? (1.7.7).
 </listitem>
 
+<listitem>
+???? ? ????????? ?????? ???? ???? <header>Age</header>
+? ????? ??????????? ???????????? ?? ???? <header>Cache-Control</header>,
+????? ??????????? ????? ?????????????? ????????? (1.27.3).
+</listitem>
+
 </list>
 ????????? ?????? ??? ????? ?? ???? ????? ????????? ????? ???? ?????????
 ??? ?????? ????????? <link id="fastcgi_ignore_headers"/>.
@@ -764,8 +770,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 <header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-? <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) ? <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -774,8 +780,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-? <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> ? <header>Age</header>
 ?????? ????????? <link id="fastcgi_cache_valid">???????????</link> ??????;
 </listitem>
 
diff --git a/xml/ru/docs/http/ngx_http_proxy_module.xml b/xml/ru/docs/http/ngx_http_proxy_module.xml
--- a/xml/ru/docs/http/ngx_http_proxy_module.xml
+++ b/xml/ru/docs/http/ngx_http_proxy_module.xml
@@ -10,7 +10,7 @@
 <module name="?????? ngx_http_proxy_module"
         link="/ru/docs/http/ngx_http_proxy_module.html"
         lang="ru"
-        rev="77">
+        rev="78">
 
 <section id="summary">
 
@@ -669,6 +669,12 @@ proxy_cache_valid any      1m;
 ? ?????? ??????????????? ????? ????????? ??????? (1.7.7).
 </listitem>
 
+<listitem>
+???? ? ????????? ?????? ???? ???? <header>Age</header>
+? ????? ??????????? ???????????? ?? ???? <header>Cache-Control</header>,
+????? ??????????? ????? ?????????????? ????????? (1.27.3).
+</listitem>
+
 </list>
 ????????? ?????? ??? ????? ?? ???? ????? ????????? ????? ???? ?????????
 ??? ?????? ????????? <link id="proxy_ignore_headers"/>.
@@ -1010,8 +1016,8 @@ NTLM</link> ????????????? ?????? 1.1.
 <header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-? <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) ? <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -1020,8 +1026,8 @@ NTLM</link> ????????????? ?????? 1.1.
 
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-? <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> ? <header>Age</header>
 ?????? ????????? <link id="proxy_cache_valid">???????????</link> ??????;
 </listitem>
 
diff --git a/xml/ru/docs/http/ngx_http_scgi_module.xml b/xml/ru/docs/http/ngx_http_scgi_module.xml
--- a/xml/ru/docs/http/ngx_http_scgi_module.xml
+++ b/xml/ru/docs/http/ngx_http_scgi_module.xml
@@ -10,7 +10,7 @@
 <module name="?????? ngx_http_scgi_module"
         link="/ru/docs/http/ngx_http_scgi_module.html"
         lang="ru"
-        rev="42">
+        rev="43">
 
 <section id="summary">
 
@@ -640,6 +640,12 @@ scgi_cache_valid any      1m;
 ? ?????? ??????????????? ????? ????????? ??????? (1.7.7).
 </listitem>
 
+<listitem>
+???? ? ????????? ?????? ???? ???? <header>Age</header>
+? ????? ??????????? ???????????? ?? ???? <header>Cache-Control</header>,
+????? ??????????? ????? ?????????????? ????????? (1.27.3).
+</listitem>
+
 </list>
 ????????? ?????? ??? ????? ?? ???? ????? ????????? ????? ???? ?????????
 ??? ?????? ????????? <link id="scgi_ignore_headers"/>.
@@ -729,8 +735,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 <header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-? <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) ? <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -739,8 +745,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-? <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> ? <header>Age</header>
 ?????? ????????? <link id="scgi_cache_valid">???????????</link> ??????;
 </listitem>
 
diff --git a/xml/ru/docs/http/ngx_http_upstream_module.xml b/xml/ru/docs/http/ngx_http_upstream_module.xml
--- a/xml/ru/docs/http/ngx_http_upstream_module.xml
+++ b/xml/ru/docs/http/ngx_http_upstream_module.xml
@@ -10,7 +10,7 @@
 <module name="?????? ngx_http_upstream_module"
         link="/ru/docs/http/ngx_http_upstream_module.html"
         lang="ru"
-        rev="90">
+        rev="91">
 
 <section id="summary">
 
@@ -581,6 +581,12 @@ server {
 <link id="var_upstream_addr">$upstream_addr</link>.
 </tag-desc>
 
+<tag-name id="var_upstream_cache_age"><var>$upstream_cache_age</var>
+</tag-name>
+<tag-desc>
+??????? ???????? ???? (1.27.3).
+</tag-desc>
+
 <tag-name id="var_upstream_cache_key"><var>$upstream_cache_key</var>
 </tag-name>
 <tag-desc>
diff --git a/xml/ru/docs/http/ngx_http_uwsgi_module.xml b/xml/ru/docs/http/ngx_http_uwsgi_module.xml
--- a/xml/ru/docs/http/ngx_http_uwsgi_module.xml
+++ b/xml/ru/docs/http/ngx_http_uwsgi_module.xml
@@ -10,7 +10,7 @@
 <module name="?????? ngx_http_uwsgi_module"
         link="/ru/docs/http/ngx_http_uwsgi_module.html"
         lang="ru"
-        rev="51">
+        rev="52">
 
 <section id="summary">
 
@@ -639,6 +639,12 @@ uwsgi_cache_valid any      1m;
 ? ?????? ??????????????? ????? ????????? ??????? (1.7.7).
 </listitem>
 
+<listitem>
+???? ? ????????? ?????? ???? ???? <header>Age</header>
+? ????? ??????????? ???????????? ?? ???? <header>Cache-Control</header>,
+????? ??????????? ????? ?????????????? ????????? (1.27.3).
+</listitem>
+
 </list>
 ????????? ?????? ??? ????? ?? ???? ????? ????????? ????? ???? ?????????
 ??? ?????? ????????? <link id="uwsgi_ignore_headers"/>.
@@ -728,8 +734,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 <header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-? <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) ? <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -738,8 +744,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-? <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> ? <header>Age</header>
 ?????? ????????? <link id="uwsgi_cache_valid">???????????</link> ??????;
 </listitem>
 

-- 
Maxim Dounin
http://mdounin.ru/

From maksim.yevmenkin at gmail.com  Fri Jul 26 20:14:39 2024
From: maksim.yevmenkin at gmail.com (Maksim Yevmenkin)
Date: Fri, 26 Jul 2024 13:14:39 -0700
Subject: Updating r->cache in ngx_http_upstream_process_cache_control()
Message-ID: <CAFPOs6qnmyQnogOF0bo10YRtLNBtaofm10Q8EZ5oMpxAQr79cQ@mail.gmail.com>

Hello,

I would like to understand the rationale for updating
r->cache->valid_sec (and other fields in r->cache) in
ngx_http_upstream_process_cache_control().

It appears that ngx_http_upstream_process_cache_control() can be
called (via ngx_http_proxy_process_header()) in two scenarios: when
sending a cached response to the client, and when receiving a response
from the upstream server.

Obviously, calling ngx_http_upstream_process_cache_control() when
receiving a response from the upstream server is completely expected.
However, calling ngx_http_upstream_process_cache_control(), and more
specifically, modifying r->cache fields while sending a cached
(NGX_HTTP_CACHE_HIT and possibly others) response is not completely
clear to me.

Thanks!
Max

From mdounin at mdounin.ru  Sat Jul 27 02:07:15 2024
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sat, 27 Jul 2024 05:07:15 +0300
Subject: Updating r->cache in ngx_http_upstream_process_cache_control()
In-Reply-To: <CAFPOs6qnmyQnogOF0bo10YRtLNBtaofm10Q8EZ5oMpxAQr79cQ@mail.gmail.com>
References: <CAFPOs6qnmyQnogOF0bo10YRtLNBtaofm10Q8EZ5oMpxAQr79cQ@mail.gmail.com>
Message-ID: <ZqRWU-J8fVC9u0Vz@mdounin.ru>

Hello!

On Fri, Jul 26, 2024 at 01:14:39PM -0700, Maksim Yevmenkin wrote:

> Hello,
> 
> I would like to understand the rationale for updating
> r->cache->valid_sec (and other fields in r->cache) in
> ngx_http_upstream_process_cache_control().
> 
> It appears that ngx_http_upstream_process_cache_control() can be
> called (via ngx_http_proxy_process_header()) in two scenarios: when
> sending a cached response to the client, and when receiving a response
> from the upstream server.
> 
> Obviously, calling ngx_http_upstream_process_cache_control() when
> receiving a response from the upstream server is completely expected.
> However, calling ngx_http_upstream_process_cache_control(), and more
> specifically, modifying r->cache fields while sending a cached
> (NGX_HTTP_CACHE_HIT and possibly others) response is not completely
> clear to me.

Parsing cache validity time from cached responses is used at least 
when sending revalidated responses (NGX_HTTP_CACHE_REVALIDATED).

For cached responses with other cache statuses 
(NGX_HTTP_CACHE_HIT, NGX_HTTP_CACHE_STALE) this probably can be 
avoided, at least with current code, but it is generally safe, and 
avoiding parsing would be at most minor optimization.  Further, 
such an optimization might not be trivial to maintain, since cache 
statuses might be added or changed over time.

In general, though, updating r->cache->valid_sec from headers 
processing functions as currently implemented is wrong, and can 
lead to incorrect results when switching to other upstream servers 
per proxy_next_upstream after parsing some headers.  It should be 
either explicitly cleared in ngx_http_upstream_reinit(), or the 
parsed value should be kept in u->headers_in till actually used 
(so it will be cleared by existing generic code in 
ngx_http_upstream_reinit()).

-- 
Maxim Dounin
http://mdounin.ru/

From maksim.yevmenkin at gmail.com  Mon Jul 29 17:00:29 2024
From: maksim.yevmenkin at gmail.com (Maksim Yevmenkin)
Date: Mon, 29 Jul 2024 10:00:29 -0700
Subject: Updating r->cache in ngx_http_upstream_process_cache_control()
In-Reply-To: <ZqRWU-J8fVC9u0Vz@mdounin.ru>
References: <CAFPOs6qnmyQnogOF0bo10YRtLNBtaofm10Q8EZ5oMpxAQr79cQ@mail.gmail.com>
 <ZqRWU-J8fVC9u0Vz@mdounin.ru>
Message-ID: <CAFPOs6qsDmGReFnVrWdPmYB1Cju9e7S78+3g2A_eNZG+vgFYDg@mail.gmail.com>

Hello!

[...]

thank you for the response.

> Parsing cache validity time from cached responses is used at least
> when sending revalidated responses (NGX_HTTP_CACHE_REVALIDATED).
>
> For cached responses with other cache statuses
> (NGX_HTTP_CACHE_HIT, NGX_HTTP_CACHE_STALE) this probably can be
> avoided, at least with current code, but it is generally safe, and
> avoiding parsing would be at most minor optimization.  Further,
> such an optimization might not be trivial to maintain, since cache
> statuses might be added or changed over time.

Just to clarify, I'm not suggesting that parsing is an issue.

My concern is updating r->cache->valid_sec (and other variables) while
serving a cached object. As the cached object is served (HIT),
r->cache->valid_sec is advancing compared to node->valid_sec.

> In general, though, updating r->cache->valid_sec from headers
> processing functions as currently implemented is wrong, and can

Understood, I just wanted to clarify your response about it being
"generally safe."

> lead to incorrect results when switching to other upstream servers
> per proxy_next_upstream after parsing some headers.  It should be
> either explicitly cleared in ngx_http_upstream_reinit(), or the
> parsed value should be kept in u->headers_in till actually used
> (so it will be cleared by existing generic code in
> ngx_http_upstream_reinit()).

Could you please help me understand how ngx_http_upstream_reinit()
would be beneficial in the case of a cache HIT?

thanks,
max

From mdounin at mdounin.ru  Mon Jul 29 23:40:52 2024
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 30 Jul 2024 02:40:52 +0300
Subject: Updating r->cache in ngx_http_upstream_process_cache_control()
In-Reply-To: <CAFPOs6qsDmGReFnVrWdPmYB1Cju9e7S78+3g2A_eNZG+vgFYDg@mail.gmail.com>
References: <CAFPOs6qnmyQnogOF0bo10YRtLNBtaofm10Q8EZ5oMpxAQr79cQ@mail.gmail.com>
 <ZqRWU-J8fVC9u0Vz@mdounin.ru>
 <CAFPOs6qsDmGReFnVrWdPmYB1Cju9e7S78+3g2A_eNZG+vgFYDg@mail.gmail.com>
Message-ID: <ZqgohPtlcG5k0qR8@mdounin.ru>

Hello!

On Mon, Jul 29, 2024 at 10:00:29AM -0700, Maksim Yevmenkin wrote:

> Hello!
> 
> [...]
> 
> thank you for the response.
> 
> > Parsing cache validity time from cached responses is used at least
> > when sending revalidated responses (NGX_HTTP_CACHE_REVALIDATED).
> >
> > For cached responses with other cache statuses
> > (NGX_HTTP_CACHE_HIT, NGX_HTTP_CACHE_STALE) this probably can be
> > avoided, at least with current code, but it is generally safe, and
> > avoiding parsing would be at most minor optimization.  Further,
> > such an optimization might not be trivial to maintain, since cache
> > statuses might be added or changed over time.
> 
> Just to clarify, I'm not suggesting that parsing is an issue.
> 
> My concern is updating r->cache->valid_sec (and other variables) while
> serving a cached object. As the cached object is served (HIT),
> r->cache->valid_sec is advancing compared to node->valid_sec.

Updating r->cache->valid_sec for cached responses is safe, as this 
field is not used (as a cache node validity time) after parsing 
cached response headers.

And for some cached responses (REVALIDATED) it is actually used as 
an updated cached response validity time.

> > In general, though, updating r->cache->valid_sec from headers
> > processing functions as currently implemented is wrong, and can
> 
> Understood, I just wanted to clarify your response about it being
> "generally safe."
> 
> > lead to incorrect results when switching to other upstream servers
> > per proxy_next_upstream after parsing some headers.  It should be
> > either explicitly cleared in ngx_http_upstream_reinit(), or the
> > parsed value should be kept in u->headers_in till actually used
> > (so it will be cleared by existing generic code in
> > ngx_http_upstream_reinit()).
> 
> Could you please help me understand how ngx_http_upstream_reinit()
> would be beneficial in the case of a cache HIT?

The issue with updating r->cache->valid_sec from headers 
processing functions as described in the quoted paragraph is 
unrelated to cached responses, but instead manifests itself when 
receiving responses from upstream servers.

-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Wed Jul 31 15:55:25 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Wed, 31 Jul 2024 18:55:25 +0300
Subject: [nginx-site] Documented Age header handling.
Message-ID: <hg.ca4c8ac67c0e.1722441325.7485189136187354939@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx-site/rev/ca4c8ac67c0e
branches:  
changeset: 3092:ca4c8ac67c0e
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Wed Jul 31 18:33:11 2024 +0300
description:
Documented Age header handling.

diffstat:

 xml/en/docs/http/ngx_http_fastcgi_module.xml  |  12 +++++++++---
 xml/en/docs/http/ngx_http_proxy_module.xml    |  12 +++++++++---
 xml/en/docs/http/ngx_http_scgi_module.xml     |  12 +++++++++---
 xml/en/docs/http/ngx_http_upstream_module.xml |   8 +++++++-
 xml/en/docs/http/ngx_http_uwsgi_module.xml    |  12 +++++++++---
 xml/ru/docs/http/ngx_http_fastcgi_module.xml  |  16 +++++++++++-----
 xml/ru/docs/http/ngx_http_proxy_module.xml    |  16 +++++++++++-----
 xml/ru/docs/http/ngx_http_scgi_module.xml     |  16 +++++++++++-----
 xml/ru/docs/http/ngx_http_upstream_module.xml |   8 +++++++-
 xml/ru/docs/http/ngx_http_uwsgi_module.xml    |  16 +++++++++++-----
 10 files changed, 94 insertions(+), 34 deletions(-)

diffs (410 lines):

diff --git a/xml/en/docs/http/ngx_http_fastcgi_module.xml b/xml/en/docs/http/ngx_http_fastcgi_module.xml
--- a/xml/en/docs/http/ngx_http_fastcgi_module.xml
+++ b/xml/en/docs/http/ngx_http_fastcgi_module.xml
@@ -10,7 +10,7 @@
 <module name="Module ngx_http_fastcgi_module"
         link="/en/docs/http/ngx_http_fastcgi_module.html"
         lang="en"
-        rev="54">
+        rev="55">
 
 <section id="summary">
 
@@ -649,6 +649,12 @@ with another value, such a response will
 taking into account the corresponding request header fields (1.7.7).
 </listitem>
 
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
 </list>
 Processing of one or more of these response header fields can be disabled
 using the <link id="fastcgi_ignore_headers"/> directive.
@@ -766,7 +772,7 @@ The following fields can be ignored: <he
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -777,7 +783,7 @@ effect:
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
 set the parameters of response <link id="fastcgi_cache_valid">caching</link>;
 </listitem>
 
diff --git a/xml/en/docs/http/ngx_http_proxy_module.xml b/xml/en/docs/http/ngx_http_proxy_module.xml
--- a/xml/en/docs/http/ngx_http_proxy_module.xml
+++ b/xml/en/docs/http/ngx_http_proxy_module.xml
@@ -10,7 +10,7 @@
 <module name="Module ngx_http_proxy_module"
         link="/en/docs/http/ngx_http_proxy_module.html"
         lang="en"
-        rev="77">
+        rev="78">
 
 <section id="summary">
 
@@ -667,6 +667,12 @@ with another value, such a response will
 taking into account the corresponding request header fields (1.7.7).
 </listitem>
 
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
 </list>
 Processing of one or more of these response header fields can be disabled
 using the <link id="proxy_ignore_headers"/> directive.
@@ -1011,7 +1017,7 @@ The following fields can be ignored: <he
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -1022,7 +1028,7 @@ effect:
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
 set the parameters of response <link id="proxy_cache_valid">caching</link>;
 </listitem>
 
diff --git a/xml/en/docs/http/ngx_http_scgi_module.xml b/xml/en/docs/http/ngx_http_scgi_module.xml
--- a/xml/en/docs/http/ngx_http_scgi_module.xml
+++ b/xml/en/docs/http/ngx_http_scgi_module.xml
@@ -10,7 +10,7 @@
 <module name="Module ngx_http_scgi_module"
         link="/en/docs/http/ngx_http_scgi_module.html"
         lang="en"
-        rev="42">
+        rev="43">
 
 <section id="summary">
 
@@ -640,6 +640,12 @@ with another value, such a response will
 taking into account the corresponding request header fields (1.7.7).
 </listitem>
 
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
 </list>
 Processing of one or more of these response header fields can be disabled
 using the <link id="scgi_ignore_headers"/> directive.
@@ -731,7 +737,7 @@ The following fields can be ignored: <he
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -742,7 +748,7 @@ effect:
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
 set the parameters of response <link id="scgi_cache_valid">caching</link>;
 </listitem>
 
diff --git a/xml/en/docs/http/ngx_http_upstream_module.xml b/xml/en/docs/http/ngx_http_upstream_module.xml
--- a/xml/en/docs/http/ngx_http_upstream_module.xml
+++ b/xml/en/docs/http/ngx_http_upstream_module.xml
@@ -10,7 +10,7 @@
 <module name="Module ngx_http_upstream_module"
         link="/en/docs/http/ngx_http_upstream_module.html"
         lang="en"
-        rev="90">
+        rev="91">
 
 <section id="summary">
 
@@ -574,6 +574,12 @@ are separated by commas and colons like 
 <link id="var_upstream_addr">$upstream_addr</link> variable.
 </tag-desc>
 
+<tag-name id="var_upstream_cache_age"><var>$upstream_cache_age</var>
+</tag-name>
+<tag-desc>
+age of the cache item (1.27.3).
+</tag-desc>
+
 <tag-name id="var_upstream_cache_key"><var>$upstream_cache_key</var>
 </tag-name>
 <tag-desc>
diff --git a/xml/en/docs/http/ngx_http_uwsgi_module.xml b/xml/en/docs/http/ngx_http_uwsgi_module.xml
--- a/xml/en/docs/http/ngx_http_uwsgi_module.xml
+++ b/xml/en/docs/http/ngx_http_uwsgi_module.xml
@@ -10,7 +10,7 @@
 <module name="Module ngx_http_uwsgi_module"
         link="/en/docs/http/ngx_http_uwsgi_module.html"
         lang="en"
-        rev="51">
+        rev="52">
 
 <section id="summary">
 
@@ -640,6 +640,12 @@ with another value, such a response will
 taking into account the corresponding request header fields (1.7.7).
 </listitem>
 
+<listitem>
+If the header includes the <header>Age</header> field
+and caching time is set with the <header>Cache-Control</header> field,
+caching time is reduced accordingly (1.27.3).
+</listitem>
+
 </list>
 Processing of one or more of these response header fields can be disabled
 using the <link id="uwsgi_ignore_headers"/> directive.
@@ -731,7 +737,7 @@ The following fields can be ignored: <he
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
-and <header>Vary</header> (1.7.7).
+<header>Vary</header> (1.7.7), and <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -742,7 +748,7 @@ effect:
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
 <header>Cache-Control</header>, <header>Set-Cookie</header>,
-and <header>Vary</header>
+<header>Vary</header>, and <header>Age</header>
 set the parameters of response <link id="uwsgi_cache_valid">caching</link>;
 </listitem>
 
diff --git a/xml/ru/docs/http/ngx_http_fastcgi_module.xml b/xml/ru/docs/http/ngx_http_fastcgi_module.xml
--- a/xml/ru/docs/http/ngx_http_fastcgi_module.xml
+++ b/xml/ru/docs/http/ngx_http_fastcgi_module.xml
@@ -10,7 +10,7 @@
 <module name="?????? ngx_http_fastcgi_module"
         link="/ru/docs/http/ngx_http_fastcgi_module.html"
         lang="ru"
-        rev="54">
+        rev="55">
 
 <section id="summary">
 
@@ -649,6 +649,12 @@ fastcgi_cache_valid any      1m;
 ? ?????? ??????????????? ????? ????????? ??????? (1.7.7).
 </listitem>
 
+<listitem>
+???? ? ????????? ?????? ???? ???? <header>Age</header>
+? ????? ??????????? ???????????? ?? ???? <header>Cache-Control</header>,
+????? ??????????? ????? ?????????????? ????????? (1.27.3).
+</listitem>
+
 </list>
 ????????? ?????? ??? ????? ?? ???? ????? ????????? ????? ???? ?????????
 ??? ?????? ????????? <link id="fastcgi_ignore_headers"/>.
@@ -764,8 +770,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 <header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-? <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) ? <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -774,8 +780,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-? <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> ? <header>Age</header>
 ?????? ????????? <link id="fastcgi_cache_valid">???????????</link> ??????;
 </listitem>
 
diff --git a/xml/ru/docs/http/ngx_http_proxy_module.xml b/xml/ru/docs/http/ngx_http_proxy_module.xml
--- a/xml/ru/docs/http/ngx_http_proxy_module.xml
+++ b/xml/ru/docs/http/ngx_http_proxy_module.xml
@@ -10,7 +10,7 @@
 <module name="?????? ngx_http_proxy_module"
         link="/ru/docs/http/ngx_http_proxy_module.html"
         lang="ru"
-        rev="77">
+        rev="78">
 
 <section id="summary">
 
@@ -669,6 +669,12 @@ proxy_cache_valid any      1m;
 ? ?????? ??????????????? ????? ????????? ??????? (1.7.7).
 </listitem>
 
+<listitem>
+???? ? ????????? ?????? ???? ???? <header>Age</header>
+? ????? ??????????? ???????????? ?? ???? <header>Cache-Control</header>,
+????? ??????????? ????? ?????????????? ????????? (1.27.3).
+</listitem>
+
 </list>
 ????????? ?????? ??? ????? ?? ???? ????? ????????? ????? ???? ?????????
 ??? ?????? ????????? <link id="proxy_ignore_headers"/>.
@@ -1010,8 +1016,8 @@ NTLM</link> ????????????? ?????? 1.1.
 <header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-? <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) ? <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -1020,8 +1026,8 @@ NTLM</link> ????????????? ?????? 1.1.
 
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-? <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> ? <header>Age</header>
 ?????? ????????? <link id="proxy_cache_valid">???????????</link> ??????;
 </listitem>
 
diff --git a/xml/ru/docs/http/ngx_http_scgi_module.xml b/xml/ru/docs/http/ngx_http_scgi_module.xml
--- a/xml/ru/docs/http/ngx_http_scgi_module.xml
+++ b/xml/ru/docs/http/ngx_http_scgi_module.xml
@@ -10,7 +10,7 @@
 <module name="?????? ngx_http_scgi_module"
         link="/ru/docs/http/ngx_http_scgi_module.html"
         lang="ru"
-        rev="42">
+        rev="43">
 
 <section id="summary">
 
@@ -640,6 +640,12 @@ scgi_cache_valid any      1m;
 ? ?????? ??????????????? ????? ????????? ??????? (1.7.7).
 </listitem>
 
+<listitem>
+???? ? ????????? ?????? ???? ???? <header>Age</header>
+? ????? ??????????? ???????????? ?? ???? <header>Cache-Control</header>,
+????? ??????????? ????? ?????????????? ????????? (1.27.3).
+</listitem>
+
 </list>
 ????????? ?????? ??? ????? ?? ???? ????? ????????? ????? ???? ?????????
 ??? ?????? ????????? <link id="scgi_ignore_headers"/>.
@@ -729,8 +735,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 <header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-? <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) ? <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -739,8 +745,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-? <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> ? <header>Age</header>
 ?????? ????????? <link id="scgi_cache_valid">???????????</link> ??????;
 </listitem>
 
diff --git a/xml/ru/docs/http/ngx_http_upstream_module.xml b/xml/ru/docs/http/ngx_http_upstream_module.xml
--- a/xml/ru/docs/http/ngx_http_upstream_module.xml
+++ b/xml/ru/docs/http/ngx_http_upstream_module.xml
@@ -10,7 +10,7 @@
 <module name="?????? ngx_http_upstream_module"
         link="/ru/docs/http/ngx_http_upstream_module.html"
         lang="ru"
-        rev="90">
+        rev="91">
 
 <section id="summary">
 
@@ -581,6 +581,12 @@ server {
 <link id="var_upstream_addr">$upstream_addr</link>.
 </tag-desc>
 
+<tag-name id="var_upstream_cache_age"><var>$upstream_cache_age</var>
+</tag-name>
+<tag-desc>
+??????? ???????? ???? (1.27.3).
+</tag-desc>
+
 <tag-name id="var_upstream_cache_key"><var>$upstream_cache_key</var>
 </tag-name>
 <tag-desc>
diff --git a/xml/ru/docs/http/ngx_http_uwsgi_module.xml b/xml/ru/docs/http/ngx_http_uwsgi_module.xml
--- a/xml/ru/docs/http/ngx_http_uwsgi_module.xml
+++ b/xml/ru/docs/http/ngx_http_uwsgi_module.xml
@@ -10,7 +10,7 @@
 <module name="?????? ngx_http_uwsgi_module"
         link="/ru/docs/http/ngx_http_uwsgi_module.html"
         lang="ru"
-        rev="51">
+        rev="52">
 
 <section id="summary">
 
@@ -639,6 +639,12 @@ uwsgi_cache_valid any      1m;
 ? ?????? ??????????????? ????? ????????? ??????? (1.7.7).
 </listitem>
 
+<listitem>
+???? ? ????????? ?????? ???? ???? <header>Age</header>
+? ????? ??????????? ???????????? ?? ???? <header>Cache-Control</header>,
+????? ??????????? ????? ?????????????? ????????? (1.27.3).
+</listitem>
+
 </list>
 ????????? ?????? ??? ????? ?? ???? ????? ????????? ????? ???? ?????????
 ??? ?????? ????????? <link id="uwsgi_ignore_headers"/>.
@@ -728,8 +734,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 <header>X-Accel-Expires</header>, <header>X-Accel-Limit-Rate</header> (1.1.6),
 <header>X-Accel-Buffering</header> (1.1.6),
 <header>X-Accel-Charset</header> (1.1.6), <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44)
-? <header>Vary</header> (1.7.7).
+<header>Cache-Control</header>, <header>Set-Cookie</header> (0.8.44),
+<header>Vary</header> (1.7.7) ? <header>Age</header> (1.27.3).
 </para>
 
 <para>
@@ -738,8 +744,8 @@ nginx ?? ???????? ??????? ???? ????????? <header>Status</header> ?
 
 <listitem>
 <header>X-Accel-Expires</header>, <header>Expires</header>,
-<header>Cache-Control</header>, <header>Set-Cookie</header>
-? <header>Vary</header>
+<header>Cache-Control</header>, <header>Set-Cookie</header>,
+<header>Vary</header> ? <header>Age</header>
 ?????? ????????? <link id="uwsgi_cache_valid">???????????</link> ??????;
 </listitem>
 

From mdounin at mdounin.ru  Wed Jul 31 16:18:00 2024
From: mdounin at mdounin.ru (=?iso-8859-1?q?Maxim_Dounin?=)
Date: Wed, 31 Jul 2024 19:18:00 +0300
Subject: [nginx] Keepalive connections now respect lingering_timeout on s...
Message-ID: <hg.3c71158f5a34.1722442680.3576798278264758257@ns3.mdounin.ru>

details:   http://freenginx.org/hg/nginx/rev/3c71158f5a34
branches:  
changeset: 9307:3c71158f5a34
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Wed Jul 31 17:50:58 2024 +0300
description:
Keepalive connections now respect lingering_timeout on shutdown.

During graceful shutdown keepalive connections are now closed only after
at least lingering_timeout of inactivity.  To do so, c->idle is only
set on keepalive connections after lingering_timeout expires.

This ensures that the connection close race will less likely result in
connections being reset when a client sends a request at the same time
when the connection is closed by ngx_close_idle_connections() during
graceful shutdown.

diffstat:

 src/http/ngx_http_request.c |  40 +++++++++++++++++++++++++++++++++-------
 src/http/ngx_http_request.h |   2 ++
 2 files changed, 35 insertions(+), 7 deletions(-)

diffs (83 lines):

diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c
+++ b/src/http/ngx_http_request.c
@@ -3099,6 +3099,7 @@ ngx_http_set_keepalive(ngx_http_request_
 {
     int                        tcp_nodelay;
     ngx_buf_t                 *b, *f;
+    ngx_msec_t                 timer;
     ngx_chain_t               *cl, *ln;
     ngx_event_t               *rev, *wev;
     ngx_connection_t          *c;
@@ -3299,10 +3300,19 @@ ngx_http_set_keepalive(ngx_http_request_
     r->http_state = NGX_HTTP_KEEPALIVE_STATE;
 #endif
 
-    c->idle = 1;
     ngx_reusable_connection(c, 1);
 
-    ngx_add_timer(rev, clcf->keepalive_timeout);
+    if (clcf->lingering_close
+        && clcf->lingering_timeout > 0)
+    {
+        timer = ngx_min(clcf->keepalive_timeout, clcf->lingering_timeout);
+        hc->keepalive_timeout = clcf->keepalive_timeout - timer;
+        ngx_add_timer(rev, timer);
+
+    } else {
+        c->idle = 1;
+        ngx_add_timer(rev, clcf->keepalive_timeout);
+    }
 
     if (rev->ready) {
         ngx_post_event(rev, &ngx_posted_events);
@@ -3313,16 +3323,32 @@ ngx_http_set_keepalive(ngx_http_request_
 static void
 ngx_http_keepalive_handler(ngx_event_t *rev)
 {
-    size_t             size;
-    ssize_t            n;
-    ngx_buf_t         *b;
-    ngx_connection_t  *c;
+    size_t                  size;
+    ssize_t                 n;
+    ngx_buf_t              *b;
+    ngx_connection_t       *c;
+    ngx_http_connection_t  *hc;
 
     c = rev->data;
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http keepalive handler");
 
-    if (rev->timedout || c->close) {
+    if (rev->timedout) {
+        if (c->idle || ngx_exiting) {
+            ngx_http_close_connection(c);
+            return;
+        }
+
+        hc = c->data;
+
+        c->idle = 1;
+        rev->timedout = 0;
+        ngx_add_timer(rev, hc->keepalive_timeout);
+
+        return;
+    }
+
+    if (c->close) {
         ngx_http_close_connection(c);
         return;
     }
diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -332,6 +332,8 @@ typedef struct {
 
     ngx_chain_t                      *free;
 
+    ngx_msec_t                        keepalive_timeout;
+
     unsigned                          ssl:1;
     unsigned                          proxy_protocol:1;
 } ngx_http_connection_t;