[nginx] SSL: compatibility with OpenSSL 4.0 error handling.

[Maxim Dounin]

details:   http://freenginx.org/hg/nginx/rev/ff5becaf5444
branches:  
changeset: 9486:ff5becaf5444
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Sun Mar 22 16:27:12 2026 +0300
description:
SSL: compatibility with OpenSSL 4.0 error handling.

As of OpenSSL 4.0 alpha 1, errors during reading are remembered in the
SSL connection structure, and further attempts to write to the connections
are rejected with SSL_ERROR_SSL error and no additional details.

While rejecting such attempts is probably correct, lack of the additional
error details makes it hard to figure out what actually happened, and
to do appropriate logging.

In particular, "[crit] ... SSL_write() failed" errors were observed in
the ssl_stapling.t test, where the socket is closed right after sending
the request, leading to RST sent with TLSv1.3 in response to the tickets
sent after the handshake, and often observed by the server while reading
the request (but not yet processed).

To make sure such errors are not reported as "[crit] ... SSL_write() failed",
we now don't try to call SSL_write() after an error was detected by
ngx_ssl_recv().

diffstat:

 src/event/ngx_event_openssl.c |  12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diffs (29 lines):

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -3378,6 +3378,12 @@ ngx_ssl_write(ngx_connection_t *c, u_cha
     }
 #endif
 
+    if (c->ssl->last == NGX_ERROR) {
+        c->write->ready = 0;
+        c->write->error = 1;
+        return NGX_ERROR;
+    }
+
     ngx_ssl_clear_error(c->log);
 
     ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL to write: %uz", size);
@@ -3490,6 +3496,12 @@ ngx_ssl_write_early(ngx_connection_t *c,
     size_t     written;
     ngx_err_t  err;
 
+    if (c->ssl->last == NGX_ERROR) {
+        c->write->ready = 0;
+        c->write->error = 1;
+        return NGX_ERROR;
+    }
+
     ngx_ssl_clear_error(c->log);
 
     ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL to write: %uz", size);