[PATCH 5 of 5] SSL: compatibility with OpenSSL 4.0 error handling

[Maxim Dounin]

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1773536056 -10800
#      Sun Mar 15 03:54:16 2026 +0300
# Node ID fa4e9722724a24462c5162c5ab0a081beb50d787
# Parent  e13d7e2a2a80a32bffd14e444edc05039b35c066
SSL: compatibility with OpenSSL 4.0 error handling.

As of OpenSSL 4.0 alpha 1, errors during reading are remembered in the
SSL connection structure, and further attempts to write to the connections
are rejected with SSL_ERROR_SSL error and no additional details.

While rejecting such attempts is probably correct, lack of the additional
error details makes it hard to figure out what actually happened, and
to do appropriate logging.

In particular, "[crit] ... SSL_write() failed" errors were observed in
the ssl_stapling.t test, where the socket is closed right after sending
the request, leading to RST sent with TLSv1.3 in response to the tickets
sent after the handshake, and often observed by the server while reading
the request (but not yet processed).

To make sure such errors are not reported as "[crit] ... SSL_write() failed",
we now don't try to call SSL_write() after an error was detected by
ngx_ssl_recv().

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -3378,6 +3378,12 @@ ngx_ssl_write(ngx_connection_t *c, u_cha
     }
 #endif
 
+    if (c->ssl->last == NGX_ERROR) {
+        c->write->ready = 0;
+        c->write->error = 1;
+        return NGX_ERROR;
+    }
+
     ngx_ssl_clear_error(c->log);
 
     ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL to write: %uz", size);
@@ -3490,6 +3496,12 @@ ngx_ssl_write_early(ngx_connection_t *c,
     size_t     written;
     ngx_err_t  err;
 
+    if (c->ssl->last == NGX_ERROR) {
+        c->write->ready = 0;
+        c->write->error = 1;
+        return NGX_ERROR;
+    }
+
     ngx_ssl_clear_error(c->log);
 
     ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL to write: %uz", size);