Instead of testing if a conn-stream exists or not, we rely on CS_EP_ORPHAN
endpoint flag. In addition, if possible, we access the endpoint from the
mux_pt context. Finally, the endpoint flags are now reported in trace
messages.
All old flags CS_FL_* are now moved in the endpoint scope and renamed
CS_EP_* accordingly. It is a systematic replacement. There is no true change
except for the health-check and the endpoint reset. Here it is a bit special
because the same conn-stream is reused. Thus, we must handle endpoint
allocation errors. To do so, cs_reset_endp() has been adapted.
Thanks to this last change, it will now be possible to simplify the
multiplexer and probably the applets too. A review must also be performed to
remove some flags in the channel or the stream-interface. The HTX will
probably be simplified too. Finally, there is now some place in the
conn-stream to move info from the stream-interface.
The conn-stream endpoint is now shared between the conn-stream and the
applet or the multiplexer. If the mux or the applet is created first, it is
responsible to also create the endpoint and share it with the conn-stream.
If the conn-stream is created first, it is the opposite.
When the endpoint is only owned by an applet or a mux, it is called an
orphan endpoint (there is no conn-stream). When it is only owned by a
conn-stream, it is called a detached endpoint (there is no mux/applet).
The last entity that owns an endpoint is responsible to release it. When a
mux or an applet is detached from a conn-stream, the conn-stream
relinquishes the endpoint to recreate a new one. This way, the endpoint
state is never lost for the mux or the applet.
It is a transient commit to prepare next changes. Now, when a conn-stream is
created from an applet or a multiplexer, an endpoint is always provided. In
addition, the API to create a conn-stream was specialized to have one
function per type.
The next step will be to share the endpoint structure.
It is a transient commit to prepare next changes. It is possible to pass a
pre-allocated endpoint to create a new conn-stream. If it is NULL, a new
endpoint is created, otherwise the existing one is used. There no more
change at the conn-stream level.
In the applets, all conn-stream are created with no pre-allocated
endpoint. But for multiplexers, an endpoint is systematically created before
creating the conn-stream.
Some CS flags, only related to the endpoint, are moved into the endpoint
struct. More will probably moved later. Those ones are not critical. So it
is pretty safe to move them now and this will ease next changes.
Group the endpoint target of a conn-stream, its context and the associated
flags in a dedicated structure in the conn-stream. It is not inlined in the
conn-stream structure. There is a dedicated pool.
For now, there is no complexity. It is just an indirection to get the
endpoint or its context. But the purpose of this structure is to be able to
share a refcounted context between the mux and the conn-stream. This way, it
will be possible to preserve it when the mux is detached from the
conn-stream.
The function cs_init() is only called by cs_new(). The conn-stream
initialization will be reviewed. It is easier to do it in cs_new() instead
of using a dedicated function. cs_new() is pretty simple, there is no reason
to split the code in this case.
This change is only significant for the multiplexer part. For the applets,
the context and the endpoint are the same. Thus, there is no much change. For
the multiplexer part, the connection was used to set the conn-stream
endpoint and the mux's stream was the context. But it is a bit strange
because once a mux is installed, it takes over the connection. In a
wonderful world, the connection should be totally hidden behind the mux. The
stream-interface and, in a lesser extent, the stream, still access the
connection because that was inherited from the pre-multiplexer era.
Now, the conn-stream endpoint is the mux's stream (an opaque entity for the
conn-stream) and the connection is the context. Dedicated functions have
been added to attached an applet or a mux to a conn-stream.
The appctx owner is now always a conn-stream. Thus, it can be set during the
appctx allocation. But, to do so, the conn-stream must be created first. It
is not a problem on the server side because the conn-stream is created with
the stream. On the client side, we must take care to create the conn-stream
first.
This change should ease other changes about the applets bootstrapping.
This patch is mandatory to invert the endpoint and the context in the
conn-stream. There is no common type (at least for now) for the entity
representing a mux (h1s, h2s...), thus we must set its type when the
endpoint is attached to a conn-stream. There is 2 types for the conn-stream
endpoints: the mux (CS_FL_ENDP_MUX) and the applet (CS_FL_ENDP_APP).
For now there is no much change. Only the appctx is passed as argument when
the .init callback function is called. And it is not possible to yield at
this stage. It is not a problem because the feature is not used. Only the
lua defines this callback function for the lua TCP/HTTP services. The idea
is to be able to use it for all applets to initialize the appctx context.
cs_free() must not be called when we fail to allocate the conn-stream in
h1s_new_cs() function. This bug was introduced by the commit cda94accb
("MAJOR: stream/conn_stream: Move the stream-interface into the
conn-stream").
It is 2.6-specific, no backport is needed.
It was mentioned in issue #12 that expired entries would appear with a
negative expire delay in "show cache". Instead of listing them, let's
just evict them.
This could be backported to all versions since this was reported on
1.8 already.
It was reported in issue #13 that a GOAWAY frame was sent on timeout even
if no SETTINGS frame was sent. The approach imagined by then was to track
the fact that a SETTINGS frame was already sent to avoid this, but that's
already what is done through the state, though it doesn't stand due to the
fact that we switch the frame to the error state. Thus instead what we're
doing here is to instead set the GOAWAY_FAILED flag in h2c_error() before
switching to the ERROR state when the state indicates we've not yet sent
settings, and refrain from sending anything from the h2c_send_goaway_error()
function for such states.
This could be backported to all versions where it applies well.
qcs by_id field has been replaced by a new field named "id". Adjust the
h3_debug_printf traces. This is the case since the introduction of the
qc_stream_desc type.
In issue #1184, cppcheck reports that an incorrect format "%d" was
used to print an unsigned in the debug code, though values are always
very small and this will never be an issue.
In issue #1184, cppcheck complains about some inconsistent printf
formats. At least the one in peer_prepare_hellomsg() that uses "%u"
for the int "min_ver" is wrong. Let's force other types to make it
happy, though constants cannot cause trouble.
cppcheck reports in issue #1184 a type mismatch between "%d" and the
unsigned int "misses" in the standalone debug code of lru.c. Let's
switch to "%u".
We used to check if the transport layer was ssl_sock to decide to log
"~" after a frontend's name. Now that QUIC is present, this doesn't work
anymore. Better rely on the transport layer's get_ssl_sock_ctx() method.
Coverity found in issue #1646 that I added a double-close bug in last
commit e4d09cedb ("MINOR: sock: check configured limits at the sock layer,
not the listener's") because the error path already closes the FD. No
backport needed.
In issue #1645, coverity suspects some dead code due to a pair of
remaining tests on "if (!ctx)". While all other functions test the
context earlier, these ones used to only test the connection and the
transport. It's still not very clear to me if there are certain error
cases that can lead to no SSL being initially set while the rest is
ready, and the SSL arriving later, but better preserve this original
construct by testing first the connection and only later the context.
First gcc, then now coverity report possible null derefs in situations
where we know these cannot happen since we call the functions in
contexts that guarantee the existence of the connection and the method
used. Let's introduce an unchecked version of the function for such
cases, just like we had to do with objt_*. This allows us to remove the
ALREADY_CHECKED() statements (which coverity doesn't see), and addresses
github issues #1643, #1644, #1647.
Some compilers see a possible null deref after conn_get_ssl_sock_ctx()
in ssl_sock_parse_heartbeat, which cannot happen there, so let's mark
it as safe. No backport needed.
It was supposed to be there, and probably was not placed there due to
historic limitations in listener_accept(), but now there does not seem
to be a remaining valid reason for keeping the quic_conn out of the
handle. In addition in new_quic_cli_conn() the handle->fd was incorrectly
set to the listener's FD.
By being able to return the ssl_sock_ctx, we're now enabling the whole
set of SSL sample fetch methods to work on the current SSL context of
the QUIC connection, as seen in the following test showing a request
forwarded to an HTTP/1 server with plenty of SSL headers filled:
00000001:decrypt.clireq[000f:ffffffff]: GET / HTTP/1.1
00000001:decrypt.clihdr[000f:ffffffff]: host: localhost
00000001:decrypt.clihdr[000f:ffffffff]: user-agent: nghttp3/ngtcp2 client
00000001:decrypt.clihdr[000f:ffffffff]: x-src: 127.0.0.1
00000001:decrypt.clihdr[000f:ffffffff]: x-dst: 127.0.0.4
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_f_serial: D16197E7D3E634E9
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_f_key_alg: rsaEncryption
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_f_sig_alg: RSA-SHA1
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_fc: 1
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_fc_has_sni: 1
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_fc_sni: blah
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_fc_alpn: h3
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_fc_protocol: TLSv1.3
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_fc_cipher: TLS_AES_256_GCM_SHA384
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_fc_alg_keysize: 256
00000001:decrypt.clihdr[000f:ffffffff]: x-ssl_fc_use_keysize: 256
00000001:decrypt.clihdr[000f:ffffffff]: x-forwarded-for: 127.0.0.1
The code is trivial, but this is marked as medium as there's always
the risk that some of the callable functions do not like being called
on such SSL contexts.
The SSL functions must not use conn->xprt_ctx anymore but find the context
by calling conn_get_ssl_sock_ctx(), which will properly pass through the
transport layers to retrieve the desired information. Otherwise when the
functions are called on a QUIC connection, they refuse to work for not
being called on the proper transport.
Historically there was a single way to have an SSL transport on a
connection, so detecting if the transport layer was SSL and a context
was present was sufficient to detect SSL. With QUIC, things have changed
because QUIC also relies on SSL, but the context is embedded inside the
quic_conn and the transport layer doesn't match expectations outside,
making it difficult to detect that SSL is in use over the connection.
The approach taken here to improve this consists in adding a new method
at the transport layer, get_ssl_sock_ctx(), to retrieve this often needed
ssl_sock_ctx, and to use this to detect the presence of SSL. This will
even allow some simplifications and cleanups to be made in the SSL code
itself, and QUIC will be able to provide one to export its ssl_sock_ctx.
These functions will allow the connection layer to retrieve a quic_conn's
source or destination when possible. The quic_conn holds the peer's address
but not the local one, and the sockets API doesn't always makes that easy
for datagrams. Thus for frontend connection what we're doing here is to
retrieve the listener's address when the destination address is desired.
Now it finally becomes possible to fetch the source and destination using
"src" and "dst", and to pass an incoming connection's endpoints via the
proxy protocol.
The mux didn't have its flags nor name set, as seen in this output of
"haproxy -vv":
Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
quic : mode=HTTP side=FE mux= flags=
h2 : mode=HTTP side=FE|BE mux=H2 flags=HTX|CLEAN_ABRT|HOL_RISK|NO_UPG
This might have random impacts at certain points like forcing some
connections to close instead of aborting a stream, or not always
handling certain streams as fully HTX-compliant.
Certain functions cannot be called on an FD-less conn because they are
normally called as part of the protocol-specific setup/teardown sequence.
Better place a few BUG_ON() to make sure none of them is called in other
situations. If any of them would trigger in ambiguous conditions, it would
always be possible to replace it with an error.
Some syscalls at the TCP level act directly on the FD. Some of them
are used by TCP actions like set-tos, set-mark, silent-drop, others
try to retrieve TCP info, get the source or destination address. These
ones must not be called with an invalid FD coming from an FD-less
connection, so let's add the relevant tests for this. It's worth
noting that all these ones already have fall back plans (do nothing,
error, or switch to alternate implementation).
QUIC connections do not use a file descriptor, instead they use the
quic equivalent which is the quic_conn. A number of our historical
functions at the connection level continue to unconditionally touch
the file descriptor and this may have consequences once QUIC starts
to be used.
This patch adds a new flag on QUIC connections, CO_FL_FDLESS, to
mention that the connection doesn't have a file descriptor, hence the
FD-based API must never be used on them.
From now on it will be possible to intrument existing functions to
panic when this flag is present.
listener_accept() used to continue to enforce the FD limits relative to
global.maxsock by itself while it's the last FD-specific test in the
whole file. This test has nothing to do there, it ought to be placed in
sock_accept_conn() which is the one in charge of FD allocation and tests.
Similar tests are already located there by the way. The only tiny
difference is that listener_accept() used to pause for one second when
this limit was reached, while other similar conditions were pausing only
100ms, so now the same 100ms will apply. But that's not important and
could even be considered as an improvement.
OpenSSL 3.0 warns that ERR_func_error_string() is deprecated. Using
ERR_peek_error_func() solves it instead, and this function was added to
the compat layer by commit 1effd9aa0 ("MINOR: ssl: Remove call to
ERR_func_error_string with OpenSSLv3").
The OpenSSL engine API is deprecated starting with OpenSSL 3.0.
In order to have a clean build this feature is now disabled by default.
It can be reactivated with USE_ENGINE=1 on the build line.
Shawn Heisey reported that the proxy's description was unreadable in dark
color scheme. This is because the text color is changed in the table but
not the cell's background.
This should be backported to 2.5.
In "haproxy -vv" we produce a list of available muxes with their
capabilities, but that list is often quite large for terminals due
to excess of spaces, so let's reduce them a bit to make the output
more readable.
The new 'close-spread-time' global option can be used to spread idle and
active HTTP connction closing after a SIGUSR1 signal is received. This
allows to limit bursts of reconnections when too many idle connections
are closed at once. Indeed, without this new mechanism, in case of
soft-stop, all the idle connections would be closed at once (after the
grace period is over), and all active HTTP connections would be closed
by appending a "Connection: close" header to the next response that goes
over it (or via a GOAWAY frame in case of HTTP2).
This patch adds the support of this new option for HTTP as well as HTTP2
connections. It works differently on active and idle connections.
On active connections, instead of sending systematically the GOAWAY
frame or adding the 'Connection: close' header like before once the
soft-stop has started, a random based on the remainder of the close
window is calculated, and depending on its result we could decide to
keep the connection alive. The random will be recalculated for any
subsequent request/response on this connection so the GOAWAY will still
end up being sent, but we might wait a few more round trips. This will
ensure that goaways are distributed along a longer time window than
before.
On idle connections, a random factor is used when determining the expire
field of the connection's task, which should naturally spread connection
closings on the time window (see h2c_update_timeout).
This feature request was described in GitHub issue #1614.
This patch should be backported to 2.5. It depends on "BUG/MEDIUM:
mux-h2: make use of http-request and keep-alive timeouts" which
refactorized the timeout management of HTTP2 connections.
We modify the key update feature implementation to support reusable cipher contexts
as this is done for the other cipher contexts for packet decryption and encryption.
To do so we attach a context to the quic_tls_kp struct and initialize it each time
the underlying secret key is updated. Same thing when we rotate the secrets keys,
we rotate the contexts as the same time.
These settings are potentially cancelled by others setting initialization shared
with SSL sock bindings. This will have to be clarified when we will adapt the
QUIC bindings configuration.
Add ->ctx new member field to quic_tls_secrets struct to store the cipher context
for each QUIC TLS context TX/RX parts.
Add quic_tls_rx_ctx_init() and quic_tls_tx_ctx_init() functions to initialize
these cipher context for RX and TX parts respectively.
Make qc_new_isecs() call these two functions to initialize the cipher contexts
of the Initial secrets. Same thing for ha_quic_set_encryption_secrets() to
initialize the cipher contexts of the subsequent derived secrets (ORTT, Handshake,
1RTT).
Modify quic_tls_decrypt() and quic_tls_encrypt() to always use the same cipher
context without allocating it each time they are called.
When a QUIC connection is accepted, the wrong field is set from the
client's source address, it's the destination instead of the source.
No backport needed.
