upstream/haproxy
1
0
Fork 0
mirror of https://github.com/haproxy/haproxy.git synced 2026-04-04 16:46:45 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
haproxy/src
History
Willy Tarreau ba4c7a1597 BUG/MEDIUM: sched: allow a bit more TASK_HEAVY to be processed when needed 
As reported in github issue #1881, there are situations where an excess
of TLS handshakes can cause a livelock. What's happening is that normally
we process at most one TLS handshake per loop iteration to maintain the
latency low. This is done by tagging them with TASK_HEAVY, queuing these
tasklets in the TL_HEAVY queue. But if something slows down the loop, such
as a connect() call when no more ports are available, we could end up
processing no more than a few hundred or thousands handshakes per second.

If the llmit becomes lower than the rate of incoming handshakes, we will
accumulate them and at some point users will get impatient and give up or
retry. Then a new problem happens: the queue fills up with even more
handshake attempts, only one of which will be handled per iteration, so
we can end up processing only outdated handshakes at a low rate, with
basically nothing else in the queue. This can for example happen in
parallel with health checks that don't require incoming handshakes to
succeed to continue to cause some activity that could maintain the high
latency stuff active.

Here we're taking a slightly different approach. First, instead of always
allowing only one handshake per loop (and usually it's critical for
latency), we take the current situation into account:
  - if configured with tune.sched.low-latency, the limit remains 1
  - if there are other non-heavy tasks, we set the limit to 1 + one
    per 1024 tasks, so that a heavily loaded queue of 4k handshakes
    per thread will be able to drain them at ~4 per loops with a
    limited impact on latency
  - if there are no other tasks, the limit grows to 1 + one per 128
    tasks, so that a heavily loaded queue of 4k handshakes per thread
    will be able to drain them at ~32 per loop with still a very
    limited impact on latency since only I/O will get delayed.

It was verified on a 56-core Xeon-8480 that this did not degrade the
latency; all requests remained below 1ms end-to-end in full close+
handshake, and even 500us under low-lat + busy-polling.

This must be backported to 2.4.
2023-02-17 16:01:34 +01:00
..
acl.c BUILD: acl: use __fallthrough in parse_acl_expr() 2022-11-14 11:14:02 +01:00
action.c BUG/MINOR: rules: Fix check_capture() function to use the right rule arguments 2022-04-25 15:28:21 +02:00
activity.c BUG/MINOR: clock: do not mix wall-clock and monotonic time in uptime calculation 2023-02-08 11:06:55 +01:00
applet.c MEDIUM: channel: Use CF_WRITE_EVENT instead of CF_WRITE_PARTIAL 2023-01-09 18:41:08 +01:00
arg.c CLEANUP: arg: remove extra check in make_arg_list arg escaping 2022-11-22 16:27:52 +01:00
auth.c MINOR: auth: silence null dereference warning in check_user() 2022-11-24 15:24:02 +01:00
backend.c MEDIUM: channel: Use CF_READ_EVENT instead of CF_READ_PARTIAL 2023-01-09 18:41:08 +01:00
base64.c
cache.c BUG/MEDIUM: cache: use the correct time reference when comparing dates 2023-02-08 11:10:33 +01:00
calltrace.c BUILD: calltrace: fix wrong include when building with TRACE=1 2022-04-19 08:23:30 +02:00
cbuf.c CLEANUP: pool/tree-wide: remove suffix "_pool" from certain pool names 2022-06-23 11:49:09 +02:00
cfgcond.c
cfgdiag.c
cfgparse-global.c MINOR: global: Add an option to disable the data fast-forward 2023-02-17 10:17:02 +01:00
cfgparse-listen.c MEDIUM: proxy/http_ext: implement dynamic http_ext 2023-01-27 15:18:59 +01:00
cfgparse-quic.c MINOR: quic: add config for retransmit limit 2023-02-03 11:56:46 +01:00
cfgparse-ssl.c MINOR: ssl: rename confusing ssl_bind_kws 2023-02-16 16:03:45 +01:00
cfgparse-tcp.c MINOR: listener: move TCP_FO to bind_conf 2023-02-03 18:00:20 +01:00
cfgparse-unix.c CLEANUP: tree-wide: remove 25 occurrences of unneeded fcntl.h 2022-04-26 10:59:48 +02:00
cfgparse.c BUG/MINOR: config: crt-list keywords mistaken for bind ssl keywords 2023-02-16 16:14:37 +01:00
channel.c MEDIUM: channel: Use CF_READ_EVENT instead of CF_READ_PARTIAL 2023-01-09 18:41:08 +01:00
check.c MINOR: check: use atomic for s->consecutive_errors 2022-12-07 17:04:08 +01:00
chunk.c MINOR: chunk: inline alloc_trash_chunk() 2022-08-17 10:45:22 +02:00
cli.c MEDIUM: listener/config: make the "thread" parser rely on thread_sets 2023-02-03 18:00:21 +01:00
clock.c MEDIUM: clock: force internal time to wrap early after boot 2023-02-08 11:10:33 +01:00
compression.c CLEANUP: compression: move the default setting of maxzlibmem to defaults 2022-04-25 19:42:43 +02:00
connection.c MINOR: connection: add a BUG_ON() to detect destroying connection in idle list 2023-01-25 15:30:49 +01:00
cpuset.c MEDIUM: cpu-map: replace the process number with the thread group number 2022-07-15 19:43:10 +02:00
debug.c BUG/MEDIUM: debug/thread: make the debug handler not wait for !rdv_requests 2023-01-19 19:22:17 +01:00
dgram.c
dict.c
dns.c MINOR: channel/applets: Stop to test CF_WRITE_ERROR flag if CF_SHUTW is enough 2023-01-09 18:41:08 +01:00
dynbuf.c
eb32sctree.c
eb32tree.c
eb64tree.c
ebimtree.c
ebistree.c
ebmbtree.c
ebpttree.c
ebsttree.c
ebtree.c
errors.c CLEANUP: assorted typo fixes in the code and comments 2022-10-30 17:17:56 +01:00
ev_epoll.c MINOR: pollers: only update the local date during busy polling 2022-09-21 09:06:28 +02:00
ev_evports.c MINOR: pollers: only update the local date during busy polling 2022-09-21 09:06:28 +02:00
ev_kqueue.c MINOR: pollers: only update the local date during busy polling 2022-09-21 09:06:28 +02:00
ev_poll.c BUG/MINOR: signals/poller: ensure wakeup from signals 2022-09-09 11:15:22 +02:00
ev_select.c MEDIUM: fd/poller: turn update_mask to group-local IDs 2022-07-15 20:16:30 +02:00
event_hdl.c MINOR: server/event_hdl: add support for SERVER_UP and SERVER_DOWN events 2022-12-06 10:22:07 +01:00
extcheck.c MINOR: checks: use the lighter PRNG for spread checks 2022-10-12 21:49:30 +02:00
fcgi-app.c BUG/MINOR: fcgi-app: prevent 'use-fcgi-app' in default section 2023-01-27 15:18:59 +01:00
fcgi.c BUG/MAJOR: fcgi: Fix uninitialized reserved bytes 2022-12-09 12:23:14 +01:00
fd.c BUG/MINOR: thread: always reload threads_enabled in loops 2023-01-19 19:22:17 +01:00
filters.c MEDIUM: listener: move the analysers mask to the bind_conf 2023-02-03 18:00:20 +01:00
fix.c
flt_bwlim.c MINOR: bwlim: Remove useless test on CF_READ_ERROR to detect the last packet 2023-02-13 09:43:38 +01:00
flt_http_comp.c BUILD: compression: use __fallthrough in comp_http_payload() 2022-11-14 11:14:02 +01:00
flt_spoe.c BUG/MEDIUM: spoe: Don't set the default traget for the SPOE agent frontend 2023-02-13 11:37:27 +01:00
flt_trace.c CLEANUP: Apply ist.cocci 2021-11-08 12:08:26 +01:00
freq_ctr.c BUG/MEDIUM: freq-ctr: Don't compute overshoot value for empty counters 2022-12-14 10:44:17 +01:00
frontend.c REORG: rename cs_utils.h to sc_strm.h 2022-05-27 19:33:35 +02:00
h1.c BUG/CRITICAL: http: properly reject empty http header field names 2023-02-14 08:48:54 +01:00
h1_htx.c MINOR: htx: Add an HTX value for the extra field is payload length is unknown 2023-01-13 11:51:11 +01:00
h2.c MINOR: http: extract content-length parsing from H2 2022-12-14 11:34:18 +01:00
h3.c BUG/MINOR: h3: fix crash due to h3 traces 2023-01-31 16:09:47 +01:00
h3_stats.c MINOR: h3: Add a statistics module for h3 2022-05-30 09:59:26 +02:00
haproxy.c MINOR: haproxy: Add an command option to disable data fast-forward 2023-02-17 10:17:02 +01:00
hash.c BUILD: hash: use __fallthrough in hash_djb2() 2022-11-14 11:14:02 +01:00
hlua.c MINOR: htx: Add an HTX value for the extra field is payload length is unknown 2023-01-13 11:51:11 +01:00
hlua_fcn.c MEDIUM: stick-table: switch the table lock to rwlock 2022-10-12 14:19:05 +02:00
hpack-dec.c BUG/CRITICAL: http: properly reject empty http header field names 2023-02-14 08:48:54 +01:00
hpack-enc.c
hpack-huff.c BUG/MEDIUM: hpack: fix incorrect huffman decoding of some control chars 2023-01-26 11:36:39 +01:00
hpack-tbl.c BUILD/DEBUG: hpack-tbl: fix format string in standalone debug code 2022-04-12 08:30:08 +02:00
hq_interop.c MINOR: mux-quic: refactor snd_buf 2022-09-20 15:35:29 +02:00
http.c MINOR: http: extract content-length parsing from H2 2022-12-14 11:34:18 +01:00
http_acl.c
http_act.c BUG/MINOR: http: Memory leak of http redirect rules' format string 2023-01-06 16:42:24 +01:00
http_ana.c MINOR: global: Add an option to disable the data fast-forward 2023-02-17 10:17:02 +01:00
http_client.c BUG/MINOR: httpclient/log: free of invalid ptr with httpclient_log_format 2022-12-22 15:39:31 +01:00
http_conv.c BUG/MEDIUM: http-conv: Fix url_enc() to not crush const samples 2022-04-08 10:12:59 +02:00
http_ext.c BUG/MINOR: http_ext/7239: ipv6 dumping relies on out of scope variables 2023-01-30 15:14:08 +01:00
http_fetch.c MINOR: htx: Add an HTX value for the extra field is payload length is unknown 2023-01-13 11:51:11 +01:00
http_htx.c MINOR: http_htx: add http_prepend_header() to prepend value to header 2023-01-27 15:18:59 +01:00
http_rules.c BUILD: http: remove the two unused constructors in rules and ana 2022-04-25 19:26:26 +02:00
htx.c OPTIM: htx: inline the most common memcpy(8) 2023-02-03 13:39:18 +01:00
init.c MINOR: init: add the pre-check callback 2022-04-22 15:45:47 +02:00
jwt.c BUG/MINOR: jwt: Wrong return value checked 2023-01-20 10:27:37 +01:00
lb_chash.c
lb_fas.c
lb_fwlc.c
lb_fwrr.c
lb_map.c
listener.c MEDIUM: listener/config: make the "thread" parser rely on thread_sets 2023-02-03 18:00:21 +01:00
log.c MINOR: listener: remove the useless ->default_target field 2023-02-03 18:00:20 +01:00
lru.c BUILD/DEBUG: lru: fix printf format in debug code 2022-04-12 08:19:33 +02:00
mailers.c BUG/MEDIUM: mailers: Set the object type for check attached to an email alert 2022-06-08 15:28:38 +02:00
map.c MINOR: channel/applets: Stop to test CF_WRITE_ERROR flag if CF_SHUTW is enough 2023-01-09 18:41:08 +01:00
mjson.c
mqtt.c MEDIUM: mqtt: support mqtt_is_valid and mqtt_field_value converters for MQTTv3.1 2022-03-22 09:25:52 +01:00
mux_fcgi.c BUG/MINOR: mux-fcgi: Correctly set pathinfo 2023-01-18 07:53:05 +01:00
mux_h1.c BUG/MINOR: mux-h1: Don't report an H1C error on client timeout 2023-02-13 09:43:38 +01:00
mux_h2.c MINOR: stats: add by HTTP version cumulated number of sessions and requests 2023-02-03 17:55:49 +01:00
mux_pt.c MEDIUM: mux-pt: Don't always set a final error on SE on the sending path 2022-11-17 14:33:15 +01:00
mux_quic.c MINOR: stats: add by HTTP version cumulated number of sessions and requests 2023-02-03 17:55:49 +01:00
mworker-prog.c BUILD: mworker: include tools.h for platforms without unsetenv() 2022-01-28 19:04:02 +01:00
mworker.c MINOR: channel/applets: Stop to test CF_WRITE_ERROR flag if CF_SHUTW is enough 2023-01-09 18:41:08 +01:00
namespace.c
ncbuf.c CLEANUP: ncbuf: use standard BUG_ON with DEBUG_STRICT 2022-11-29 15:15:27 +01:00
pattern.c BUG/MEDIUM: pattern: only visit equivalent nodes when skipping versions 2022-08-01 11:59:46 +02:00
payload.c CLEANUP: check: rename all occurrences of stconn "cs" to "sc" 2022-05-27 19:33:35 +02:00
peers.c BUG/MEDIUM: peers: make "show peers" more careful about partial initialization 2023-01-12 17:09:34 +01:00
pipe.c
pool.c BUG/MINOR: pool/stats: Use ullong to report total pool usage in bytes in stats 2022-12-22 13:46:21 +01:00
proto_quic.c MINOR: listener: remove the now useless LI_F_QUIC_LISTENER flag 2023-02-03 18:00:20 +01:00
proto_sockpair.c MINOR: sockpair: move send_fd_uxst() error message in caller 2022-07-25 16:11:11 +02:00
proto_tcp.c MINOR: listener: move TCP_FO to bind_conf 2023-02-03 18:00:20 +01:00
proto_udp.c BUG/MINOR: listeners: fix suspend/resume of inherited FDs 2023-01-16 14:00:50 +01:00
proto_uxdg.c MINOR: protocol: replace ctrl_type with xprt_type and clarify it 2022-05-20 18:39:43 +02:00
proto_uxst.c MINOR: protocol: replace ctrl_type with xprt_type and clarify it 2022-05-20 18:39:43 +02:00
protocol.c MINOR: quic: Add "no-quic" global option 2023-01-17 16:35:20 +01:00
proxy.c MEDIUM: listener: move the analysers mask to the bind_conf 2023-02-03 18:00:20 +01:00
qmux_http.c BUG/MEDIUM: mux-quic: properly trim HTX buffer on snd_buf reset 2022-09-20 15:35:33 +02:00
qmux_trace.c BUILD: quic: Fix build for m68k cross-compilation 2022-10-18 12:04:10 +02:00
qpack-dec.c BUG/CRITICAL: http: properly reject empty http header field names 2023-02-14 08:48:54 +01:00
qpack-enc.c BUG/MINOR: qpack: support bigger prefix-integer encoding 2022-05-30 14:30:05 +02:00
qpack-tbl.c CLEANUP: qpack: properly use the QPACK macros not HPACK ones in debug code 2022-11-24 15:38:26 +01:00
queue.c BUG/MEDIUM: queue/threads: limit the number of entries dequeued at once 2022-07-30 10:00:59 +02:00
quic_cc.c CLEANUP: quic: fix headers 2022-10-03 16:25:17 +02:00
quic_cc_cubic.c BUG/MEDIUM: quic: Floating point exception in cubic_root() 2022-08-03 14:27:20 +02:00
quic_cc_newreno.c CLEANUP: quic: create a dedicated quic_conn module 2022-10-03 16:25:17 +02:00
quic_conn.c BUG/MINOR: quic: Wrong datagram dispatch because of qc_check_dcid() 2023-02-13 16:14:24 +01:00
quic_frame.c MINOR: quic: refactor frame deallocation 2023-02-03 11:55:41 +01:00
quic_loss.c BUG/MINOR: quic: Too big PTO during handshakes 2023-02-03 17:55:55 +01:00
quic_sock.c MEDIUM: quic-sock: fix udp source address for send on listener socket 2023-01-20 17:06:04 +01:00
quic_stats.c MINOR: quic: detect connection migration 2022-12-02 14:45:43 +01:00
quic_stream.c MINOR: quic: New quic_cstream object implementation 2022-10-13 10:12:03 +02:00
quic_tls.c MINOR: quic: Replace v2 draft definitions by those of the final 2 version 2023-01-17 16:35:20 +01:00
quic_tp.c MINOR: quic: Update version_information transport parameter to draft-14 2023-02-06 11:54:07 +01:00
raw_sock.c OPTIM: global: move byte counts out of global and per-thread 2023-01-12 16:37:45 +01:00
regex.c BUG/MINOR: regex: Properly handle PCRE2 lib compiled without JIT support 2022-09-01 19:34:46 +02:00
resolvers.c BUG/MEDIUM: stats: fix resolvers dump 2023-02-06 07:53:03 +01:00
ring.c MINOR: channel/applets: Stop to test CF_WRITE_ERROR flag if CF_SHUTW is enough 2023-01-09 18:41:08 +01:00
sample.c MINOR: sample: add ARGC_OPT 2023-01-27 15:18:59 +01:00
server.c BUG/MINOR: server/add: ensure minconn/maxconn consistency when adding server 2023-02-08 14:48:21 +01:00
server_state.c BUILD: server-state: avoid using not-so-portable isblank() 2022-01-28 19:04:02 +01:00
session.c BUG/MEDIUM: quic: fix crash when "option nolinger" is set in the frontend 2023-02-09 18:04:10 +01:00
sha1.c
shctx.c MINOR: shctx: add a few BUG_ON() for consistency checks 2021-11-19 19:25:13 +01:00
signal.c BUG/MINOR: signals/poller: ensure wakeup from signals 2022-09-09 11:15:22 +02:00
sink.c BUG/MINOR: sink: free the forwarding task on exit 2023-01-26 15:49:32 +01:00
slz.c IMPORT: slz: declare len to fix debug build when optimal match is enabled 2022-11-14 11:14:02 +01:00
sock.c MINOR: sockpair: move send_fd_uxst() error message in caller 2022-07-25 16:11:11 +02:00
sock_inet.c BUG/MEDIUM: listener: duplicate inherited FDs if needed 2023-01-11 11:27:20 +01:00
sock_unix.c BUG/MEDIUM: listener: duplicate inherited FDs if needed 2023-01-11 11:27:20 +01:00
ssl_ckch.c MINOR: ssl-ckch: Stop to test CF_WRITE_ERROR to commit CA/CRL file 2023-02-13 09:43:38 +01:00
ssl_crtlist.c MINOR: ssl: rename confusing ssl_bind_kws 2023-02-16 16:03:45 +01:00
ssl_ocsp.c MINOR: ssl/ocsp: add a function to check the OCSP update configuration 2023-02-08 11:40:31 +01:00
ssl_sample.c BUG/MINOR: ssl: don't initialize the keylog callback when not required 2022-11-18 15:24:23 +01:00
ssl_sock.c MEDIUM: quic: Remove qc_conn_finalize() from the ClientHello TLS callbacks 2023-02-03 17:55:55 +01:00
ssl_utils.c MINOR: ssl: Move OCSP code to a dedicated source file 2022-12-21 11:21:07 +01:00
stats.c BUG/MINOR: clock/stats: also use start_time not start_date in HTML info 2023-02-10 16:53:35 +01:00
stconn.c BUG/MEDIUM: stconn: Don't rearm the read expiration date if EOI was reached 2023-02-17 10:16:25 +01:00
stick_table.c BUG/MEDIUM: stick-table: do not leave entries in end of window during purge 2023-02-08 08:55:02 +01:00
stream.c MINOR: global: Add an option to disable the data fast-forward 2023-02-17 10:17:02 +01:00
task.c BUG/MEDIUM: sched: allow a bit more TASK_HEAVY to be processed when needed 2023-02-17 16:01:34 +01:00
tcp_act.c MEDIUM: tcp-act: add parameter rst-ttl to silent-drop 2022-11-19 04:53:47 +01:00
tcp_rules.c MINOR: channel: Stop to test CF_READ_ERROR flag if CF_SHUTR is enough 2023-01-09 18:41:08 +01:00
tcp_sample.c MINOR: Revert part of clarifying samples support per os commit 2022-09-03 06:11:08 +02:00
tcpcheck.c BUILD: tcpcheck: use __fallthrough in check_proxy_tcpcheck() 2022-11-14 11:14:02 +01:00
thread.c BUG/MEDIUM: thread: fix extraneous shift in the thread_set parser 2023-02-06 18:01:50 +01:00
time.c
tools.c MINOR: listener: also support "quic+" as an address prefix 2023-01-16 14:00:51 +01:00
trace.c MINOR: trace: add a trace_no_cb() dummy callback for when to use no callback 2023-01-26 15:49:43 +01:00
uri_auth.c
uri_normalizer.c
vars.c BUILD: vars: use __fallthrough in var_accounting_{diff,add}() 2022-11-14 11:14:02 +01:00
version.c
wdt.c BUG/MEDIUM: wdt: fix wrong thread being checked for sleeping 2023-02-17 16:01:34 +01:00
xprt_handshake.c CLEANUP: pool/tree-wide: remove suffix "_pool" from certain pool names 2022-06-23 11:49:09 +02:00
xprt_quic.c BUG/MINOR: quic: fix subscribe operation 2022-10-26 18:18:26 +02:00