HAProxy - Load balancer
Find a file
Willy Tarreau f93c7be87f MEDIUM: activity: collect memory allocator statistics with USE_MEMORY_PROFILING
When built with USE_MEMORY_PROFILING the main memory allocation functions
are diverted to collect statistics per caller. It is a bit tricky because
the only way to call the original ones is to find their pointer, which
requires dlsym(), and which is not available everywhere.

Thus all functions are designed to call their fallback function (the
original one), which is preset to an initialization function that is
supposed to call dlsym() to resolve the missing symbols, and vanish.
This saves expensive tests in the critical path.

A second problem is that dlsym() calls calloc() to initialize some
error messages. After plenty of tests with posix_memalign(), valloc()
and friends, it turns out that returning NULL still makes it happy.
Thus we currently use a visit counter (in_memprof) to detect if we're
reentering, in which case all allocation functions return NULL.

In order to convert a return address to an entry in the stats, we
perform a cheap hash consisting in multiplying the pointer by a
balanced number (as many zeros as ones) and keeping the middle bits.
The hash is already pretty good like this, achieving to store up to
638 entries in a 2048-entry table without collision. But in order to
further refine this and improve the fill ratio of the table, in case
of collision we move up to 16 adjacent entries to find a free place.
This remains quite cheap and manages to store all of these inside a
1024-entries hash table with even less risk of collision.

Also, free(NULL) does not produce any stats. By doing so we reduce
from 638 to 208 the average number of entries needed for a basic
config using SSL. free(NULL) not only provides no information as it's
a NOP, but keeping it is pure pollution as it happens all the time.

When DEBUG_MEM_STATS is enabled, malloc/calloc/realloc are redefined as
macros, preventing the code from compiling. Thus, when this option is
detected, the macros are undefined as they are pointless there anyway.

The functions are optimized to quickly jump to the fallback and as such
become almost invisible in terms of processing time, execpt an extra
"if" on a read_mostly variable and a jump. Considering that this only
happens for pool misses and library routines, this remains acceptable.

Performance tests in SSL (the most stressful test) shows less than 1%
performance loss when profiling is enabled on 2c4t.

The code was written in a way to ease backporting to modern versions
(2.2+) if needed, so it keeps the long names for integers and doesn't
use the _INC version of the atomic ops.
2021-05-05 19:09:19 +02:00
.github CI: Github Actions: switch to LibreSSL-3.3.3 2021-05-05 11:29:05 +02:00
addons CLEANUP: lists/tree-wide: rename some list operations to avoid some confusion 2021-04-21 09:20:17 +02:00
admin MINOR: freq_ctr: unify freq_ctr and freq_ctr_period into freq_ctr 2021-04-11 11:11:27 +02:00
dev CLEANUP: dev/flags: remove useless test in the stdin number parser 2021-04-03 15:29:10 +02:00
doc MINOR: activity: add a "memory" entry to "profiling" 2021-05-05 18:55:02 +02:00
examples CLEANUP: assorted typo fixes in the code and comments 2020-06-26 11:27:28 +02:00
include MINOR: activity: add a "memory" entry to "profiling" 2021-05-05 18:55:02 +02:00
reg-tests REGTESTS: add minimal CLI "add map" tests 2021-04-29 16:19:03 +02:00
scripts BUILD: Makefile: move REGTESTST_TYPE default setting 2021-02-05 11:41:16 +01:00
src MEDIUM: activity: collect memory allocator statistics with USE_MEMORY_PROFILING 2021-05-05 19:09:19 +02:00
tests CLEANUP: lists/tree-wide: rename some list operations to avoid some confusion 2021-04-21 09:20:17 +02:00
.cirrus.yml CI: cirrus: install "pcre" package 2021-04-15 08:35:46 +02:00
.gitattributes MINOR: Configure the cpp userdiff driver for *.[ch] in .gitattributes 2021-02-22 18:17:57 +01:00
.gitignore CLEANUP: contrib: remove the last references to the now dead contrib/ directory 2021-04-21 15:13:58 +02:00
.travis.yml CI: travis-ci: enable weekly graviton2 builds 2021-04-15 17:10:22 +02:00
BRANCHES DOC: fix some spelling issues over multiple files 2021-01-08 14:53:47 +01:00
CHANGELOG [RELEASE] Released version 2.4-dev18 2021-05-01 08:25:15 +02:00
CONTRIBUTING CLEANUP: contrib: remove the last references to the now dead contrib/ directory 2021-04-21 15:13:58 +02:00
INSTALL BUILD: compression: switch SLZ from out-of-tree to in-tree 2021-04-22 16:08:25 +02:00
LICENSE LICENSE: add licence exception for OpenSSL 2012-09-07 13:52:26 +02:00
MAINTAINERS CONTRIB: move spoa_example out of the tree 2021-04-21 09:39:06 +02:00
Makefile MINOR: cpuset: define a platform-independent cpuset type 2021-04-23 16:06:49 +02:00
README DOC: create a BRANCHES file to explain the life cycle 2019-06-15 22:00:14 +02:00
ROADMAP DOC: update the outdated ROADMAP file 2019-06-15 21:59:54 +02:00
SUBVERS BUILD: use format tags in VERDATE and SUBVERS files 2013-12-10 11:22:49 +01:00
VERDATE [RELEASE] Released version 2.4-dev18 2021-05-01 08:25:15 +02:00
VERSION [RELEASE] Released version 2.4-dev18 2021-05-01 08:25:15 +02:00

The HAProxy documentation has been split into a number of different files for
ease of use.

Please refer to the following files depending on what you're looking for :

  - INSTALL for instructions on how to build and install HAProxy
  - BRANCHES to understand the project's life cycle and what version to use
  - LICENSE for the project's license
  - CONTRIBUTING for the process to follow to submit contributions

The more detailed documentation is located into the doc/ directory :

  - doc/intro.txt for a quick introduction on HAProxy
  - doc/configuration.txt for the configuration's reference manual
  - doc/lua.txt for the Lua's reference manual
  - doc/SPOE.txt for how to use the SPOE engine
  - doc/network-namespaces.txt for how to use network namespaces under Linux
  - doc/management.txt for the management guide
  - doc/regression-testing.txt for how to use the regression testing suite
  - doc/peers.txt for the peers protocol reference
  - doc/coding-style.txt for how to adopt HAProxy's coding style
  - doc/internals for developer-specific documentation (not all up to date)