bind9/lib/isc
Ondřej Surý 50f357cb36
Refactor the dns_adb unit
The dns_adb unit has been refactored to be much simpler.  Following
changes have been made:

1. Simplify the ADB to always allow GLUE and hints

   There were only two places where dns_adb_createfind() was used - in
   the dns_resolver unit where hints and GLUE addresses were ok, and in
   the dns_zone where dns_adb_createfind() would be called without
   DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set.

   Simplify the logic by allowing hint and GLUE addresses when looking
   up the nameserver addresses to notify.  The difference is negligible
   and would cause a difference in the notified addresses only when
   there's mismatch between the parent and child addresses and we
   haven't cached the child addresses yet.

2. Drop the namebuckets and entrybuckets

   Formerly, the namebuckets and entrybuckets were used to reduced the
   lock contention when accessing the double-linked lists stored in each
   bucket.  In the previous refactoring, the custom hashtable for the
   buckets has been replaced with isc_ht/isc_hashmap, so only a single
   item (mostly, see below) would end up in each bucket.

   Removing the entrybuckets has been straightforward, the only matching
   was done on the isc_sockaddr_t member of the dns_adbentry.

   Removing the zonebuckets required GLUEOK and HINTOK bits to be
   removed because the find could match entries with-or-without the bits
   set, and creating a custom key that stores the
   DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a
   straightforward lookup into the hashtable without traversing a list
   that contains items with different flags.

3. Remove unassociated entries from ADB database

   Previously, the adbentries could live in the ADB database even after
   unlinking them from dns_adbnames.  Such entries would show up as
   "Unassociated entries" in the ADB dump.  The benefit of keeping such
   entries is little - the chance that we link such entry to a adbname
   is small, and it's simpler to evict unlinked entries from the ADB
   cache (and the hashtable) than create second LRU cleaning mechanism.

   Unlinked ADB entries are now directly deleted from the hash
   table (hashmap) upon destruction.

4. Cleanup expired entries from the hash table

   When buckets were still in place, the code would keep the buckets
   always allocated and never shrink the hash table (hashmap).  With
   proper reference counting in place, we can delete the adbnames from
   the hash table and the LRU list.

5. Stop purging the names early when we hit the time limit

   Because the LRU list is now time ordered, we can stop purging the
   names when we find a first entry that doesn't fullfil our time-based
   eviction criteria because no further entry on the LRU list will meet
   the criteria.

Future work:

1. Lock contention

   In this commit, the focus was on correctness of the data structure,
   but in the future, the lock contention in the ADB database needs to
   be addressed.  Currently, we use simple mutex to lock the hash
   tables, because we almost always need to use a write lock for
   properly purging the hashtables.  The ADB database needs to be
   sharded (similar to the effect that buckets had in the past).  Each
   shard would contain own hashmap and own LRU list.

2. Time-based purging

   The ADB names and entries stay intact when there are no lookups.
   When we add separate shards, a timer needs to be added for time-based
   cleaning in case there's no traffic hashing to the inactive shard.

3. Revisit the 30 minutes limit

   The ADB cache is capped at 30 minutes.  This needs to be revisited,
   and at least the limit should be configurable (in both directions).
2022-11-30 10:03:24 +01:00
..
include Refactor the dns_adb unit 2022-11-30 10:03:24 +01:00
netmgr TCP: use uv_try_write() to optimise sends 2022-11-29 13:41:10 +02:00
.gitignore [master] update gitignore files; use rev-parse to get srcid 2014-06-17 13:49:30 -07:00
aes.c Remove EVP_CIPHER_CTX_new() and EVP_CIPHER_CTX_free() shims 2022-03-02 10:48:09 +00:00
ascii.c Consolidate some ASCII tables in isc/ascii and isc/hex 2022-09-12 12:18:57 +01:00
assertions.c Simplify way we tag unreachable code with only ISC_UNREACHABLE() 2022-03-25 08:33:43 +01:00
astack.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
async.c isc_async_run() runs events in reverse order 2022-10-31 05:43:45 -07:00
backtrace.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
base32.c Remove use of the inline keyword used as suggestion to compiler 2022-03-25 08:33:43 +01:00
base64.c Remove use of the inline keyword used as suggestion to compiler 2022-03-25 08:33:43 +01:00
bind9.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
buffer.c change ISC__BUFFER macros to inline functions 2022-09-26 23:49:27 -07:00
commandline.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
condition.c De-duplicate some calls to strerror_r() 2022-10-17 11:58:26 +01:00
counter.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
crc64.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
dir.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
entropy.c Switch the CSPRNG function from RAND_bytes() to uv_random() 2022-09-26 15:13:11 +02:00
errno.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
errno2result.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
errno2result.h Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
error.c Include the function name when reporting unexpected errors 2022-10-17 13:43:59 +01:00
event.c Simplify the isc_event API 2022-08-25 12:24:25 +02:00
file.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
fsaccess.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
fsaccess_common.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
fsaccess_common_p.h Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
glob.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
hash.c Improve reporting for pthread_once errors 2022-10-14 16:39:21 +02:00
hashmap.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
heap.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
hex.c Consolidate some ASCII tables in isc/ascii and isc/hex 2022-09-12 12:18:57 +01:00
hmac.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
ht.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
httpd.c Be more resilient when destroying the httpd requests 2022-11-25 16:20:34 +01:00
interfaceiter.c De-duplicate __FILE__, __LINE__ 2022-10-17 11:58:26 +01:00
iterated_hash.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
jemalloc_shim.h Replace isc_mem_*_aligned(..., alignment) with isc_mem_*x(..., flags) 2022-10-05 16:44:05 +02:00
job.c New event loop handling API 2022-08-25 12:24:29 +02:00
job_p.h New event loop handling API 2022-08-25 12:24:29 +02:00
lex.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
lib.c Use custom isc_mem based allocator for libxml2 2022-09-27 17:10:42 +02:00
log.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
loop.c Properly name the loop->mctx 2022-11-08 13:32:13 +01:00
loop_p.h Convert isc_ratelimiter API to use on-loop timers 2022-09-21 14:25:33 -07:00
Makefile.am Add isc_hashmap API that implements Robin Hood hashing 2022-11-10 15:07:19 +01:00
managers.c Update netmgr, tasks, and applications to use isc_loopmgr 2022-08-26 09:09:24 +02:00
md.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
mem.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
mem_p.h Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
meminfo.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
mutex.c Improve reporting for pthread_once errors 2022-10-14 16:39:21 +02:00
mutex_p.h Use library constructor to create default mutex attr once 2022-07-13 13:19:32 +02:00
mutexblock.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
net.c De-duplicate some calls to strerror_r() 2022-10-17 11:58:26 +01:00
netaddr.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
netscope.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
nonce.c Switch the CSPRNG function from RAND_bytes() to uv_random() 2022-09-26 15:13:11 +02:00
openssl_shim.c Do not provide a shim for SSL_SESSION_is_resumable() 2022-05-23 18:25:18 +03:00
openssl_shim.h Do not provide a shim for SSL_SESSION_is_resumable() 2022-05-23 18:25:18 +03:00
os.c Remove use of the inline keyword used as suggestion to compiler 2022-03-25 08:33:43 +01:00
os_p.h Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
parseint.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
picohttpparser.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
picohttpparser.h Add picohttpparser.{c.h} from https://github.com/h2o/picohttpparser 2022-10-14 11:26:54 +02:00
portset.c Use designated initializers instead of memset()/MEM_ZERO for structs 2022-10-05 16:44:05 +02:00
quota.c REQUIRE should not have side effects 2022-07-05 12:22:55 -07:00
radix.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
random.c Switch the CSPRNG function from RAND_bytes() to uv_random() 2022-09-26 15:13:11 +02:00
random_p.h Move random number re-seeding out of the hot path 2022-09-19 16:27:12 +02:00
ratelimiter.c Refactor isc_ratelimiter API 2022-09-30 10:36:30 +02:00
regex.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
region.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
resource.c Avoid dead code warning when using a constant boolean 2022-10-05 15:51:05 +00:00
result.c Add dns_rdata_checksvcb 2022-10-29 00:22:54 +11:00
rwlock.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
safe.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
serial.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
signal.c New event loop handling API 2022-08-25 12:24:29 +02:00
siphash.c Add a case-insensitive option directly to siphash 2-4 implementation 2022-10-04 10:32:40 +02:00
sockaddr.c De-duplicate __FILE__, __LINE__ 2022-10-17 11:58:26 +01:00
stats.c Assert if statistics counter underflows in the developer mode 2022-02-10 17:18:09 +01:00
stdio.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
stdtime.c Deduplicate time unit conversion factors 2022-11-25 13:23:36 +00:00
string.c Use strnstr implementation from FreeBSD if not provided by OS 2022-10-04 14:21:41 +11:00
symtab.c Consolidate some ASCII tables in isc/ascii and isc/hex 2022-09-12 12:18:57 +01:00
syslog.c Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
task.c Update netmgr, tasks, and applications to use isc_loopmgr 2022-08-26 09:09:24 +02:00
tests Move all the unit tests to /tests/<libname>/ 2022-05-28 14:53:02 -07:00
thread.c Fix error reporting for POSIX Threads functions 2022-09-09 20:25:47 +02:00
tid.c New event loop handling API 2022-08-25 12:24:29 +02:00
time.c Deduplicate time unit conversion factors 2022-11-25 13:23:36 +00:00
timer.c Require isc_timer to be manipulated on the timer loop 2022-09-21 14:25:33 -07:00
tls.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
tm.c Consolidate some ASCII tables in isc/ascii and isc/hex 2022-09-12 12:18:57 +01:00
trampoline.c Move random number re-seeding out of the hot path 2022-09-19 16:27:12 +02:00
trampoline_p.h Update the copyright information in all files in the repository 2022-01-11 09:05:02 +01:00
url.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
utf8.c Update sources to Clang 15 formatting 2022-11-29 08:54:34 +01:00
uv.c Add and use semantic patch to replace isc_mem_get/allocate+memset 2022-10-05 16:44:05 +02:00
work.c New event loop handling API 2022-08-25 12:24:29 +02:00
xml.c Add autoconf option to enable memory leak detection in libraries 2022-09-27 17:53:04 +02:00