upstream/bind9
1
0
Fork 0
mirror of https://github.com/isc-projects/bind9.git synced 2026-02-27 03:51:16 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
32580 commits 18 branches 854 tags 440 MiB
Commit graph

12782 commits

Author SHA1 Message Date
Evan Hunt
86eddebc83 Purge memory pool upon plugin destruction 
The typical sequence of events for AAAA queries which trigger recursion
for an A RRset at the same name is as follows:

 1. Original query context is created.
 2. An AAAA RRset is found in cache.
 3. Client-specific data is allocated from the filter-aaaa memory pool.
 4. Recursion is triggered for an A RRset.
 5. Original query context is torn down.

 6. Recursion for an A RRset completes.
 7. A second query context is created.
 8. Client-specific data is retrieved from the filter-aaaa memory pool.
 9. The response to be sent is processed according to configuration.
10. The response is sent.
11. Client-specific data is returned to the filter-aaaa memory pool.
12. The second query context is torn down.

However, steps 6-12 are not executed if recursion for an A RRset is
canceled.  Thus, if named is in the process of recursing for A RRsets
when a shutdown is requested, the filter-aaaa memory pool will have
outstanding allocations which will never get released.  This in turn
leads to a crash since every memory pool must not have any outstanding
allocations by the time isc_mempool_destroy() is called.

Fix by creating a stub query context whenever fetch_callback() is called,
including cancellation events. When the qctx is destroyed, it will ensure
the client is detached and the plugin memory is freed.
 2020-09-25 13:32:34 -07:00
Matthijs Mekking
d14c2d0d73 rndc dumpdb -expired: print when RRsets expired 
When calling 'rndc dumpdb -expired', also print when the RRset expired.
 2020-09-23 16:09:26 +02:00
Matthijs Mekking
388cc666e5 Handle ancient rrsets in bind_rdataset 
An ancient RRset is one still in the cache but expired, and awaiting
cleanup.
 2020-09-23 16:08:29 +02:00
Matthijs Mekking
17d5bd4493 Include expired rdatasets in iteration functions 
By changing the check in 'rdatasetiter_first' and 'rdatasetiter_next'
from "now > header->rdh_ttl" to "now - RBDTB_VIRTUAL > header->rdh_ttl"
we include expired rdataset entries so that they can be used for
"rndc dumpdb -expired".
 2020-09-23 16:08:29 +02:00
Matthijs Mekking
8beda7d2ea Add -expired flag to rndc dumpdb command 
This flag is the same as -cache, but will use a different style format
that will also print expired entries (awaiting cleanup) from the cache.
 2020-09-23 16:08:29 +02:00
Mark Andrews
c37b251eb9 It appears that you can't change what you are polling for while connecting. 
WARNING: ThreadSanitizer: data race
    Read of size 8 at 0x000000000001 by thread T1 (mutexes: write M1):
    #0 epoll_ctl <null>
    #1 watch_fd lib/isc/unix/socket.c:704:8
    #2 wakeup_socket lib/isc/unix/socket.c:897:11
    #3 process_ctlfd lib/isc/unix/socket.c:3362:3
    #4 process_fds lib/isc/unix/socket.c:3275:10
    #5 netthread lib/isc/unix/socket.c:3516:10

    Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M2):
    #0 connect <null>
    #1 isc_socket_connect lib/isc/unix/socket.c:4737:7
    #2 resquery_send lib/dns/resolver.c:2892:13
    #3 fctx_query lib/dns/resolver.c:2202:12
    #4 fctx_try lib/dns/resolver.c:4300:11
    #5 resquery_connected lib/dns/resolver.c:3130:4
    #6 dispatch lib/isc/task.c:1152:7
    #7 run lib/isc/task.c:1344:2

    Location is file descriptor 513 created by thread T2 at:
    #0 connect <null>
    #1 isc_socket_connect lib/isc/unix/socket.c:4737:7
    #2 resquery_send lib/dns/resolver.c:2892:13
    #3 fctx_query lib/dns/resolver.c:2202:12
    #4 fctx_try lib/dns/resolver.c:4300:11
    #5 resquery_connected lib/dns/resolver.c:3130:4
    #6 dispatch lib/isc/task.c:1152:7
    #7 run lib/isc/task.c:1344:2

    Mutex M1 (0x000000000016) created at:
    #0 pthread_mutex_init <null>
    #1 isc__mutex_init lib/isc/pthreads/mutex.c:288:8
    #2 setup_thread lib/isc/unix/socket.c:3584:3
    #3 isc_socketmgr_create2 lib/isc/unix/socket.c:3825:3
    #4 create_managers bin/named/main.c:932:11
    #5 setup bin/named/main.c:1223:11
    #6 main bin/named/main.c:1523:2

    Mutex M2 is already destroyed.

    Thread T1 'isc-socket-1' (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_socketmgr_create2 lib/isc/unix/socket.c:3826:3
    #3 create_managers bin/named/main.c:932:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    Thread T2 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: data race in epoll_ctl
 2020-09-23 13:54:06 +10:00
Mark Andrews
a669c919c8 Address lock order inversions. 
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
    Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000000) => M1

    Mutex M2 acquired here while holding mutex M1 in thread T1:
    #0 pthread_mutex_lock <null>
    #1 dns_view_findzonecut lib/dns/view.c:1310:2
    #2 fctx_create lib/dns/resolver.c:5070:13
    #3 dns_resolver_createfetch lib/dns/resolver.c:10813:12
    #4 dns_resolver_prime lib/dns/resolver.c:10442:12
    #5 dns_view_find lib/dns/view.c:1176:4
    #6 dbfind_name lib/dns/adb.c:3833:11
    #7 dns_adb_createfind lib/dns/adb.c:3155:12
    #8 findname lib/dns/resolver.c:3497:11
    #9 fctx_getaddresses lib/dns/resolver.c:3808:3
    #10 fctx_try lib/dns/resolver.c:4197:12
    #11 fctx_start lib/dns/resolver.c:4824:4
    #12 dispatch lib/isc/task.c:1152:7
    #13 run lib/isc/task.c:1344:2

    Mutex M1 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null>
    #1 dns_resolver_createfetch lib/dns/resolver.c:10767:2
    #2 dns_resolver_prime lib/dns/resolver.c:10442:12
    #3 dns_view_find lib/dns/view.c:1176:4
    #4 dbfind_name lib/dns/adb.c:3833:11
    #5 dns_adb_createfind lib/dns/adb.c:3155:12
    #6 findname lib/dns/resolver.c:3497:11
    #7 fctx_getaddresses lib/dns/resolver.c:3808:3
    #8 fctx_try lib/dns/resolver.c:4197:12
    #9 fctx_start lib/dns/resolver.c:4824:4
    #10 dispatch lib/isc/task.c:1152:7
    #11 run lib/isc/task.c:1344:2

    Mutex M1 acquired here while holding mutex M2 in thread T1:
    #0 pthread_mutex_lock <null>
    #1 dns_resolver_shutdown lib/dns/resolver.c:10530:4
    #2 view_flushanddetach lib/dns/view.c:632:4
    #3 dns_view_detach lib/dns/view.c:689:2
    #4 qctx_destroy lib/ns/query.c:5152:2
    #5 fetch_callback lib/ns/query.c:5749:3
    #6 dispatch lib/isc/task.c:1152:7
    #7 run lib/isc/task.c:1344:2

    Mutex M2 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null>
    #1 view_flushanddetach lib/dns/view.c:630:3
    #2 dns_view_detach lib/dns/view.c:689:2
    #3 qctx_destroy lib/ns/query.c:5152:2
    #4 fetch_callback lib/ns/query.c:5749:3
    #5 dispatch lib/isc/task.c:1152:7
    #6 run lib/isc/task.c:1344:2

    Thread T1 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_mutex_lock
 2020-09-23 01:13:28 +00:00
Mark Andrews
f0d9bf7c30 Clone the saved / query message buffers 
The message buffer passed to ns__client_request is only valid for
the life of the the ns__client_request call.  Save a copy of it
when we recurse or process a update as ns__client_request will
return before those operations complete.
 2020-09-23 10:37:42 +10:00
Mark Andrews
1090876693 Address lock-order-inversion 
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
    Cycle in lock order graph: M1 (0x000000000001) => M2 (0x000000000002) => M1

    Mutex M2 acquired here while holding mutex M1 in thread T1:
    #0 pthread_rwlock_wrlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:52:4
    #2 zone_postload lib/dns/zone.c:5101:2
    #3 receive_secure_db lib/dns/zone.c:16206:11
    #4 dispatch lib/isc/task.c:1152:7
    #5 run lib/isc/task.c:1344:2

    Mutex M1 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null>
    #1 receive_secure_db lib/dns/zone.c:16204:2
    #2 dispatch lib/isc/task.c:1152:7
    #3 run lib/isc/task.c:1344:2

    Mutex M1 acquired here while holding mutex M2 in thread T1:
    #0 pthread_mutex_lock <null>
    #1 get_raw_serial lib/dns/zone.c:2518:2
    #2 zone_gotwritehandle lib/dns/zone.c:2559:4
    #3 dispatch lib/isc/task.c:1152:7
    #4 run lib/isc/task.c:1344:2

    Mutex M2 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 zone_gotwritehandle lib/dns/zone.c:2552:2
    #3 dispatch lib/isc/task.c:1152:7
    #4 run lib/isc/task.c:1344:2

    Thread T1 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_wrlock
 2020-09-22 11:43:44 +00:00
Ondřej Surý
d4976e0ebe Add separate prefetch nmhandle to ns_client_t 
As the query_prefetch() or query_rpzfetch() could be called during
"regular" fetch, we need to introduce separate storage for attaching
the nmhandle during prefetching the records.  The query_prefetch()
and query_rpzfetch() are guarded for re-entrance by .query.prefetch
member of ns_client_t, so we can reuse the same .prefetchhandle for
both.
 2020-09-22 09:56:26 +02:00
Mark Andrews
48d54368d5 Remove the memmove call on dns_rbtnode_t structure that contains atomics 
Calling the plain memmove on the structure that contains atomic members
triggers following TSAN warning (even when we don't really use the
atomic members in the code):

    WARNING: ThreadSanitizer: data race
      Read of size 8 at 0x000000000001 by thread T1 (mutexes: write M1, write M2):
	#0 memmove <null>
	#1 memmove /usr/include/x86_64-linux-gnu/bits/string_fortified.h:40:10
	#2 deletefromlevel lib/dns/rbt.c:2675:3
	#3 dns_rbt_deletenode lib/dns/rbt.c:2143:2
	#4 delete_node lib/dns/rbtdb.c
	#5 decrement_reference lib/dns/rbtdb.c:2202:4
	#6 prune_tree lib/dns/rbtdb.c:2259:3
	#7 dispatch lib/isc/task.c:1152:7
	#8 run lib/isc/task.c:1344:2

      Previous atomic write of size 8 at 0x000000000001 by thread T2 (mutexes: read M3):
	#0 __tsan_atomic64_fetch_sub <null>
	#1 decrement_reference lib/dns/rbtdb.c:2103:7
	#2 detachnode lib/dns/rbtdb.c:5440:6
	#3 dns_db_detachnode lib/dns/db.c:588:2
	#4 qctx_clean lib/ns/query.c:5104:3
	#5 ns_query_done lib/ns/query.c:10868:2
	#6 query_sign_nodata lib/ns/query.c
	#7 query_nodata lib/ns/query.c:8438:11
	#8 query_gotanswer lib/ns/query.c
	#9 query_lookup lib/ns/query.c:5624:10
	#10 ns__query_start lib/ns/query.c:5500:10
	#11 query_setup lib/ns/query.c:5224:11
	#12 ns_query_start lib/ns/query.c:11357:8
	#13 ns__client_request lib/ns/client.c:2166:3
	#14 udp_recv_cb lib/isc/netmgr/udp.c:414:2
	#15 uv__udp_recvmsg /home/ondrej/Projects/tsan/libuv/src/unix/udp.c
	#16 uv__udp_io /home/ondrej/Projects/tsan/libuv/src/unix/udp.c:180:5
	#17 uv__io_poll /home/ondrej/Projects/tsan/libuv/src/unix/linux-core.c:461:11
	#18 uv_run /home/ondrej/Projects/tsan/libuv/src/unix/core.c:385:5
	#19 nm_thread lib/isc/netmgr/netmgr.c:500:11

      Location is heap block of size 132 at 0x000000000030 allocated by thread T3:
	#0 malloc <null>
	#1 default_memalloc lib/isc/mem.c:713:8
	#2 mem_get lib/isc/mem.c:622:8
	#3 mem_allocateunlocked lib/isc/mem.c:1268:8
	#4 isc___mem_allocate lib/isc/mem.c:1288:7
	#5 isc__mem_allocate lib/isc/mem.c:2453:10
	#6 isc___mem_get lib/isc/mem.c:1037:11
	#7 isc__mem_get lib/isc/mem.c:2432:10
	#8 create_node lib/dns/rbt.c:2239:9
	#9 dns_rbt_addnode lib/dns/rbt.c:1435:12
	#10 findnodeintree lib/dns/rbtdb.c:2895:12
	#11 findnode lib/dns/rbtdb.c:2941:10
	#12 dns_db_findnode lib/dns/db.c:439:11
	#13 diff_apply lib/dns/diff.c:306:5
	#14 dns_diff_apply lib/dns/diff.c:459:10
	#15 do_one_tuple lib/ns/update.c:444:11
	#16 update_one_rr lib/ns/update.c:495:10
	#17 update_action lib/ns/update.c:3123:6
	#18 dispatch lib/isc/task.c:1152:7
	#19 run lib/isc/task.c:1344:2

      Mutex M1 is already destroyed.

      Mutex M2 is already destroyed.

      Mutex M3 is already destroyed.

      Thread T1 (running) created by main thread at:
	#0 pthread_create <null>
	#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
	#2 isc_taskmgr_create lib/isc/task.c:1434:3
	#3 create_managers bin/named/main.c:915:11
	#4 setup bin/named/main.c:1223:11
	#5 main bin/named/main.c:1523:2

      Thread T2 (running) created by main thread at:
	#0 pthread_create <null>
	#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
	#2 isc_nm_start lib/isc/netmgr/netmgr.c:223:3
	#3 create_managers bin/named/main.c:909:15
	#4 setup bin/named/main.c:1223:11
	#5 main bin/named/main.c:1523:2

      Thread T3 (running) created by main thread at:
	#0 pthread_create <null>
	#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
	#2 isc_taskmgr_create lib/isc/task.c:1434:3
	#3 create_managers bin/named/main.c:915:11
	#4 setup bin/named/main.c:1223:11
	#5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: data race in memmove
 2020-09-21 08:58:20 +00:00
Ondřej Surý
79ca724d46 Handle the errors from sysconf() call in isc_meminfo_totalphys() 
isc_meminfo_totalphys() would return invalid memory size when sysconf()
call would fail, because ((size_t)-1 * -1) is very large number.
 2020-09-21 10:55:00 +02:00
Michał Kępień
dc8a7791bd Fix updating summary RPZ DB for mixed-case RPZs 
Each dns_rpz_zone_t structure keeps a hash table of the names this RPZ
database contains.  Here is what happens when an RPZ is updated:

  - a new hash table is prepared for the new version of the RPZ by
    iterating over it; each name found is added to the summary RPZ
    database,

  - every name added to the new hash table is searched for in the old
    hash table; if found, it is removed from the old hash table,

  - the old hash table is iterated over; all names found in it are
    removed from the summary RPZ database (because at that point the old
    hash table should only contain names which are not present in the
    new version of the RPZ),

  - the new hash table replaces the old hash table.

When the new version of the RPZ is iterated over, if a given name is
spelled using a different letter case than in the old version of the
RPZ, the new variant will hash to a different value than the old
variant, which means it will not be removed from the old hash table.
When the old hash table is subsequently iterated over to remove
seemingly deleted names, the old variant of the name will still be
there, causing the name to be deleted from the summary RPZ database
(which effectively causes a given rule to be ignored).

The issue can be triggered not just by altering the case of existing
names in an RPZ, but also by adding sibling names spelled with a
different letter case.  This is because RBT code preserves case when
node splitting occurs.  The end result is that when the RPZ is iterated
over, a given name may be using a different case than in the zone file
(or XFR contents).

Fix by downcasing all names found in the RPZ database before adding them
to the summary RPZ database.
 2020-09-21 09:28:36 +02:00
Ondřej Surý
0110d1ab17 Exclude isc_mem_isovermem from ThreadSanitizer 
The .is_overmem member of isc_mem_t structure is intentionally accessed
unlocked as 100% accuracy isn't necessary here.

Without the attribute, following TSAN warning would show up:

    WARNING: ThreadSanitizer: data race
      Write of size 1 at 0x000000000001 by thread T1 (mutexes: write M1, write M2):
	#0 isc___mem_put lib/isc/mem.c:1119:19
	#1 isc__mem_put lib/isc/mem.c:2439:2
	#2 dns_rdataslab_fromrdataset lib/dns/rdataslab.c:327:2
	#3 addrdataset lib/dns/rbtdb.c:6761:11
	#4 dns_db_addrdataset lib/dns/db.c:719:10
	#5 cache_name lib/dns/resolver.c:6538:13
	#6 cache_message lib/dns/resolver.c:6628:14
	#7 resquery_response lib/dns/resolver.c:7883:13
	#8 dispatch lib/isc/task.c:1152:7
	#9 run lib/isc/task.c:1344:2

      Previous read of size 1 at 0x000000000001 by thread T2 (mutexes: write M3):
	#0 isc_mem_isovermem lib/isc/mem.c:1553:15
	#1 addrdataset lib/dns/rbtdb.c:6866:25
	#2 dns_db_addrdataset lib/dns/db.c:719:10
	#3 addoptout lib/dns/ncache.c:281:10
	#4 dns_ncache_add lib/dns/ncache.c:101:10
	#5 ncache_adderesult lib/dns/resolver.c:6668:12
	#6 ncache_message lib/dns/resolver.c:6845:11
	#7 rctx_ncache lib/dns/resolver.c:9174:11
	#8 resquery_response lib/dns/resolver.c:7894:2
	#9 dispatch lib/isc/task.c:1152:7
	#10 run lib/isc/task.c:1344:2

      Location is heap block of size 328 at 0x000000000020 allocated by thread T3:
	#0 malloc <null>
	#1 default_memalloc lib/isc/mem.c:713:8
	#2 mem_create lib/isc/mem.c:763:8
	#3 isc_mem_create lib/isc/mem.c:2425:2
	#4 configure_view bin/named/server.c:4494:4
	#5 load_configuration bin/named/server.c:9062:3
	#6 run_server bin/named/server.c:9771:2
	#7 dispatch lib/isc/task.c:1152:7
	#8 run lib/isc/task.c:1344:2

    [...]

    SUMMARY: ThreadSanitizer: data race lib/isc/mem.c:1119:19 in isc___mem_put
 2020-09-17 13:51:50 +00:00
Mark Andrews
9e584a4511 Pause dbiterator ealier to prevent lock-order-inversion 
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
    Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000000) => M1

    Mutex M2 acquired here while holding mutex M1 in thread T1:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 findnodeintree lib/dns/rbtdb.c:2877:2
    #3 findnode lib/dns/rbtdb.c:2941:10
    #4 dns_db_findnode lib/dns/db.c:439:11
    #5 resume_addnsec3chain lib/dns/zone.c:3776:11
    #6 rss_post lib/dns/zone.c:20659:3
    #7 setnsec3param lib/dns/zone.c:20471:3
    #8 dispatch lib/isc/task.c:1152:7
    #9 run lib/isc/task.c:1344:2

    Mutex M1 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null>
    #1 rss_post lib/dns/zone.c:20658:3
    #2 setnsec3param lib/dns/zone.c:20471:3
    #3 dispatch lib/isc/task.c:1152:7
    #4 run lib/isc/task.c:1344:2

    Mutex M1 acquired here while holding mutex M2 in thread T2:
    #0 pthread_mutex_lock <null>
    #1 zone_nsec3chain lib/dns/zone.c:8666:5
    #2 zone_maintenance lib/dns/zone.c:11063:4
    #3 zone_timer lib/dns/zone.c:14098:2
    #4 dispatch lib/isc/task.c:1152:7
    #5 run lib/isc/task.c:1344:2

    Mutex M2 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 resume_iteration lib/dns/rbtdb.c:9357:2
    #3 dbiterator_next lib/dns/rbtdb.c:9647:3
    #4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
    #5 zone_nsec3chain lib/dns/zone.c:8656:13
    #6 zone_maintenance lib/dns/zone.c:11063:4
    #7 zone_timer lib/dns/zone.c:14098:2
    #8 dispatch lib/isc/task.c:1152:7
    #9 run lib/isc/task.c:1344:2
 2020-09-17 07:03:56 +00:00
Mark Andrews
2e63de94aa Pause the database iterator to release rwlock 2020-09-17 07:03:56 +00:00
Mark Andrews
fbed962204 Pause dbiterator to release rwlock to prevent lock-order-inversion. 
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
    Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000001) => M1

    Mutex M2 acquired here while holding mutex M1 in thread T1:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 getsigningtime lib/dns/rbtdb.c:8198:2
    #3 dns_db_getsigningtime lib/dns/db.c:979:11
    #4 set_resigntime lib/dns/zone.c:3887:11
    #5 dns_zone_markdirty lib/dns/zone.c:11119:4
    #6 update_action lib/ns/update.c:3376:3
    #7 dispatch lib/isc/task.c:1152:7
    #8 run lib/isc/task.c:1344:2

    Mutex M1 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null>
    #1 dns_zone_markdirty lib/dns/zone.c:11089:2
    #2 update_action lib/ns/update.c:3376:3
    #3 dispatch lib/isc/task.c:1152:7
    #4 run lib/isc/task.c:1344:2

    Mutex M1 acquired here while holding mutex M2 in thread T1:
    #0 pthread_mutex_lock <null>
    #1 zone_nsec3chain lib/dns/zone.c:8502:3
    #2 zone_maintenance lib/dns/zone.c:11056:4
    #3 zone_timer lib/dns/zone.c:14091:2
    #4 dispatch lib/isc/task.c:1152:7
    #5 run lib/isc/task.c:1344:2

    Mutex M2 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 resume_iteration lib/dns/rbtdb.c:9357:2
    #3 dbiterator_current lib/dns/rbtdb.c:9695:3
    #4 dns_dbiterator_current lib/dns/dbiterator.c:101:10
    #5 zone_nsec3chain lib/dns/zone.c:8539:3
    #6 zone_maintenance lib/dns/zone.c:11056:4
    #7 zone_timer lib/dns/zone.c:14091:2
    #8 dispatch lib/isc/task.c:1152:7
    #9 run lib/isc/task.c:1344:2

    Thread T1 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
 2020-09-17 07:03:56 +00:00
Mark Andrews
c9dbad97b2 Pause dbiterator to release rwlock to prevent lock-order-inversion. 
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
    Cycle in lock order graph: M1 (0x000000000001) => M2 (0x000000000000) => M1

    Mutex M2 acquired here while holding mutex M1 in thread T1:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 zone_sign lib/dns/zone.c:9247:3
    #3 zone_maintenance lib/dns/zone.c:11047:4
    #4 zone_timer lib/dns/zone.c:14090:2
    #5 dispatch lib/isc/task.c:1152:7
    #6 run lib/isc/task.c:1344:2

    Mutex M1 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 resume_iteration lib/dns/rbtdb.c:9357:2
    #3 dbiterator_next lib/dns/rbtdb.c:9647:3
    #4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
    #5 zone_sign lib/dns/zone.c:9488:13
    #6 zone_maintenance lib/dns/zone.c:11047:4
    #7 zone_timer lib/dns/zone.c:14090:2
    #8 dispatch lib/isc/task.c:1152:7
    #9 run lib/isc/task.c:1344:2

    Mutex M1 acquired here while holding mutex M2 in thread T2:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 findnodeintree lib/dns/rbtdb.c:2877:2
    #3 findnode lib/dns/rbtdb.c:2941:10
    #4 dns_db_findnode lib/dns/db.c:439:11
    #5 dns_db_getsoaserial lib/dns/db.c:780:11
    #6 dump_done lib/dns/zone.c:11428:15
    #7 dump_quantum lib/dns/masterdump.c:1487:2
    #8 dispatch lib/isc/task.c:1152:7
    #9 run lib/isc/task.c:1344:2

    Mutex M2 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 dump_done lib/dns/zone.c:11426:4
    #3 dump_quantum lib/dns/masterdump.c:1487:2
    #4 dispatch lib/isc/task.c:1152:7
    #5 run lib/isc/task.c:1344:2

    Thread T1 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    Thread T2 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
 2020-09-17 07:03:56 +00:00
Mark Andrews
98025e15d0 Pause dbiterator to release rwlock to prevent lock-order-inversion. 
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
    Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000000) => M1

    Mutex M2 acquired here while holding mutex M1 in thread T1:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 getsigningtime lib/dns/rbtdb.c:8198:2
    #3 dns_db_getsigningtime lib/dns/db.c:979:11
    #4 set_resigntime lib/dns/zone.c:3887:11
    #5 dns_zone_markdirty lib/dns/zone.c:11115:4
    #6 update_action lib/ns/update.c:3376:3
    #7 dispatch lib/isc/task.c:1152:7
    #8 run lib/isc/task.c:1344:2

    Mutex M1 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null>
    #1 dns_zone_markdirty lib/dns/zone.c:11085:2
    #2 update_action lib/ns/update.c:3376:3
    #3 dispatch lib/isc/task.c:1152:7
    #4 run lib/isc/task.c:1344:2

    Mutex M1 acquired here while holding mutex M2 in thread T2:
    #0 pthread_mutex_lock <null>
    #1 zone_nsec3chain lib/dns/zone.c:8274:3
    #2 zone_maintenance lib/dns/zone.c:11052:4
    #3 zone_timer lib/dns/zone.c:14087:2
    #4 dispatch lib/isc/task.c:1152:7
    #5 run lib/isc/task.c:1344:2

    Mutex M2 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 resume_iteration lib/dns/rbtdb.c:9357:2
    #3 dbiterator_next lib/dns/rbtdb.c:9647:3
    #4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
    #5 zone_nsec3chain lib/dns/zone.c:8412:13
    #6 zone_maintenance lib/dns/zone.c:11052:4
    #7 zone_timer lib/dns/zone.c:14087:2
    #8 dispatch lib/isc/task.c:1152:7
    #9 run lib/isc/task.c:1344:2

    Thread T1 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    Thread T2 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
 2020-09-17 07:03:56 +00:00
Mark Andrews
e185e37137 Pause dbiterator to release rwlock to prevent lock-order-inversion. 
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
    Cycle in lock order graph: M1 (0x000000000001) => M2 (0x000000000002) => M3 (0x000000000000) => M1

    Mutex M2 acquired here while holding mutex M1 in thread T1:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 findnodeintree lib/dns/rbtdb.c:2877:2
    #3 findnode lib/dns/rbtdb.c:2941:10
    #4 dns_db_findnode lib/dns/db.c:439:11
    #5 copy_non_dnssec_records lib/dns/zone.c:16031:11
    #6 receive_secure_db lib/dns/zone.c:16163:12
    #7 dispatch lib/isc/task.c:1152:7
    #8 run lib/isc/task.c:1344:2

    Mutex M1 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 resume_iteration lib/dns/rbtdb.c:9357:2
    #3 dbiterator_first lib/dns/rbtdb.c:9407:3
    #4 dns_dbiterator_first lib/dns/dbiterator.c:43:10
    #5 receive_secure_db lib/dns/zone.c:16160:16
    #6 dispatch lib/isc/task.c:1152:7
    #7 run lib/isc/task.c:1344:2

    Mutex M3 acquired here while holding mutex M2 in thread T2:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 zone_sign lib/dns/zone.c:9244:3
    #3 zone_maintenance lib/dns/zone.c:11044:4
    #4 zone_timer lib/dns/zone.c:14087:2
    #5 dispatch lib/isc/task.c:1152:7
    #6 run lib/isc/task.c:1344:2

    Mutex M2 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 resume_iteration lib/dns/rbtdb.c:9357:2
    #3 dbiterator_next lib/dns/rbtdb.c:9647:3
    #4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
    #5 zone_sign lib/dns/zone.c:9485:13
    #6 zone_maintenance lib/dns/zone.c:11044:4
    #7 zone_timer lib/dns/zone.c:14087:2
    #8 dispatch lib/isc/task.c:1152:7
    #9 run lib/isc/task.c:1344:2

    Mutex M1 acquired here while holding mutex M3 in thread T3:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 findnodeintree lib/dns/rbtdb.c:2877:2
    #3 findnode lib/dns/rbtdb.c:2941:10
    #4 dns_db_findnode lib/dns/db.c:439:11
    #5 zone_get_from_db lib/dns/zone.c:5602:11
    #6 get_raw_serial lib/dns/zone.c:2520:12
    #7 zone_gotwritehandle lib/dns/zone.c:2559:4
    #8 dispatch lib/isc/task.c:1152:7
    #9 run lib/isc/task.c:1344:2

    Mutex M3 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 zone_gotwritehandle lib/dns/zone.c:2552:2
    #3 dispatch lib/isc/task.c:1152:7
    #4 run lib/isc/task.c:1344:2

    Thread T1 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    Thread T2 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    Thread T3 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
 2020-09-17 07:03:56 +00:00
Mark Andrews
9e5f83c499 Address lock-order-inversion between the keytable and the db locks. 
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
    Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000000) => M1

    Mutex M2 acquired here while holding mutex M1 in thread T1:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 dns_keytable_find lib/dns/keytable.c:522:2
    #3 sync_keyzone lib/dns/zone.c:4560:12
    #4 dns_zone_synckeyzone lib/dns/zone.c:4635:11
    #5 mkey_refresh bin/named/server.c:15423:2
    #6 named_server_mkeys bin/named/server.c:15727:4
    #7 named_control_docommand bin/named/control.c:236:12
    #8 control_command bin/named/controlconf.c:365:17
    #9 dispatch lib/isc/task.c:1152:7
    #10 run lib/isc/task.c:1344:2

    Mutex M1 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 resume_iteration lib/dns/rbtdb.c:9357:2
    #3 dbiterator_first lib/dns/rbtdb.c:9407:3
    #4 dns_dbiterator_first lib/dns/dbiterator.c:43:10
    #5 dns_rriterator_first lib/dns/rriterator.c:71:15
    #6 sync_keyzone lib/dns/zone.c:4543:16
    #7 dns_zone_synckeyzone lib/dns/zone.c:4635:11
    #8 mkey_refresh bin/named/server.c:15423:2
    #9 named_server_mkeys bin/named/server.c:15727:4
    #10 named_control_docommand bin/named/control.c:236:12
    #11 control_command bin/named/controlconf.c:365:17
    #12 dispatch lib/isc/task.c:1152:7
    #13 run lib/isc/task.c:1344:2

    Mutex M1 acquired here while holding mutex M2 in thread T1:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 zone_find lib/dns/rbtdb.c:4029:2
    #3 dns_db_find lib/dns/db.c:500:11
    #4 addifmissing lib/dns/zone.c:4481:11
    #5 dns_keytable_forall lib/dns/keytable.c:786:4
    #6 sync_keyzone lib/dns/zone.c:4586:2
    #7 dns_zone_synckeyzone lib/dns/zone.c:4635:11
    #8 mkey_refresh bin/named/server.c:15423:2
    #9 named_server_mkeys bin/named/server.c:15727:4
    #10 named_control_docommand bin/named/control.c:236:12
    #11 control_command bin/named/controlconf.c:365:17
    #12 dispatch lib/isc/task.c:1152:7
    #13 run lib/isc/task.c:1344:2

    Mutex M2 previously acquired by the same thread here:
    #0 pthread_rwlock_rdlock <null>
    #1 isc_rwlock_lock lib/isc/rwlock.c:48:3
    #2 dns_keytable_forall lib/dns/keytable.c:770:2
    #3 sync_keyzone lib/dns/zone.c:4586:2
    #4 dns_zone_synckeyzone lib/dns/zone.c:4635:11
    #5 mkey_refresh bin/named/server.c:15423:2
    #6 named_server_mkeys bin/named/server.c:15727:4
    #7 named_control_docommand bin/named/control.c:236:12
    #8 control_command bin/named/controlconf.c:365:17
    #9 dispatch lib/isc/task.c:1152:7
    #10 run lib/isc/task.c:1344:2

    Thread T1 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
 2020-09-17 07:03:56 +00:00
Michał Kępień
6be91b9d99 Update library API versions 2020-09-16 22:34:06 +02:00
Michał Kępień
5ae33351f2 Deprecate the "glue-cache" option 
No issues with the glue cache feature have been reported since its
introduction in BIND 9.12.  As the rationale for introducing the
"glue-cache" option was to have a safety switch readily available in
case the glue cache turns out to cause problems, it is time to deprecate
the option.  Glue cache will be permanently enabled in a future release,
at which point the "glue-cache" option will be made obsolete.
 2020-09-16 11:18:07 +02:00
Ondřej Surý
7b07f22969 Fix the clang 12 warnings with multi-line strings in string arrays 
The clang 12 has a new warning that warns when using multi-line strings
in the string arrays, f.e.:

    { "aa",
      "b"
      "b",
      "cc" }

would generate warning like this:

    private_test.c:162:7: error: suspicious concatenation of string literals in an array initialization; did you mean to separate the elements with a comma? [-Werror,-Wstring-concatenation]
                                      "33333/RSASHA1" };
                                      ^
    private_test.c:161:7: note: place parentheses around the string literal to silence warning
                                      "Done removing signatures for key "
                                      ^
    private_test.c:197:7: error: suspicious concatenation of string literals in an array initialization; did you mean to separate the elements with a comma? [-Werror,-Wstring-concatenation]
                                      "NSEC chain",
                                      ^
    private_test.c:196:7: note: place parentheses around the string literal to silence warning
                                      "Removing NSEC3 chain 1 0 30 DEAF / creating "
                                      ^
    2 errors generated.
 2020-09-16 11:01:39 +02:00
Evan Hunt
dcee985b7f update all copyright headers to eliminate the typo 2020-09-14 16:20:40 -07:00
Mark Andrews
baf165ffd0 clear pointer before subtracting reference 2020-09-14 11:02:33 +10:00
Witold Kręcicki
00e04a86c8 tracing of active sockets and handles 
If NETMGR_TRACE is defined, we now maintain a list of active sockets
in the netmgr object and a list of active handles in each socket
object; by walking the list and printing `backtrace` in a debugger
we can see where they were created, to assist in in debugging of
reference counting errors.

On shutdown, if netmgr finds there are still active sockets after
waiting, isc__nm_dump_active() will be called to log the list of
active sockets and their underlying handles, along with some details
about them.
 2020-09-11 12:17:57 -07:00
Evan Hunt
2f2d60a989 limit the time we wait for netmgr to be destroyed 
if more than 10 seconds pass while we wait for netmgr events to
finish running on shutdown, something is almost certainly wrong
and we should assert and crash.
 2020-09-11 12:17:57 -07:00
Ondřej Surý
89c534d3b9 properly lock the setting/unsetting of callbacks in isc_nmsocket_t 
changes to socket callback functions were not thread safe.
 2020-09-11 12:17:57 -07:00
Evan Hunt
57b4dde974 change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach 
Attaching and detaching handle pointers will make it easier to
determine where and why reference counting errors have occurred.

A handle needs to be referenced more than once when multiple
asynchronous operations are in flight, so callers must now maintain
multiple handle pointers for each pending operation. For example,
ns_client objects now contain:

        - reqhandle:    held while waiting for a request callback (query,
                        notify, update)
        - sendhandle:   held while waiting for a send callback
        - fetchhandle:  held while waiting for a recursive fetch to
                        complete
        - updatehandle: held while waiting for an update-forwarding
                        task to complete

control channel connection objects now contain:

        - readhandle: held while waiting for a read callback
        - sendhandle: held while waiting for a send callback
        - cmdhandle:  held while an rndc command is running

httpd connections contain:

        - readhandle: held while waiting for a read callback
        - sendhandle: held while waiting for a send callback
 2020-09-11 12:17:57 -07:00
Witold Kręcicki
7eb4564895 assorted small netmgr-related changes 
- rename isc_nmsocket_t->tcphandle to statichandle
- cancelread functions now take handles instead of sockets
- add a 'client' flag in socket objects, currently unused, to
  indicate whether it is to be used as a client or server socket
 2020-09-11 10:24:36 -07:00
Mark Andrews
584dbffab1 Silence REVERSE_INULL warning (CID 306652) 2020-09-09 21:01:55 +00:00
Mark Andrews
7b3c7f52c2 Turn off TSAN for isc_log_wouldlog 2020-09-09 14:12:29 +10:00
Mark Andrews
71ef3a8038 Only test node->data if we care about whether data is present or not. 
WARNING: ThreadSanitizer: data race (pid=28788)
  Write of size 8 at 0x7b200002e060 by thread T1 (mutexes: write M2947):
    #0 add32 /builds/isc-projects/bind9/lib/dns/rbtdb.c:6638:18 (libdns.so.1110+0xe7843)
    #1 addrdataset /builds/isc-projects/bind9/lib/dns/rbtdb.c:6975:12 (libdns.so.1110+0xe4185)
    #2 dns_db_addrdataset /builds/isc-projects/bind9/lib/dns/db.c:783:10 (libdns.so.1110+0x650ee)
    #3 validated /builds/isc-projects/bind9/lib/dns/resolver.c:5140:11 (libdns.so.1110+0x1909f7)
    #4 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
    #5 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)

  Previous read of size 8 at 0x7b200002e060 by thread T5 (mutexes: write M521146194917735760):
    #0 dns_rbt_findnode /builds/isc-projects/bind9/lib/dns/rbt.c:1708:9 (libdns.so.1110+0xd910d)
    #1 cache_find /builds/isc-projects/bind9/lib/dns/rbtdb.c:5098:11 (libdns.so.1110+0xe188e)
    #2 dns_db_find /builds/isc-projects/bind9/lib/dns/db.c:554:11 (libdns.so.1110+0x642bb)
    #3 dns_view_find2 /builds/isc-projects/bind9/lib/dns/view.c:1068:11 (libdns.so.1110+0x1cc2c4)
    #4 dbfind_name /builds/isc-projects/bind9/lib/dns/adb.c:3714:11 (libdns.so.1110+0x46a4b)
    #5 dns_adb_createfind2 /builds/isc-projects/bind9/lib/dns/adb.c:3133:12 (libdns.so.1110+0x45278)
    #6 findname /builds/isc-projects/bind9/lib/dns/resolver.c:3166:11 (libdns.so.1110+0x1827f0)
    #7 fctx_getaddresses /builds/isc-projects/bind9/lib/dns/resolver.c:3462:3 (libdns.so.1110+0x18032d)
    #8 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3819:12 (libdns.so.1110+0x17e174)
    #9 fctx_start /builds/isc-projects/bind9/lib/dns/resolver.c:4219:4 (libdns.so.1110+0x1787a3)
    #10 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
    #11 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
 2020-09-09 14:12:29 +10:00
Mark Andrews
992a79a14b Address lock-order-inversion 
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=12714)
  Cycle in lock order graph: M100252 (0x7b7c00010a08) => M1171 (0x7b7400000dc8) => M100252

  Mutex M1171 acquired here while holding mutex M100252 in thread T1:
    #0 pthread_mutex_lock <null> (delv+0x4483a6)
    #1 dns_resolver_createfetch3 /builds/isc-projects/bind9/lib/dns/resolver.c:9585:2 (libdns.so.1110+0x1769fd)
    #2 dns_resolver_createfetch /builds/isc-projects/bind9/lib/dns/resolver.c:9504:10 (libdns.so.1110+0x174e17)
    #3 create_fetch /builds/isc-projects/bind9/lib/dns/validator.c:1156:10 (libdns.so.1110+0x1c1e5f)
    #4 validatezonekey /builds/isc-projects/bind9/lib/dns/validator.c:2124:13 (libdns.so.1110+0x1c3b6d)
    #5 start_positive_validation /builds/isc-projects/bind9/lib/dns/validator.c:2301:10 (libdns.so.1110+0x1bfde9)
    #6 validator_start /builds/isc-projects/bind9/lib/dns/validator.c:3647:12 (libdns.so.1110+0x1bef62)
    #7 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
    #8 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)

  Mutex M100252 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null> (delv+0x4483a6)
    #1 validator_start /builds/isc-projects/bind9/lib/dns/validator.c:3628:2 (libdns.so.1110+0x1bee31)
    #2 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
    #3 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)

  Mutex M100252 acquired here while holding mutex M1171 in thread T1:
    #0 pthread_mutex_lock <null> (delv+0x4483a6)
    #1 dns_validator_destroy /builds/isc-projects/bind9/lib/dns/validator.c:3912:2 (libdns.so.1110+0x1bf788)
    #2 validated /builds/isc-projects/bind9/lib/dns/resolver.c:4916:2 (libdns.so.1110+0x18fdfd)
    #3 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
    #4 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)

  Mutex M1171 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null> (delv+0x4483a6)
    #1 validated /builds/isc-projects/bind9/lib/dns/resolver.c:4907:2 (libdns.so.1110+0x18fc3d)
    #2 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
    #3 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)

  Thread T1 'isc-worker0000' (tid=12729, running) created by main thread at:
    #0 pthread_create <null> (delv+0x42afdb)
    #1 isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:60:8 (libisc.so.1107+0x726d8)
    #2 isc__taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1468:7 (libisc.so.1107+0x4d635)
    #3 isc_taskmgr_createinctx /builds/isc-projects/bind9/lib/isc/task.c:2091:11 (libisc.so.1107+0x4f4ac)
    #4 main /builds/isc-projects/bind9/bin/delv/delv.c:1639:2 (delv+0x4b7f96)

SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/builds/isc-projects/bind9/bin/delv/.libs/delv+0x4483a6) in pthread_mutex_lock
 2020-09-09 14:08:33 +10:00
Mark Andrews
cdcfde9e65 Address lock-order-inversion 
Obtain references to view->redirect and view->managed_keys then
release view->lock so dns_zone_setviewcommit and dns_zone_setviewrevert
can obtain the view->lock while holding zone->lock.

WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=9132)
  Cycle in lock order graph: M987831431424375936 (0x000000000000) => M1012319771577875480 (0x000000000000) => M987831431424375936

  Mutex M1012319771577875480 acquired here while holding mutex M987831431424375936 in thread T2:
    #0 pthread_mutex_lock <null> (named+0x4642a6)
    #1 dns_zone_setviewcommit /builds/isc-projects/bind9/lib/dns/zone.c:1571:2 (libdns.so.1110+0x1d74eb)
    #2 dns_view_setviewcommit /builds/isc-projects/bind9/lib/dns/view.c:2388:3 (libdns.so.1110+0x1cfe29)
    #3 load_configuration /builds/isc-projects/bind9/bin/named/./server.c:8188:3 (named+0x51eadd)
    #4 loadconfig /builds/isc-projects/bind9/bin/named/./server.c:9438:11 (named+0x510c66)
    #5 ns_server_reconfigcommand /builds/isc-projects/bind9/bin/named/./server.c:9773:2 (named+0x510b41)
    #6 ns_control_docommand /builds/isc-projects/bind9/bin/named/control.c:243:12 (named+0x4e451a)
    #7 control_recvmessage /builds/isc-projects/bind9/bin/named/controlconf.c:465:13 (named+0x4e9056)
    #8 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
    #9 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)

  Mutex M987831431424375936 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null> (named+0x4642a6)
    #1 dns_view_setviewcommit /builds/isc-projects/bind9/lib/dns/view.c:2382:2 (libdns.so.1110+0x1cfde7)
    #2 load_configuration /builds/isc-projects/bind9/bin/named/./server.c:8188:3 (named+0x51eadd)
    #3 loadconfig /builds/isc-projects/bind9/bin/named/./server.c:9438:11 (named+0x510c66)
    #4 ns_server_reconfigcommand /builds/isc-projects/bind9/bin/named/./server.c:9773:2 (named+0x510b41)
    #5 ns_control_docommand /builds/isc-projects/bind9/bin/named/control.c:243:12 (named+0x4e451a)
    #6 control_recvmessage /builds/isc-projects/bind9/bin/named/controlconf.c:465:13 (named+0x4e9056)
    #7 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
    #8 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)

  Mutex M987831431424375936 acquired here while holding mutex M1012319771577875480 in thread T7:
    #0 pthread_mutex_lock <null> (named+0x4642a6)
    #1 dns_view_findzonecut2 /builds/isc-projects/bind9/lib/dns/view.c:1300:2 (libdns.so.1110+0x1cc93a)
    #2 dns_view_findzonecut /builds/isc-projects/bind9/lib/dns/view.c:1261:9 (libdns.so.1110+0x1cc864)
    #3 fctx_create /builds/isc-projects/bind9/lib/dns/resolver.c:4459:13 (libdns.so.1110+0x1779d3)
    #4 dns_resolver_createfetch3 /builds/isc-projects/bind9/lib/dns/resolver.c:9628:12 (libdns.so.1110+0x176cb6)
    #5 dns_resolver_createfetch /builds/isc-projects/bind9/lib/dns/resolver.c:9504:10 (libdns.so.1110+0x174e17)
    #6 zone_refreshkeys /builds/isc-projects/bind9/lib/dns/zone.c:10061:12 (libdns.so.1110+0x2055a5)
    #7 zone_maintenance /builds/isc-projects/bind9/lib/dns/zone.c:10274:5 (libdns.so.1110+0x203a78)
    #8 zone_timer /builds/isc-projects/bind9/lib/dns/zone.c:13106:2 (libdns.so.1110+0x1e815a)
    #9 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
    #10 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)

  Mutex M1012319771577875480 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null> (named+0x4642a6)
    #1 zone_refreshkeys /builds/isc-projects/bind9/lib/dns/zone.c:9951:2 (libdns.so.1110+0x204dc3)
    #2 zone_maintenance /builds/isc-projects/bind9/lib/dns/zone.c:10274:5 (libdns.so.1110+0x203a78)
    #3 zone_timer /builds/isc-projects/bind9/lib/dns/zone.c:13106:2 (libdns.so.1110+0x1e815a)
    #4 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
    #5 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)

  Thread T2 'isc-worker0001' (tid=9163, running) created by main thread at:
    #0 pthread_create <null> (named+0x446edb)
    #1 isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:60:8 (libisc.so.1107+0x726d8)
    #2 isc__taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1468:7 (libisc.so.1107+0x4d635)
    #3 isc_taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:2109:11 (libisc.so.1107+0x4f587)
    #4 create_managers /builds/isc-projects/bind9/bin/named/./main.c:886:11 (named+0x4f1a97)
    #5 setup /builds/isc-projects/bind9/bin/named/./main.c:1305:11 (named+0x4f05ee)
    #6 main /builds/isc-projects/bind9/bin/named/./main.c:1556:2 (named+0x4ef12d)

  Thread T7 'isc-worker0006' (tid=9168, running) created by main thread at:
    #0 pthread_create <null> (named+0x446edb)
    #1 isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:60:8 (libisc.so.1107+0x726d8)
    #2 isc__taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1468:7 (libisc.so.1107+0x4d635)
    #3 isc_taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:2109:11 (libisc.so.1107+0x4f587)
    #4 create_managers /builds/isc-projects/bind9/bin/named/./main.c:886:11 (named+0x4f1a97)
    #5 setup /builds/isc-projects/bind9/bin/named/./main.c:1305:11 (named+0x4f05ee)
    #6 main /builds/isc-projects/bind9/bin/named/./main.c:1556:2 (named+0x4ef12d)

SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/builds/isc-projects/bind9/bin/named/.libs/named+0x4642a6) in pthread_mutex_lock
 2020-09-09 14:08:33 +10:00
Mark Andrews
552e0b852e isc_mutex_init_errcheck prototype should not be under ISC_MUTEX_PROFILE 2020-09-09 13:58:31 +10:00
Michał Kępień
549ddca256 Include BIND documentation in Windows zips 
As generated documentation files are no longer stored in the BIND Git
repository, put a copy of the PDF version of the BIND ARM generated by
the "docs" GitLab CI job into the Windows zips to make it easily
available to the end users on that platform.

Make sure Windows zips also contain certain documentation files included
in source tarballs to make the contents of each release more consistent
across different platforms.
 2020-09-03 10:35:37 +02:00
Ondřej Surý
0a22024c27 Print diagnostics on dns_name_issubdomain() failure in fctx_create() 
Log diagnostic message when dns_name_issubdomain() in the fctx_create()
when the resolver is qname minimizing and forwarding at the same time.
 2020-09-02 18:08:15 +02:00
Mark Andrews
22f499cdc4 watch_fd also requires thread->fdlock[lockid] to be held 2020-09-02 12:17:48 +00:00
Mark Andrews
e923e62f6c remove dead code 2020-09-02 12:17:48 +00:00
Matthijs Mekking
f9ef5120c1 Log when CDS/CDNSKEY is published in zone. 
Log when named decides to add a CDS/CDNSKEY record to the zone. Now
you understand how the bug was found that was fixed in the previous
commits.
 2020-09-02 12:00:16 +02:00
Matthijs Mekking
c8205bfa0e Fix CDS (non-)publication 
The CDS/CDNSKEY record will be published when the DS is in the
rumoured state. However, with the introduction of the rndc '-checkds'
command, the logic in the keymgr was changed to prevent the DS
state to go in RUMOURED unless the specific command was given. Hence,
the CDS was never published before it was seen in the parent.

Initially I thought this was a policy approval rule, however it is
actually a DNSSEC timing rule. Remove the restriction from
'keymgr_policy_approval' and update the 'keymgr_transition_time'
function. When looking to move the DS state to OMNIPRESENT it will
no longer calculate the state from its last change, but from when
the DS was seen in the parent, "DS Publish". If the time was not set,
default to next key event of an hour.

Similarly for moving the DS state to HIDDEN, the time to wait will
be derived from the "DS Delete" time, not from when the DS state
last changed.
 2020-09-02 12:00:14 +02:00
Mark Andrews
b8e4b6d303 isc_ratelimiter needs to hold a reference to its task 
to prevent the task subsystem shutting down before the
ratelimiter is freed.
 2020-09-02 10:32:33 +10:00
Mark Andrews
b1c424ddf3 remove unused variable sock 2020-09-01 22:24:32 +00:00
Diego Fronza
230d79c191 Fix resolution of unusual ip6.arpa names 
Before this commit, BIND was unable to resolve ip6.arpa names like
the one reported in issue #1847 when using query minimization.

As reported in the issue, an attempt to resolve a name like
'rec-test-dom-158937817846788.test123.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.3.4.3.5.4.0.8.2.6.0.1.0.0.2.ip6.arpa'
using default settings would fail.

The reason was that query minimization algorithm in 'fctx_minimize_qname'
would divide any ip6.arpa names in increasing number of labels,
7,11, ... up to 35, thus limiting the destination name (minimized) to a number
of 35 labels.

In case the last query minimization attempt (with 35 labels) would fail with
NXDOMAIN, BIND would attempt the query mininimization again with the exact
same QNAME, limited on the 35 labels, and that in turn would fail again.

This fix avoids this fail loop by considering the extra labels that may appear
in the leftmost part of an ip6.arpa name, those after the IPv6 part.
 2020-09-01 15:47:00 -03:00
Mark Andrews
6278899a38 Use memory_order_acq_rel in isc_refcount_decrement. 
While

if (isc_refcount_decrement() == 1) {	// memory_order_release
	isc_refcount_destroy();		// memory_order_acquire
	...
}

is theoretically the most efficent in practice, using
memory_order_acq_rel produces the same code on x86_64 and doesn't
trigger tsan data races (which use a idealistic model) if
isc_refcount_destroy() is not called immediately.  In fact
isc_refcount_destroy() could be removed if we didn't want
to check for the count being 0 when isc_refcount_destroy() is
called.

https://stackoverflow.com/questions/49112732/memory-order-in-shared-pointer-destructor
 2020-09-01 12:05:18 +00:00
Ondřej Surý
e0380d437d Handle EPROTO errno from recvmsg 
It was discovered, that some systems might set EPROTO instead of EACCESS
on recvmsg() call causing spurious syslog messages from the socket
code.  This commit returns soft handling of EPROTO errno code to the
socket code. [GL #1928]
 2020-08-28 20:45:52 +02:00
Ondřej Surý
78543ad5a7 Fix off-by-one error when calculating new hashtable size 
When calculating the new hashtable bitsize, there was an off-by-one
error that would allow the new bitsize to be larger than maximum allowed
causing assertion failure in the rehash() function.
 2020-08-28 16:21:21 +02:00
Mark Andrews
2ca4d35037 Refactor totext_loc 2020-08-26 15:31:31 +02:00