Commit graph

3864 commits

Author SHA1 Message Date
Ondřej Surý
0ad77036d9 Handle the errors from sysconf() call in isc_meminfo_totalphys()
isc_meminfo_totalphys() would return invalid memory size when sysconf()
call would fail, because ((size_t)-1 * -1) is very large number.

(cherry picked from commit 79ca724d46)
2020-09-21 10:58:37 +02:00
Ondřej Surý
8b1e4a5373 Exclude isc_mem_isovermem from ThreadSanitizer
The .is_overmem member of isc_mem_t structure is intentionally accessed
unlocked as 100% accuracy isn't necessary here.

Without the attribute, following TSAN warning would show up:

    WARNING: ThreadSanitizer: data race
      Write of size 1 at 0x000000000001 by thread T1 (mutexes: write M1, write M2):
	#0 isc___mem_put lib/isc/mem.c:1119:19
	#1 isc__mem_put lib/isc/mem.c:2439:2
	#2 dns_rdataslab_fromrdataset lib/dns/rdataslab.c:327:2
	#3 addrdataset lib/dns/rbtdb.c:6761:11
	#4 dns_db_addrdataset lib/dns/db.c:719:10
	#5 cache_name lib/dns/resolver.c:6538:13
	#6 cache_message lib/dns/resolver.c:6628:14
	#7 resquery_response lib/dns/resolver.c:7883:13
	#8 dispatch lib/isc/task.c:1152:7
	#9 run lib/isc/task.c:1344:2

      Previous read of size 1 at 0x000000000001 by thread T2 (mutexes: write M3):
	#0 isc_mem_isovermem lib/isc/mem.c:1553:15
	#1 addrdataset lib/dns/rbtdb.c:6866:25
	#2 dns_db_addrdataset lib/dns/db.c:719:10
	#3 addoptout lib/dns/ncache.c:281:10
	#4 dns_ncache_add lib/dns/ncache.c:101:10
	#5 ncache_adderesult lib/dns/resolver.c:6668:12
	#6 ncache_message lib/dns/resolver.c:6845:11
	#7 rctx_ncache lib/dns/resolver.c:9174:11
	#8 resquery_response lib/dns/resolver.c:7894:2
	#9 dispatch lib/isc/task.c:1152:7
	#10 run lib/isc/task.c:1344:2

      Location is heap block of size 328 at 0x000000000020 allocated by thread T3:
	#0 malloc <null>
	#1 default_memalloc lib/isc/mem.c:713:8
	#2 mem_create lib/isc/mem.c:763:8
	#3 isc_mem_create lib/isc/mem.c:2425:2
	#4 configure_view bin/named/server.c:4494:4
	#5 load_configuration bin/named/server.c:9062:3
	#6 run_server bin/named/server.c:9771:2
	#7 dispatch lib/isc/task.c:1152:7
	#8 run lib/isc/task.c:1344:2

    [...]

    SUMMARY: ThreadSanitizer: data race lib/isc/mem.c:1119:19 in isc___mem_put

(cherry picked from commit 0110d1ab17)
2020-09-17 17:35:58 +02:00
Tinderbox User
a4f73cfe8a prep 9.16.7 2020-09-16 22:50:38 +02:00
Evan Hunt
df698d73f4 update all copyright headers to eliminate the typo 2020-09-14 16:50:58 -07:00
Mark Andrews
2c1b6b2168 Silence REVERSE_INULL warning (CID 306652)
(cherry picked from commit 584dbffab1)
2020-09-10 07:54:19 +10:00
Mark Andrews
c359fa0933 Turn off TSAN for isc_log_wouldlog
(cherry picked from commit 7b3c7f52c2)
2020-09-09 16:22:39 +10:00
Mark Andrews
5b425046dd isc_mutex_init_errcheck prototype should not be under ISC_MUTEX_PROFILE
(cherry picked from commit 552e0b852e)
2020-09-09 16:22:38 +10:00
Michał Kępień
6e2a35df2d Include BIND documentation in Windows zips
As generated documentation files are no longer stored in the BIND Git
repository, put a copy of the PDF version of the BIND ARM generated by
the "docs" GitLab CI job into the Windows zips to make it easily
available to the end users on that platform.

Make sure Windows zips also contain certain documentation files included
in source tarballs to make the contents of each release more consistent
across different platforms.

(cherry picked from commit 549ddca256)
2020-09-03 12:02:19 +02:00
Mark Andrews
e6332e4a67 watch_fd also requires thread->fdlock[lockid] to be held
(cherry picked from commit 22f499cdc4)
2020-09-03 07:14:45 +10:00
Mark Andrews
eadfe4b673 remove dead code
(cherry picked from commit e923e62f6c)
2020-09-03 07:14:45 +10:00
Mark Andrews
e460d83dbb isc_ratelimiter needs to hold a reference to its task
to prevent the task subsystem shutting down before the
ratelimiter is freed.

(cherry picked from commit b8e4b6d303)
2020-09-02 11:39:36 +10:00
Mark Andrews
1af9cf78bd Use memory_order_acq_rel in isc_refcount_decrement.
While

if (isc_refcount_decrement() == 1) {	// memory_order_release
	isc_refcount_destroy();		// memory_order_acquire
	...
}

is theoretically the most efficent in practice, using
memory_order_acq_rel produces the same code on x86_64 and doesn't
trigger tsan data races (which use a idealistic model) if
isc_refcount_destroy() is not called immediately.  In fact
isc_refcount_destroy() could be removed if we didn't want
to check for the count being 0 when isc_refcount_destroy() is
called.

https://stackoverflow.com/questions/49112732/memory-order-in-shared-pointer-destructor
(cherry picked from commit 6278899a38)
2020-09-01 22:24:52 +10:00
Ondřej Surý
9b9fee13fa Handle EPROTO errno from recvmsg
It was discovered, that some systems might set EPROTO instead of EACCESS
on recvmsg() call causing spurious syslog messages from the socket
code.  This commit returns soft handling of EPROTO errno code to the
socket code. [GL #1928]

(cherry picked from commit e0380d437d)
2020-08-28 20:49:01 +02:00
Michal Nowak
79e8f1076a
Fix warnings in when build with --enable-buffer-useinline
sockaddr.c:147:49: error: pointer targets in passing argument 2 of ‘isc__buffer_putmem’ differ in signedness
    rdata.c:1780:30: error: pointer targets in passing argument 2 of ‘isc__buffer_putmem’ differ in signedness

(cherry picked from commit dd425254a7)
2020-08-25 16:08:44 +02:00
Mark Andrews
da4f189ea8 Add missing isc_mutex_init to manytasks subtest.
(cherry picked from commit 2eb5c29c83)
2020-08-25 09:58:30 +10:00
Evan Hunt
1c7e3c8515 BIND 9.16.6
-----BEGIN PGP SIGNATURE-----
 
 iQJDBAABCgAtFiEENKwGS3ftSQfs1TU17QVz/8hFYQUFAl8xFCMPHG1pY2hhbEBp
 c2Mub3JnAAoJEO0Fc//IRWEFM/IP/AyKCPJsh+grYskFAws5UqhvDM0XBvQWPZBP
 DM0lKA2BK8vaOl80iI2hlf9SrLMmBiD0f3WHigbS8i0MbnJAz0T7mFDuDmcJQSP4
 skwgwX3obpDwdzl/Tgu2V5bTbwe0WT8wdUKIT8oZnPBNuhh57hjQh3D+DTZ+YPye
 RSPq1lYtQq80QukHkzQ9JnpMzW8JYZTbwzG4swZNl4upbI+Z3Jx93LYnNBCKZuw1
 jlrrFAQZavFdQJ4fxnLicHYsnmfWOX3Lhg/wSHemwMeYgxlrWYXgKCDO+0veB1Sq
 rGVnrfmkN7fNDY9gPJOe7TBPkKLtlSs5zxPNDvfYiDUGhdNTIV/hItF+u81eKetu
 QLp0RNa+uPuCkUGe9bBcqEJ0DIVL7yfzrtxrvtceLKG6A9XIF69nfPl81dv2GjHn
 hR81v/VQC8H2fzzbfypJpTwXeAJ3HKeBahoQttwMH7hux3iatOFdKb1pulkviK0d
 fzX5TSWHK2JLyCH0ed1SPzJFYG9irGl0lYuQIO1cbmb7IZOAMWJODXpafJiJwBpR
 YgHenf+XS1bluadl6kItA2QhLsMnly+LfYO9XXhGMmIqE8Xf1RrHLCIts/hQjY0/
 B+lRvWAXvzLMb+y+W+wxe8BsNSI/RYhHxXsJRavrlCPeFeSg5CMOu4VXTkdnUKcV
 tDQuUJBh
 =p//M
 -----END PGP SIGNATURE-----

Merge tag 'v9_16_6' into v9_16

BIND 9.16.6
2020-08-20 12:08:57 -07:00
Mark Andrews
5806d856cf base32_decode*() could incorrectly decode a input.
base32_decode_char() added a extra zero octet to the output
if the fifth character was a pad character.  The length
of octets to copy to the output was set to 3 instead of 2.

(cherry picked from commit 6c7e50c267)
2020-08-14 00:11:06 +10:00
Tinderbox User
a195123ad0 prep 9.16.6 2020-08-06 08:14:40 +00:00
Ondřej Surý
ac3862a5da Fix crash in pk11_numbits() when native-pkcs11 is used
When pk11_numbits() is passed a user provided input that contains all
zeroes (via crafted DNS message), it would crash with assertion
failure.  Fix that by properly handling such input.
2020-08-05 15:51:40 +02:00
Evan Hunt
9a372f2bce Use different allocators for UDP and TCP
Each worker has a receive buffer with space for 20 DNS messages of up
to 2^16 bytes each, and the allocator function passed to uv_read_start()
or uv_udp_recv_start() will reserve a portion of it for use by sockets.
UDP can use recvmmsg() and so it needs that entire space, but TCP reads
one message at a time.

This commit introduces separate allocator functions for TCP and UDP
setting different buffer size limits, so that libuv will provide the
correct buffer sizes to each of them.
2020-08-05 12:57:58 +02:00
Witold Kręcicki
a12076cc52 netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned.
When a new IPv6 interface/address appears it's first in a tentative
state - in which we cannot bind to it, yet it's already being reported
by the route socket. Because of that BIND9 is unable to listen on any
newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or
equivalent option on other OSes) and then retrying bind() call.

(cherry picked from commit a0f7d28967)
2020-07-31 13:33:06 +02:00
Mark Andrews
14fe6e77a7 Always check the return from isc_refcount_decrement.
Created isc_refcount_decrement_expect macro to test conditionally
the return value to ensure it is in expected range.  Converted
unchecked isc_refcount_decrement to use isc_refcount_decrement_expect.
Converted INSIST(isc_refcount_decrement()...) to isc_refcount_decrement_expect.

(cherry picked from commit bde5c7632a)
2020-07-31 12:54:47 +10:00
Mark Andrews
1981fb1327 Refactor the code that counts the last log version to keep
When silencing the Coverity warning in remove_old_tsversions(), the code
was refactored to reduce the indentation levels and break down the long
code into individual functions.  This improve fix for [GL #1989].

(cherry picked from commit aca18b8b5b)
2020-07-31 10:01:36 +10:00
Ondřej Surý
ebb2b055cc Add isc_hash32() and rename isc_hash_function() to isc_hash64()
As the names suggest the original isc_hash64 function returns 64-bit
long hash values and the isc_hash32() returns 32-bit values.

(cherry picked from commit f59fd49fd8)
2020-07-30 11:57:24 +02:00
Ondřej Surý
1e5df7f3bf Add HalfSipHash 2-4 reference implementation
The HalfSipHash implementation has 32-bit keys and returns 32-bit
value.

(cherry picked from commit 344d66aaff)
2020-07-30 11:57:24 +02:00
Ondřej Surý
d89eb403f3 Remove OpenSSL based SipHash 2-4 implementation
Creation of EVP_MD_CTX and EVP_PKEY is quite expensive, so until we fix the code
to reuse the OpenSSL contexts and keys we'll use our own implementation of
siphash instead of trying to integrate with OpenSSL.

(cherry picked from commit 21d751dfc7)
2020-07-30 11:57:24 +02:00
Petr Menšík
ac79d68765 Remove few lines in unix socket handling
Reuse the same checks two times, make difference minimal.

(cherry picked from commit 72d81c4768)
2020-07-24 13:47:26 +10:00
Tinderbox User
b03a635f68 prep 9.16.5 2020-07-15 23:10:55 +02:00
Mark Andrews
d47c42a0ab Mark 'addr' as unused if HAVE_IF_NAMETOINDEX is not defined
Also 'zone' should be initialised to zero.

(cherry picked from commit e7662c4c63)
2020-07-14 10:53:06 +10:00
Mark Andrews
f771d75c9b Address potential double unlock in process_fd
(cherry picked from commit cc0089c66b)
2020-07-14 07:35:17 +10:00
Mark Andrews
94288631a9 Add changes for [GL #1989]
(cherry picked from commit 42b2290c3a)
2020-07-13 14:04:53 +10:00
Mark Andrews
67f85d648f Address overrun in remove_old_tsversions
If too many versions of log / dnstap files to be saved where requests
the memory after to_keep could be overwritten.  Force the number of
versions to be saved to a save level.  Additionally the memmove length
was incorrect.

(cherry picked from commit 6ca78bc57d)
2020-07-13 14:04:04 +10:00
Mark Andrews
dd32fb9284 Make the stdatomic shim and mutexatomic type complete
The stdatomic shims for non-C11 compilers (Windows, old gcc, ...) and
mutexatomic implemented only and minimal subset of the atomic types.
This commit adds 16-bit operations for Windows and all atomic types as
defined in standard.

(cherry picked from commit bccea5862d)
2020-07-08 10:29:59 +10:00
Matthijs Mekking
7915327aac Move dst key printtime in separate function
I'd like to use the same functionality (pretty print the datetime
of keytime metadata) in the 'rndc dnssec -status' command.  So it is
better that this logic is done in a separate function.

Since the stdtime.c code have differernt files for unix and win32,
I think the "#ifdef WIN32" define can be dropped.

(cherry picked from commit 9e03f8e8fe)
2020-07-01 09:57:44 +02:00
Evan Hunt
952461b6af restore "blackhole" functionality
the blackhole ACL was accidentally disabled with respect to client
queries during the netmgr conversion.

in order to make this work for TCP, it was necessary to add a return
code to the accept callback functions passed to isc_nm_listentcp() and
isc_nm_listentcpdns().

(cherry picked from commit 23c7373d68)
2020-06-30 21:10:31 -07:00
Witold Kręcicki
4582ef3bb2 Fix a shutdown race in netmgr udp.
We need to mark the socket as inactive early (and synchronously)
in the stoplistening process - otherwise we might destroy the
callback argument before actually stopping listening, and call
the callback on a bad memory.
2020-06-26 01:44:03 -07:00
Witold Kręcicki
97e44fa3df Make netmgr tcpdns send calls asynchronous.
isc__nm_tcpdns_send() was not asynchronous and accessed socket
internal fields in an unsafe manner, which could lead to a race
condition and subsequent crash. Fix it by moving the whole tcpdns
processing to a proper netmgr thread.
2020-06-26 01:18:27 -07:00
Mark Andrews
3f48a1e06e Add INSIST's to silence cppcheck warnings
(cherry picked from commit 0cf25d7f38)
2020-06-25 21:13:17 +10:00
Mark Andrews
b43641a55d Address potential thread issues:
Assign and then check node for NULL to address another thread
changing radix->head in the meantime.

Move 'node != NULL' check into while loop test to silence cppcheck
false positive.

Fix pointer != NULL style.

(cherry picked from commit 51f08d2095)
2020-06-25 21:11:27 +10:00
Tinderbox User
adab85b815 prep 9.16.4 2020-06-18 10:25:50 +02:00
Ondřej Surý
8b4fe6c6c5 Add missing acquire memory barrier in isc_nmhandle_unref
The ThreadSanitizer uses system synchronization primitives to check for
data race.  The netmgr handle->references was missing acquire memory
barrier before resetting and reusing the memory occupied by isc_nmhandle_t.

(cherry picked from commit 1013c0930e)
2020-06-16 08:58:33 +02:00
Witold Kręcicki
aa2282853a Fix a race in TCP accepting.
There's a possibility of a race in TCP accepting code:
T1 accepts a connection C1
T2 accepts a connection C2
T1 tries to accept a connection C3, but we hit a quota,
   isc_quota_cb_init() sets quota_accept_cb for the socket,
   we return from accept_connection
T2 drops C2, but we race in quota_release with accepting C3 so
   we don't see quota->waiting is > 0, we don't launch the callback
T1 accepts a connection C4, we are able to get the quota we clear
   the quota_accept_cb from sock->quotacb
T1 drops C1, tries to call the callback which is zeroed, sigsegv.
2020-06-10 11:39:43 -07:00
Witold Kręcicki
7ff1ff88f9 Don't clean quota cb cb_func/data, we don't own it 2020-06-10 17:52:53 +02:00
Witold Kręcicki
31897276d9 Fix 'error: ‘%s’ directive argument is null [-Werror=format-overflow=]' in assertions.c
(cherry picked from commit f4260dc0c5)
2020-06-08 15:57:31 +00:00
Witold Kręcicki
bbf1422614 Fix possible deadlock in unix/socket.c
In process_fd we lock sock->lock and then internal_accept locks mgr->lock,
in isc_sockmgr_render* functions we lock mgr->lock and then lock sock->lock,
that can cause a deadlock when accessing stats. Unlock sock->lock early in
all the internal_{send,recv,connect,accept} functions instead of late
in process_fd.
2020-06-08 15:30:10 +02:00
Mark Andrews
28a940fe69 Add +yaml support for EDE
(cherry picked from commit 0ec77c2b92)
2020-06-05 10:50:58 +10:00
Ondřej Surý
ef50067fd2 Stop building release notes as a separate sphinx-doc document
The release notes were previously built as a separate document
(including the PDF version).  It was agreed that this doesn't make much
sense, so the release notes are now included only as an appendix to the
BIND 9 ARM.

(cherry picked from commit 8eb2323ec3)
2020-06-04 11:24:53 +02:00
Witold Kręcicki
091117b7ae isc_uv_import must pass UV__IPC_SOCKET_XFER_TCP_CONNECTION, not SERVER.
As a leftover from old TCP accept code isc_uv_import passed TCP_SERVER
flag when importing a socket on Windows.
Since now we're importing/exporting accepted connections it needs to
pass TCP_CONNECTION flag.

(cherry picked from commit 801f7af6e9)
2020-06-03 23:27:24 +02:00
Witold Kręcicki
c4f91575ca Clean quota callback after calling it
(cherry picked from commit 2f252a1068)
2020-06-03 23:00:52 +02:00
Witold Kręcicki
818afe613f Redesigned TCP accepting: one listen/accept loop, passing the connected socket.
Instead of using bind() and passing the listening socket to the children
threads using uv_export/uv_import use one thread that does the accepting,
and then passes the connected socket using uv_export/uv_import to a random
worker. The previous solution had thundering herd problems (all workers
waking up on one connection and trying to accept()), this one avoids this
and is simpler.
The tcp clients quota is simplified with isc_quota_attach_cb - a callback
is issued when the quota is available.

(cherry picked from commit 60629e5b0b)
2020-06-03 23:00:52 +02:00