Commit graph

32689 commits

Author SHA1 Message Date
Diego Fronza
d727eaae6c Always return address records in additional section for NS queries 2020-10-21 12:03:42 -03:00
Diego Fronza
1cee557993 Fix transfer of glue records in stub zones if master has minimal-responses set
Stub zones don't make use of AXFR/IXFR for the transfering of zone
data, instead, a single query is issued to the master asking for
their nameserver records (NS).

That works fine unless master is configured with 'minimal-responses'
set to yes, in which case glue records are not provided by master
in the answer with nameservers authoritative for the zone, leaving
stub zones with incomplete databases.

This commit fix this problem in a simple way, when the answer with
the authoritative nameservers is received from master (stub_callback),
for each nameserver listed (save_nsrrset), a A and AAAA records for
the name is verified in the additional section, and if not present
a query is created to resolve the corresponsing missing glue.

A struct 'stub_cb_args' was added to keep relevant information for
performing a query, like TSIG key, udp size, dscp value, etc, this
information is borrowed from, and created within function 'ns_query',
where the resolving of nameserver from master starts.

A new field was added to the struct 'dns_stub', an atomic integer,
namely pending_requests, which is used to keep how many queries are
created when resolving nameserver addresses that were missing in
the glue.

When the value of pending_requests is zero we know we can release
resources, adjust zone timers, dump to zone file, etc.
2020-10-21 12:03:42 -03:00
Michal Nowak
803d12d430 Merge branch '1490-run-unit-tests-on-openbsd-in-ci' into 'main'
Add OpenBSD unit tests to CI

Closes #1490

See merge request isc-projects/bind9!4269
2020-10-21 11:15:23 +00:00
Michal Nowak
cf7db2003b
Run unit tests on OpenBSD in GitLab CI
Unlike other maintained BIND branches, the "main" BIND branch does not
require Kyua for running unit tests, which has been an obstacle for
adding an OpenBSD unit test job to GitLab CI.  Experiments show that a
complete BIND unit test suite completes in a few minutes on OpenBSD and
that unit tests are not as severely affected by OpenBSD performance
issues as system tests are.  Add a GitLab CI job which runs unit tests
on OpenBSD to every pipeline.
2020-10-21 13:06:52 +02:00
Diego dos Santos Fronza
ef050228f7 Merge branch '2195-freebsd-dnstap-system-test-failure' into 'main'
Resolve "FreeBSD dnstap system test failure"

Closes #2195

See merge request isc-projects/bind9!4213
2020-10-20 13:20:04 +00:00
Diego Fronza
d934513c42 Fix dnstap system test on FreeBSD
This commit ensures that dnstap output files captured
by fstrm_capture are properly flushed before any attempt
on reading them with dnstap-read is done.

By reading fstrm-capture source code it was noticed that
signal SIGHUP is used to flush the capture file.
2020-10-20 10:18:28 -03:00
Matthijs Mekking
7350c7555d Merge branch '2208-tcp4recverr-stat-miscount' into 'main'
Don't increment network error stats on UV_EOF

Closes #2208

See merge request isc-projects/bind9!4274
2020-10-20 11:57:35 +00:00
Matthijs Mekking
6c5ff94218 Don't increment network error stats on UV_EOF
When networking statistics was added to the netmgr (in commit
5234a8e00a), two lines were added that
increment the 'STATID_RECVFAIL' statistic: One if 'uv_read_start'
fails and one at the end of the 'read_cb'.  The latter happens
if 'nread < 0'.

According to the libuv documentation, I/O read callbacks (such as for
files and sockets) are passed a parameter 'nread'. If 'nread' is less
than 0, there was an error and 'UV_EOF' is the end of file error, which
you may want to handle differently.

In other words, we should not treat EOF as a RECVFAIL error.
2020-10-20 10:57:16 +02:00
Mark Andrews
a63ac933bb Merge branch 'marka-fix-not-enough-categories' into 'main'
Tune the minimum number of expected categories.

See merge request isc-projects/bind9!4216
2020-10-15 01:02:49 +00:00
Mark Andrews
0abb49034e Drop the expected minimum number of buckets to 4.
The previous value of 5 produced too many false errors.
2020-10-15 00:40:14 +00:00
Mark Andrews
1a4a7f4671 Merge branch 'marka-rrl-mdig-burst' into 'main'
Try to improve rrl timing

See merge request isc-projects/bind9!4126
2020-10-15 00:37:59 +00:00
Mark Andrews
92cdc7b6c7 Try to improve rrl timing
Add a +burst option to mdig so that we have a second to setup the
mdig calls then they run at the start of the next second.

RRL uses 'queries in a second' as a approximation to
'queries per second'. Getting the bursts of traffic to all happen in
the same second should prevent false negatives in the system test.

We now have a second to setup the traffic in.  Then the traffic should
be sent at the start of the next second.  If that still fails we
should move to +burst=<now+2> (further extend mdig) instead of the
implicit <now+1> as the trigger second.
2020-10-15 00:05:12 +00:00
Mark Andrews
3562cc7c89 Merge branch '2209-tsan-error-bin-named-controlconf-c-related' into 'main'
Resolve "TSAN error bin/named/controlconf.c related."

Closes #2209

See merge request isc-projects/bind9!4257
2020-10-15 00:01:36 +00:00
Mark Andrews
f95ba8aa20 Complete the isc_nmhandle_detach() in the worker thread.
isc_nmhandle_detach() needs to complete in the same thread
as shutdown_walk_cb() to avoid a race.  Clear the caller's
pointer then pass control to the worker if necessary.

    WARNING: ThreadSanitizer: data race
    Write of size 8 at 0x000000000001 by thread T1:
    #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15
    #1 control_command bin/named/controlconf.c:388:3
    #2 dispatch lib/isc/task.c:1152:7
    #3 run lib/isc/task.c:1344:2

    Previous read of size 8 at 0x000000000001 by thread T2:
    #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33
    #1 recv_data lib/isccc/ccmsg.c:109:2
    #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4
    #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3
    #4 uv_walk <null>
    #5 process_queue lib/isc/netmgr/netmgr.c:659:4
    #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10
    #7 process_queues lib/isc/netmgr/netmgr.c:590:8
    #8 async_cb lib/isc/netmgr/netmgr.c:548:2
    #9 <null> <null>
2020-10-12 17:51:09 +11:00
Ondřej Surý
8c4cf5d2e5 Merge branch 'ondrej/little-socket-fixes' into 'main'
Various little fixes in the old socket code

See merge request isc-projects/bind9!4253
2020-10-08 07:02:15 +00:00
Ondřej Surý
96ac91a18a Clean the last remnant of ISC_PLATFORM_HAVEIPV6 macro
In set_sndbuf() we were using ISC_PLATFORM_HAVEIPV6 macro that doesn't
exist anymore, because we assume that IPv6 support is always available.
2020-10-08 08:28:33 +02:00
Ondřej Surý
1672e851c8 Always set the DF flag (not only when CMSG is available)
By mistake, we were setting the DF flag only when CMSG was available for
said platform.
2020-10-08 08:28:14 +02:00
Ondřej Surý
442a45966a Merge branch 'ondrej/clean-netmgr-callback-locks' into 'main'
Clean the locking around nm callbacks

See merge request isc-projects/bind9!4220
2020-10-08 05:42:39 +00:00
Ondřej Surý
e8b56acb49 Clone the csock in accept_connection(), not in callback
If we clone the csock (children socket) in TCP accept_connection()
instead of passing the ssock (server socket) to the call back and
cloning it there we unbreak the assumption that every socket is handled
inside it's own worker thread and therefore we can get rid of (at least)
callback locking.
2020-10-08 07:24:31 +02:00
Ondřej Surý
d86a74d8a4 Change the isc__nm_tcpdns_stoplistening() to be asynchronous event
The isc__nm_tcpdns_stoplistening() would call isc__nmsocket_clearcb()
that would clear the .accept_cb from non-netmgr thread.  Change the
tcpdns_stoplistening to enqueue ievent that would get processed in the
right netmgr thread to avoid locking.
2020-10-08 07:24:31 +02:00
Mark Andrews
fe0a62654c Merge branch '2204-investigate-pipeline-system-test-failure' into 'main'
Resolve "Investigate pipeline system test failure"

Closes #2204

See merge request isc-projects/bind9!4247
2020-10-08 05:21:52 +00:00
Mark Andrews
1f63bb15b3 Restore the dns_message_reset() call before the dns_dispatch_getnext()
This was accidentally lost in the process of moving rmessage from fctx
to query.  Without this dns_message_setclass() will fail.
2020-10-08 10:55:35 +11:00
Matthijs Mekking
372142d371 Merge branch 'matthijs-kasp-test-requires-python' into 'main'
The kasp system test requires Python

See merge request isc-projects/bind9!4241
2020-10-07 15:22:23 +00:00
Matthijs Mekking
fedd621f37 The kasp system test requires Python
Only run the kasp system test if HAVE_PYTHON.
2020-10-07 10:21:43 +00:00
Mark Andrews
a3892cf783 Merge branch 'marka-wait-for-isc_nm_stoplistening' into 'main'
Wait for isc_nm_stoplistening() to complete

See merge request isc-projects/bind9!4217
2020-10-07 08:15:44 +00:00
Mark Andrews
402ac79833 Fix the data race on shutdown/reconfig in control channel
The controllistener could be freed before the event posted by
isc_nm_stoplistening() has been processed. This commit adds
a reference counter to the controllistener to determine when
to free the listener.
2020-10-07 18:24:25 +11:00
Mark Andrews
08c996c80a Merge branch '2200-the-fuzzer-dns_message_parser-c-is-leaking-memory' into 'main'
Resolve "The fuzzer dns_message_parse.c is leaking memory."

Closes #2200

See merge request isc-projects/bind9!4239
2020-10-06 13:20:36 +00:00
Mark Andrews
0d978878c5 invalidate cctx on error 2020-10-06 23:50:01 +11:00
Mark Andrews
40e54142f8 Merge branch 'marka-create_query-fix-REVERSE_INULL' into 'main'
Silence Coverity REVERSE_INULL report

Closes #2198

See merge request isc-projects/bind9!4234
2020-10-06 12:33:56 +00:00
Mark Andrews
f0a66cb5aa Silence Coverity REVERSE_INULL report
message does not need to be tested to NULL
2020-10-06 12:11:25 +00:00
Matthijs Mekking
d2d7f2a14e Merge branch 'he32-test-fix' into 'main'
Avoid a non-standard bashism: use of "==" in "test".

See merge request isc-projects/bind9!4235
2020-10-06 08:28:12 +00:00
Havard Eidnes
1f83daa4ac Avoid a non-standard bashism: use of "==" in "test". 2020-10-06 08:27:56 +00:00
Mark Andrews
4134980448 Merge branch 'marka-render_buf-was-too-big' into 'main'
render_buf was too big

See merge request isc-projects/bind9!4231
2020-10-06 07:37:30 +00:00
Mark Andrews
8faf12eace render buf was too big
It was misdeclared as a array of pointers (uint8_t*[]) and there
where too many elements (64 * 1024) instead of (64 * 1024 - 1).
2020-10-06 09:49:28 +11:00
Ondřej Surý
50df71a881 Merge branch '2183-dns-flag-day-2020' into 'main'
Resolve "DNS Flag Day 2020"

Closes #2183

See merge request isc-projects/bind9!4179
2020-10-05 15:13:12 +00:00
Ondřej Surý
096d41b4e2 Add text describing the changes done in the MR in more detail 2020-10-05 16:21:21 +02:00
Ondřej Surý
b9a42446e8 Enable DF (don't fragment) flag on listening UDP sockets
This commits uses the isc__nm_socket_dontfrag() helper function to
enable setting DF bit on the outgoing UDP packets.
2020-10-05 16:21:21 +02:00
Ondřej Surý
63c29dcdfa Change the default EDNS buffer size for dig, mdig and host to 1232
The dig should behave closely to how the resolver code in `named` work,
so we need to change the buffer size in dig, mdig and host too.
2020-10-05 16:21:21 +02:00
Ondřej Surý
f1556f8c41 Adjust legacy and digdelv tests for default 1232 EDNS Buffer Size
* the legacy test with -T maxudp512 will just fail, e.g. if the packets
  larger than 512 octets are dropped along the path, the proper response
  is to fail

* digdelv test was just expecting default server EDNS buffer size to be
  4096, the test needed only slight adjustment
2020-10-05 16:21:21 +02:00
Ondřej Surý
bb990030d3 Simplify the EDNS buffer size logic for DNS Flag Day 2020
The DNS Flag Day 2020 aims to remove the IP fragmentation problem from
the UDP DNS communication.  In this commit, we implement the required
changes and simplify the logic for picking the EDNS Buffer Size.

1. The defaults for `edns-udp-size`, `max-udp-size` and
   `nocookie-udp-size` have been changed to `1232` (the value picked by
   DNS Flag Day 2020).

2. The probing heuristics that would try 512->4096->1432->1232 buffer
   sizes has been removed and the resolver will always use just the
   `edns-udp-size` value.

3. Instead of just disabling the PMTUD mechanism on the UDP sockets, we
   now set IP_DONTFRAG (IPV6_DONTFRAG) flag.  That means that the UDP
   packets won't get ever fragmented.  If the ICMP packets are lost the
   UDP will just timeout and eventually be retried over TCP.
2020-10-05 16:21:21 +02:00
Ondřej Surý
d51f09a8d0 Merge branch 'ondrej/refactor-setsockopt' into 'main'
Refactor the setsockopt() code in network manager into helper functions

See merge request isc-projects/bind9!4227
2020-10-05 14:11:55 +00:00
Ondřej Surý
fd975a551d Split reusing the addr/port and load-balancing socket options
The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning
on different platform. In this commit, we split the function to set the
reuse of address/port and setting the load-balancing into separate
functions.

The libuv library already have multiplatform support for setting
SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address
and port, but unfortunately, when used after the load-balancing socket
options have been already set, it overrides the previous setting, so we
need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT
first and then enable the load-balancing socket option.
2020-10-05 15:18:28 +02:00
Ondřej Surý
acb6ad9e3c Use uv_os_sock_t instead of uv_os_fd_t for sockets
On POSIX based systems both uv_os_sock_t and uv_os_fd_t are both typedef
to int.  That's not true on Windows, where uv_os_sock_t is SOCKET and
uv_os_fd_t is HANDLE and they differ in level of indirection.
2020-10-05 15:18:28 +02:00
Ondřej Surý
9dc01a636b Refactor isc__nm_socket_freebind() to take fd and sa_family as args
The isc__nm_socket_freebind() has been refactored to match other
isc__nm_socket_...() helper functions and take uv_os_fd_t and
sa_family_t as function arguments.
2020-10-05 15:18:24 +02:00
Ondřej Surý
d685bbc822 Add helper function to enable DF (don't fragment) flag on UDP sockets
This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 14:55:20 +02:00
Ondřej Surý
5daaca7146 Add SO_REUSEPORT and SO_INCOMING_CPU helper functions
The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a
separate helper functions.
2020-10-05 14:54:24 +02:00
Matthijs Mekking
1567524a4f Merge branch '1749-rndc-dnssec-rollover' into 'main'
Resolve "rndc trigger rollover"

Closes #1749

See merge request isc-projects/bind9!4004
2020-10-05 09:46:48 +00:00
Matthijs Mekking
78c09f5622 Use default algorithm in kasp test if possible
These tests don't require a specific algorithm so they should use
the DEFAULT_ALGORITHM from 'conf.sh.common'.
2020-10-05 10:53:46 +02:00
Matthijs Mekking
70d1ec432f Use explicit result codes for 'rndc dnssec' cmd
It is better to add new result codes than to overload existing codes.
2020-10-05 10:53:46 +02:00
Matthijs Mekking
edc53fc416 Various rndc dnssec -checkds fixes
While working on 'rndc dnssec -rollover' I noticed the following
(small) issues:

- The key files where updated with hints set to "-when" and that
  should always be "now.
- The kasp system test did not properly update the test number when
  calling 'rndc dnssec -checkds' (and ensuring that works).
- There was a missing ']' in the rndc.c help output.
2020-10-05 10:53:46 +02:00