Ensure all shell system tests are executed with the errexit option set.
This prevents unchecked return codes from commands in the test from
interfering with the tests, since any failures need to be handled
explicitly.
In order to run the shell system tests, the pytest runner has to pick
them up somehow. Adding an extra python file with a single function
for the shell tests for each system test proved to be the most
compatible way of running the shell tests across older pytest/xdist
versions.
Modify the legacy run.sh script to ignore these pytest-runner specific
glue files when executing tests written in pytest.
The checkbashisms script reports errors like this one:
script util/check-line-length.sh does not appear to have a #! interpreter line;
you may get strange results
This commit converts the license handling to adhere to the REUSE
specification. It specifically:
1. Adds used licnses to LICENSES/ directory
2. Add "isc" template for adding the copyright boilerplate
3. Changes all source files to include copyright and SPDX license
header, this includes all the C sources, documentation, zone files,
configuration files. There are notes in the doc/dev/copyrights file
on how to add correct headers to the new files.
4. Handle the rest that can't be modified via .reuse/dep5 file. The
binary (or otherwise unmodifiable) files could have license places
next to them in <foo>.license file, but this would lead to cluttered
repository and most of the files handled in the .reuse/dep5 file are
system test files.
- serve-stale: dig wasn't always running in background when it should.
some of the serve-stale test cases are based on groups of dig calls
running simultaneously in the background: the test pauses and resumes
running after 'wait'. in some cases the final call to dig in a group
wasn't in the background, and this sometimes caused delays that
affected later test results. in another case, a test was simplified
and made more reliable by running dig in the foreground removing a
sleep.
- serve-stale: The extension of the dig timeout period from 10 to 11
seconds in commit 5307bf64ce was left undone in a few places and has
now been completed.
- serve-stale: Resolver-query-timeout was set incorrectly. a comment
above a test case in serve-stale/tests.sh says: "We configured a long
value of 30 seconds for resolver-query-timeout," but
resolver-query-timeout was actually set to 10, not 30. this is now
fixed.
- rpz: Force retransfer of the fast-expire zone, to ensure it's fully
loaded in ns3; previously it could have been left unloaded if ns5
wasn't up yet when ns3 attempted the zone transfer.
- statistics: The TCP4SendErr counter is incremented when a TCP dispatch
is canceled while sending. depending on test timing, this may have
happened by the time the statistics are dumped. worked around by
ignoring that stat couunter when checking for errors.
- hooks: Add a prereq.sh script to prevent running under TSAN.
- zero: Disabled the servfail cache so that SERVFAIL is reported only
when there actually is a failure, not repeatedly every time the same
query is sent.
The ISC_MEM_DEBUGSIZE and ISC_MEM_DEBUGCTX did sanity checks on matching
size and memory context on the memory returned to the allocator. Those
will no longer needed when most of the allocator will be replaced with
jemalloc.
The system test framework starts all named instances with the "-d 99"
command line option (unless it is overridden by a named.args file in a
given instance's working directory). This causes a lot of log messages
to be written to named.run files - currently over 5 million lines for a
single test suite run. While debugging information preserved in the log
files is essential for troubleshooting intermittent test failures, some
system tests involve sending hundreds or even thousands of queries,
which causes the relevant log files to explode in size. When multiple
tests (or even multiple test suites) are run in parallel, excessive
logging contributes considerably to the I/O load on the test host,
increasing the odds of intermittent test failures getting triggered.
Decrease the debug level for the seven most verbose named instances:
- use "-d 3" for ns2 in the "cacheclean" system test (it is the lowest
logging level at which the test still passes without the need to
apply any changes to tests.sh),
- use "-d 1" for the other six named instances.
This roughly halves the number of lines logged by each test suite run
while still leaving enough information in the logs to allow at least
basic troubleshooting in case of test failures.
This approach was chosen as it results in a greater decrease in the
number of lines logged than running all named instances with "-d 3",
without causing any test failures.
When we were in nmthread, the isc__nm_async_<proto>connect() function
executes in the same thread as the isc__nm_<proto>connect() and on a
failure, it would block indefinitely because the failure branch was
setting sock->active to false before the condition around the wait had a
chance to skip the WAIT().
This also fixes the zero system test being stuck on FreeBSD 11, so we
re-enable the test in the commit.
Due to the platform differences, on non-Linux platforms, the xfer and
ixfr tests fails and zero test gets stuck.
This commit will get reverted when we add support for netmgr
multi-threading.
The $SYSTEMTESTTOP shell variable if often set to .. in various shell
scripts inside bin/tests/system/, but most of the time it is only
used one line later, while sourcing conf.sh. This hardly improves
code readability.
$SYSTEMTESTTOP is also used for the purpose of referencing
scripts/files living in bin/tests/system/, but given that the
variable is always set to a short, relative path, we can drop it and
replace all of its occurrences with the relative path without adversely
affecting code readability.
this changes most visble uses of master/slave terminology in tests.sh
and most uses of 'type master' or 'type slave' in named.conf files.
files in the checkconf test were not updated in order to confirm that
the old syntax still works. rpzrecurse was also left mostly unchanged
to avoid interference with DNSRPS.
as "type primary" is preferred over "type master" now, it makes
sense to make "primaries" available as a synonym too.
added a correctness check to ensure "primaries" and "masters"
cannot both be used in the same zone.
The rewrite of BIND 9 build system is a large work and cannot be reasonable
split into separate merge requests. Addition of the automake has a positive
effect on the readability and maintainability of the build system as it is more
declarative, it allows conditional and we are able to drop all of the custom
make code that BIND 9 developed over the years to overcome the deficiencies of
autoconf + custom Makefile.in files.
This squashed commit contains following changes:
- conversion (or rather fresh rewrite) of all Makefile.in files to Makefile.am
by using automake
- the libtool is now properly integrated with automake (the way we used it
was rather hackish as the only official way how to use libtool is via
automake
- the dynamic module loading was rewritten from a custom patchwork to libtool's
libltdl (which includes the patchwork to support module loading on different
systems internally)
- conversion of the unit test executor from kyua to automake parallel driver
- conversion of the system test executor from custom make/shell to automake
parallel driver
- The GSSAPI has been refactored, the custom SPNEGO on the basis that
all major KRB5/GSSAPI (mit-krb5, heimdal and Windows) implementations
support SPNEGO mechanism.
- The various defunct tests from bin/tests have been removed:
bin/tests/optional and bin/tests/pkcs11
- The text files generated from the MD files have been removed, the
MarkDown has been designed to be readable by both humans and computers
- The xsl header is now generated by a simple sed command instead of
perl helper
- The <irs/platform.h> header has been removed
- cleanups of configure.ac script to make it more simpler, addition of multiple
macros (there's still work to be done though)
- the tarball can now be prepared with `make dist`
- the system tests are partially able to run in oot build
Here's a list of unfinished work that needs to be completed in subsequent merge
requests:
- `make distcheck` doesn't yet work (because of system tests oot run is not yet
finished)
- documentation is not yet built, there's a different merge request with docbook
to sphinx-build rst conversion that needs to be rebased and adapted on top of
the automake
- msvc build is non functional yet and we need to decide whether we will just
cross-compile bind9 using mingw-w64 or fix the msvc build
- contributed dlz modules are not included neither in the autoconf nor automake
Before, the zero system test could get stuck almost infinitely, because
the first test sends > 300 queries with 5 seconds timeout on each in
each pass. If named crashed early, it would took the test more than 4
hours to properly timeout.
This commit introduces a "watchdog" on the dig commands running in the
background and failing the test on timeout, failing any test if any dig
command fails to return successfully, and making the tests.sh script
shellcheck clean.
The first step in all existing setup.sh scripts is to call clean.sh. To
reduce code duplication and ensure all system tests added in the future
behave consistently with existing ones, invoke clean.sh from run.sh
before calling setup.sh.
- all tests with "recursion yes" now also specify "dnssec-validation yes",
and all tests with "recursion no" also specify "dnssec-validation no".
this must be maintained in all new tests, or else validation will fail
when we use local root zones for testing.
- clean.sh has been modified where necessary to remove managed-keys.bind
and viewname.mkeys files.
- add CHANGES note
- update copyrights and license headers
- add -j to the make commands in .gitlab-ci.yml to take
advantage of parallelization in the gitlab CI process
Conflicts:
bin/tests/system/conf.sh.in
lib/dns/win32/libdns.def.in
lib/isc/win32/file.c
The merge also needed to update files in legacy and tcp system tests
(newly introduced in master after branch was created) to introduce use
of lockfile.
