This adds a new test directory specifically for CAMP attacks. This first
test in this test directory follows multiple CNAME chains, restarting
the max-recursion-queries counter, but should bail when the global
maximum quota max-query-count is reached.
Add another option to configure how many outgoing queries per
client request is allowed. The existing 'max-recursion-queries' is
per restart, this one is a global limit.
The ns2/managed1.conf file is created by the setup.sh script. Then, in
the tests.sh script it is moved to ns2/managed.conf. The latter file
name is in mkeys extra_artifacts, but the former one is not. This is a
problem when pytest is started with the --setup-only option as it only
runs the setup.sh script (e.g., in the cross-version-config-tests CI
job) and thus failing the "Unexpected files found" assertion.
When the tests were added, the files were generated without FIPS
compatibility in mind. That made the tests fail on recent OpenSSL
versions in FIPS mode.
So, the files were regenerated on a FIPS compliant system using the
following stanza:
$ openssl dhparam -out <file> 3072
Apparently, the old files are not valid for FIPS starting with OpneSSL
3.1.X release series as "FIPS 140-3 compliance changes" are mentioned
in the changelog:
https://openssl-library.org/news/openssl-3.1-notes/
Add support for Extended DNS Errors (EDE) error 22: No reachable
authority. This occurs when after a timeout delay when the resolver is
trying to query an authority server.
Instead of cleaning the dns_badcache opportunistically, add per-loop
LRU, so each thread-loop can clean the expired entries. This also
allows removal of the atomic operations as the badcache entries are now
immutable, instead of updating the badcache entry in place, the old
entry is now deleted from the hashtable and the LRU list, and the new
entry is inserted in the LRU.
The nslookup system test checks the count of resolved addresses in
the CNAME tests using a 'grep' match on the hostname, and ignoring
lines containing the 'canonical name' string. In order to protect
the check from intermittent failures like the 'address in use' warning
message, which then automatically resolves after a retry, edit the
'grep' matching string to also ignore the comments (as the mentioned
warning message is a comment which contains the hostname).
Add test cases which check that when a XoT primary server is
unreachable or is already marked as unreachble then the next
primary server in the list is used.
Previously a ISC_R_CANCELED result code switch-case has been added to
the zone.c:zone_xfrdone() function, which did two things:
1. Schedule a new zone transfer if there's a scheduled force reload of
the zone.
2. Reset the primaries list.
This proved to be not a well-thought change and causes problems,
because the ISC_R_CANCELED code is used not only when the whole transfer
is canceled, but also when, for example, a particular primary server is
unreachable, and named still needs to continue the transfer process by
trying the next server, which it now no longer does in some cases. To
solve this issue, three changes are made:
1. Make sure dns_zone_refresh() runs on the zone's loop, so that the
sequential calls of dns_zone_stopxfr() and dns_zone_forcexfr()
functions (like done in 'rndc retransfer -force') run in intended
order and don't race with each other.
2. Since starting the new transfer is now guaranteed to run after the
previous transfer is shut down (see the previous change), remove the
special handling of the ISC_R_CANCELED case, and let the default
handler to handle it like before. This will bring back the ability to
try the next primary if the current one was interrupted with a
ISC_R_CANCELED result code.
3. Change the xfrin.c:xfrin_shutdown() function to pass the
ISC_R_SHUTTINGDOWN result code instead of ISC_R_CANCELED, as it makes
more sense.
This change adds a "none" parameter to the query-source[-v6]
options in named.conf, which forbid the usage of IPv4 or IPv6
addresses when doing upstream queries.
On some slow systems, the test might intermittently fail due to inherent
timing issues. In our CI, this most often happens in the
system:gcc:8fips:amd64 jobs.
Reintroduce logic to apply diffs when the number of pending tuples is
above 128. The previous strategy of accumulating all the tuples and
pushing them at the end leads to excessive memory consumption during
transfer.
This effectively reverts half of e3892805d6
When a test is re-run by the flaky plugin, the TestReport outcomes
collected in the pytest_runtest_makereport() hook should be overriden.
Each of the setup/call/teardown phases is reported again and since we
care about the overall outcome, their respective results should be
overriden so that only the outcome from the final test (re)run gets
reported.
Prior to this change, it lead to a situation where an extra_artifact
generated during the test might be ignored. This was caused because the
check was skipped, since the test was incorrectly considered as "failed"
in the case where the test would fail on the first run, but pass on a
subsequent flaky rerun.
QPDB is now a default implementation for both cache and zone. Remove
the venerable RBTDB database implementation, so we can fast-track the
changes to the database without having to implement the design changes
to both QPDB and RBTDB and this allows us to be more aggressive when
refactoring the database design.
Prior to introducing the pytest runner, clean.sh files were used as a
list of files that the test is expected to leave around as artifacts and
check that no extra files were created.
With the pytest runner, those scripts are no longer used, but the
ability to detect extraneous files is still useful. Add a new
"extra_artifacts" mark which can be used for the same purpose.
The query-source option has the slight quirk of allowing the address to
be specified in two ways, either as every other source option, or as an
"address" key-value pair.
For this reason, it had a separate parsing function from other X-source
options, but it is possible to extend the parsing of other X-sources to
be generic and also handle query-source.
This commit just does that.
Add a test case for Offline KSK where during the lifespan of the Signed
Key Response a KSK rollover happens. Ensure that the correct DNSKEY,
CDNSKEY, and CDS records are published at the right times.
When the zone is initially signed, the CDNSKEY/CDS RRset is not
immediately published. The DNSKEY and signatures must propagate first.
Adjust the test to allow for this case.
Add an option to dnssec-ksr keygen, -o, to create KSKs instead of ZSKs.
This way, we can create a set of KSKS for a given period too.
For KSKs we also need to set timing metadata, including "SyncPublish"
and "SyncDelete". This functionality already exists in keymgr.c so
let's make the function accessible.
Replace dnssec-keygen calls with dnssec-ksr keygen for KSK in the
ksr system test and check keys for created KSKs as well. This requires
a slight modification of the check_keys function to take into account
KSK timings and metadata.
In the ksr system test, the test_ksr_twotone case may fail if there
are two keys with the same keytag (but different algorithms), because
one key is expected to be signing and the other is not.
Switch to regular expression matching and include the algorithm in the
search string.
Configuration files in system tests which require some variables (e.g.
port numbers) filled in during test setup, can now use jinja2 templates
when `jinja2` python package is available.
Any `*.j2` file found within the system test directory will be
automatically rendered with the environment variables into a file
without the `.j2` extension by the pytest runner. E.g.
`ns1/named.conf.j2` will become `ns1/named.conf` during test setup. To
avoid automatic rendering, use `.j2.manual` extension and render the
files manually at test time.
New `templates` pytest fixture has been added. Its `render()` function
can be used to render a template with custom test variables. This can be
useful to fill in different config options during the test. With
advanced jinja2 template syntax, it can also be used to include/omit
entire sections of the config file rather than using `named1.conf.in`,
`named2.conf.in` etc.
some EDNS option names, including DAU, DHU, N3U, and CHAIN,
were not printed in dns_message_pseudosectiontotext() or
_psuedosectiontoyaml(); they were displayed as unknown options.
this has been corrected.
that code was also refactored to use switch instead of if/else,
and to look up the option code names in a table to prevent
inconsistencies between the two formats. one such inconsistency
was corrected: the "TCP-KEEPALIVE" option is now always printed
with a hyphen, instead of being "TCP KEEPALIVE" when not using
YAML. the keepalive system test has been updated to expect this.
EDNS options that print DNS names (i.e., CHAIN and Report-Channel)
now enclose them in quotation marks to ensure YAML correctness.
the auth system test has been updated to expect this when grepping
for Report-Channel options.
Coverity Scan reported a new issue for the ksr system test. There
is allegedly a null pointer dereference (FORWARD_NULL) in check_keys().
This popped up because previously we set 'retired' to 0 in case of
unlimited lifetime, but we changed it to None.
It is actually a false positive, because if lifetime is unlimited
there will be only one key in 'keys'.
However, the code would be better if we always initialized 'active'
and if it is not the first key and retired is set, set the successor
key's active time to the retire time of the predecessor key.
The test_ksr_twotwone may fail if the key id is shorter than 5 digits.
Add a leading space to the expected strings which start with the key
tag to avoid the issue.
The pytest collection mechanism has been overhauled in pytest 8.0.0,
resulting in a different node tree when collecting the tests. Ensure the
paths / names we're using that are derived from the node tree are
consistent across different pytest versions.
Particularly, this has affected the convenience symlink name (which is
supposed to be in the form of e.g. dns64_sh_dns64 for the dns64 module
and tests_sh_dns64.py module) and the test name that's logged at the
start of the test, which is supposed to include the system test
directory relative to the root system test directory as well as the
module name (e.g. dns64/tests_sh_dns64.py).
Related https://github.com/pytest-dev/pytest/issues/7777
RFC 9567 section 8.1 specifies that the agent domain cannot
be a subdomain of the domain it is reporting on. therefore,
in addition to making it illegal to configure that at the
zone level, we also need to disable send-report-channel for
any zone for which the global send-report-channel value is
a subdomain.
we also now warn if send-report-channel is configured
globally to a zone that we host, but that zone doesn't
have log-report-channel set.
the logging of error-report queries is no longer activated by
the view's "send-report-channel" option; that now only configures
the agent-domain value that is to be sent in authoritative
responses. the warning that was logged when "send-agent-domain"
was set to a value that is not a locally configured zone has
been removed.
error-report logging is now activated by the presence of an
authoritative zone with the "log-report-channel" option set to
"yes". this is not permitted in the root zone.
NOTE: a zone with "log-report-channel yes;" should contain a
"*._er" wildcard, but that requirement is not yet enforced.
add a boolean "log-report-channel" option for primary and
secondary zones, which sets the DNS_ZONEOPT_LOGREPORTS zone
flag. this option is not yet functional.
If send-report-channel is set at the zone level, it will
be stored in the zone object and used instead of the
view-level agent-domain when constructing the EDNS
Report-Channel option.
This commit adds support for the EDNS Report-Channel option,
which is returned in authoritative responses when EDNS is in use.
"send-report-channel" sets the Agent-Domain value that will be
included in EDNS Report-Channel options. This is configurable at
the options/view level; the value is a DNS name. Setting the
Agent-Domain to the root zone (".") disables the option.
When this value has been set, incoming queries matchng the form
_er.<qtype>.<qname>.<extended-error-code>._er.<agent-domain>/TXT
will be logged to the dns-reporting-agent channel at INFO level.
(Note: error reporting queries will only be accepted if sent via
TCP or with a good server cookie. If neither is present, named
returns BADCOOKIE to complete the DNS COOKIE handshake, or TC=1
to switch the client to TCP.)
When working with key timestamps, ensure we correctly set the UTC
timezone in order for the tests to work consistently regardless of the
local time setting.
Enforcing pylint standards and default for our test code seems
counter-productive. Since most of the newly added code are tests or is
test-related, encountering these checks rarely make us refactor the code
in other ways and we just disable these checks individually. Code that
is too complex or convoluted will be pointed out in reviews anyways.
Move all test cases from tests.sh to tests_ksr.py. The only test that
is not moved is the check that key id's match expected keys. The
shell-based system test checks two earlier set environment variables
against each other that has become redundant in the pytest variant,
because we now check the signed key response against a list of keys
and for each key we take into account the timing metadata. So we
already ensure that each published key is in the correct key bundle.
Write initial pytest kasp library. This contains everything that is
required for testing Offline KSK functionality with pytest.
This includes:
- addtime: adding a value to a timing metadata
- get_timing_metdata: retrieve timing metadata from keyfile
- get_metadata/get_keystate: retrieve metadata from statefile
- get_keytag: retrieve keytag from base keyfile string
- get_keyrole: get key role from statefile
- dnskey_equals: compare DNSKEY record from file against a string
- cds_equals: compare CDS derived from file against a string
- zone_is_signed: wait until a zone is completely signed
- dnssec_verify: verify a DNSSEC signed zone with dnssec-verify
- check_dnssecstatus: check rndc dnssec -status output
- check_signatures: check that signatures for a given RRset are correct
- check_dnskeys: check that the published DNSKEY RRset is correct
- check_cds: check that the published CDS RRset is correct
- check_apex: check SOA, DNSKEY, CDNSKEY, and CDS RRset
- check_subdomain: check an RRset below the apex
CookieOption with new .server/.client attributes (rather than .data) was
added to dnspython. Adjust the code to use the new attributes if
available and fall back to the old code for dnspython<2.7.0
compatibility.
there was no system test that exercised 'rndc recursing'; a
simple one has now been added; it confirms that the number of
recursing clients reported by 'rndc stats' is in agreement with
the list returned by 'rndc recursing'.
In a multi-signer setup, removing DNSKEY records from the zone should
not be treated as a key that previously exists in the keyring, thus
blocking the keymgr. Add a test case to make sure.
Test that if a key to be purged is in the keyring, it does not
prevent the keymgr from running. Normally a key that is in the keyring
should be available again on the next run, but that is not true for
a key that can be purged.
In addition, fix some wait_for_log calls, by adding the missing
'|| ret=1' parts.
Some test cases were working but for the wrong reasons. These started
to fail when I implemented the first approach for #4763, where the
existence of a DNSKEY together with an empty keyring is suspicious and
would prevent the keymgr from running.
These are:
1. kasp: The multisigner-model2.kasp zone has ZSKs from other providers
in the zone, but not yet its own keys. Pregenerate signing keys and
add them to the unsigned zone as well.
2. kasp: The dynamic-signed-inline-signing.kasp zone has a key generated
and added in the raw version of the zone. But the key file is stored
outside the key-directory for the given zone. Add '-K keys' to the
dnssec-keygen command.
Prior to running the keymgr, first make sure that existing keys
are present in the new keylist. If not, treat this as an operational
error where the keys are made offline (temporarily), possibly unwanted.
In this specific case the key files are temporary unavailable, for
example because of an operator error, or a mount failure). In such
cases, BIND should not try to roll over these keys.
The dns_dispatch_add() function registers the 'resp' entry in
'disp->mgr->qids' hash table with 'resp->port' being 0, but in
tcp_recv_success(), when looking up an entry in the hash table
after a successfully received data the port is used, so if the
local port was set (i.e. it was not 0) it fails to find the
entry and results in an unexpected error.
Set the 'resp->port' to the given local port value extracted from
'disp->local'.
This commit adds support for timestamps in iso8601 format with timezone
when logging. This is exposed through the iso8601-tzinfo printtime
suboption.
It also makes the new logging format the default for -g output,
hopefully removing the need for custom timestamp parsing in scripts.
Check that with a preexisting dump file without a write permission
no actual dump happens, and rndc returns an error result.
Also add a couple of missing entries in the cleaning script.
The shutdown system test sends queries when named is shutting down, not
in an attempt to get answers but to destabilize the server into a crash.
With isctest.query.udp() defaulting to try up to ten times with a
ten-second timeout to get a response we don't care about from a likely
terminated server, we make the test run much longer than needed because
of retries and long timeouts.
DNSRPS was the API for a commercial implementation of Response-Policy
Zones that was supposedly better. However, it was never open-sourced
and has only ever been available from a single vendor. This goes against
the principle that the open-source edition of BIND 9 should contain only
features that are generally available and universal.
This commit removes the DNSRPS implementation from BIND 9. It may be
reinstated in the subscription edition if there's enough interest from
customers, but it would have to be rewritten as a plugin (hook) instead
of hard-wiring it again in so many places.
Check that the fix in the previous commit works and that the
'ServerQuota' counter in the statistics channel is still unset
after a SERVFAIL result in a 'forward only' zone.
Remove the use of "port" when configuring query-source(-v6),
transfer-source(-v6), notify-source(-v6), parental-source(-v6),
etc. Remove the use of source ports for parental-agents.
Also remove the deprecated options use-{v4,v6}-udp-ports and
avoid-{v4,v6}udp-ports.
This change allows fallback from an IXFR failure to AXFR when the
reason is DNS_R_TOOMANYRECORDS. This is because this error condition
could be temporary only in an intermediate version of IXFR
transactions and it's possible that the latest version of the zone
doesn't have that condition. In such a case, the secondary would never
be able to update the zone (even if it could) without this fallback.
This fallback behavior is particularly useful with the recently
introduced max-records-per-type and max-types-per-name options:
the primary may not have these limitations and may temporarily
introduce "too many" records, breaking IXFR. If the primary side
subsequently deletes these records, this fallback will help recover
the zone transfer failure automatically; without it, the secondary
side would first need to increase the limit, which requires more
operational overhead and has its own adverse effect.
This change also fixes a minor glitch that DNS_R_TOOMANYRECORDS wasn't
logged in xfrin_fail.
Administrators may wish to constrain the set of cores that BIND 9 runs
on via the 'taskset', 'cpuset' or 'numactl' programs (or equivalent on
other O/S), for example to achieve higher (or more stable) performance
by more closely associating threads with individual NIC rx queues. If
the admin has used taskset, it follows that BIND ought to
automatically use the given number of CPUs rather than the system wide
count.
Co-Authored-By: Ray Bellis <ray@isc.org>
With max-recursion-queries set to 50 the resolver system test was
unstable in the "checking query resolution for a domain with a valid
glueless delegation chain" check as ns1 replied with SERVFAIL.
Return partial match from dns_db_find/dns_db_find when requested
to short circuit the closest encloser discover process. Most of the
time this will be the actual closest encloser but may not be when
there yet to be committed / cleaned up versions of the zone with
names below the actual closest encloser.
Configure a maximum of 3 allowed differences and add 5 new records.
Check that named detected that the differences exceed the allowed
limit and successfully retries with AXFR.
This limits the maximum number of received incremental zone
transfer differences for a secondary server. Upon reaching the
confgiured limit, the secondary aborts IXFR and initiates a full
zone transfer (AXFR).
If there is a keytag conflict between keys with different algorithms,
we need to supply what key algorithm is used so we can get the right
public key.
For clarity, print the algorithm on the found keys after 'check_keys'.
Test importing a Signed Key Response. Files should be loaded and once
loaded the correct bundle should be used. Alsoe test cases where the
bundle is not the first bundle in the SKR.
Prepare the system test for tests that require a server to import
created SKR files. This means the test script needs adjustments to
take into account the directory test files are located.
In addition, the check_keys function is renamed to ksr_check_keys
because the name clashes with check_keys from kasp.sh. It also has an
extra parameter added, offset, that can be used to check ksr files that
are created in the past or future.
Add a new configuration option to enable Offline KSK key management.
Offline KSK cannot work with CSK because it splits how keys with the
KSK and ZSK role operate. Therefore, one key cannot have both roles.
Add a configuration check to ensure this.
The new
isc_log_createandusechannel() function combines following calls:
isc_log_createchannel()
isc_log_usechannel()
calls into a single call that cannot fail and therefore can be used in
places where we know this cannot fail thus simplifying the error
handling.
Remove the complicated mechanism that could be (in theory) used by
external libraries to register new categories and modules with
statically defined lists in <isc/log.h>. This is similar to what we
have done for <isc/result.h> result codes. All the libraries are now
internal to BIND 9, so we don't need to provide a mechanism to register
extra categories and modules.
Add isc_logconfig_get() function to get the current logconfig and use
the getter to replace most of the little dancing around setting up
logging in the tools. Thus:
isc_log_create(mctx, &lctx, &logconfig);
isc_log_setcontext(lctx);
dns_log_setcontext(lctx);
...
...use lcfg...
...
isc_log_destroy();
is now only:
logconfig = isc_logconfig_get(lctx);
...use lcfg...
For thread-safety, isc_logconfig_get() should be surrounded by RCU read
lock, but since we never use isc_logconfig_get() in threaded context,
the only place where it is actually used (but not really needed) is
named_log_init().
The test_traffic_json and test_traffic_xml occasionally fail when
running under TSAN. This happens in CI and is most likely a result of
some instability that doesn't seem to be easily reproduced.
Some distributions (notably, debian bookworm) have deprecated the
`python` interpreter in favor of `python3`. Since our scripts are
python3 anyway, use the proper numbered version in shebang to make
scripts easily executable.
Catalog zone member zones should be reconfigured as all the other
zones during a reconfiguration. Test it by checking whether the newly
added allow-query setting affects a member zone.
implement, document, and test the 'max-query-restarts' option
which specifies the query restart limit - the number of times
we can follow CNAMEs before terminating resolution.
the number of steps that can be followed in a CNAME chain
before terminating the lookup has been reduced from 16 to 11.
(this is a hard-coded value, but will be made configurable later.)
Instead of calling dst_lib_init() and dst_lib_destroy() explicitly by
all the programs, create a separate memory context for the DST subsystem
and use the library constructor and destructor to initialize the DST
internals.
Since the support for OpenSSL Engines has been removed, we can now also
remove the checks for OPENSSL_API_LEVEL; The OpenSSL 3.x APIs will be
used when compiling with OpenSSL 3.x, and OpenSSL 1.1.xx APIs will be
used only when OpenSSL 1.1.x is used.
The OpenSSL 1.x Engines support has been deprecated in the OpenSSL 3.x
and is going to be removed. Remove the OpenSSL Engine support in favor
of OpenSSL Providers.
dns_difftuple_create() could only return success, so change
its type to void and clean up all the calls to it.
other functions that only returned a result value because of it
have been cleaned up in the same way.
Since the minimal OpenSSL version is now OpenSSL 1.1.1, remove all kind
of OpenSSL shims and checks for functions that are now always present in
the OpenSSL libraries.
Co-authored-by: Ondřej Surý <ondrej@isc.org>
Co-authored-by: Aydın Mercan <aydin@isc.org>
Add a system test that sets TSIG fudge to 0, waits three seconds and
then sends signed message to the server. This tests the path where the
time difference between the client and the server is outside of the TSIG
fudge value.
The system tests were overriding the local locale by setting LANG to C.
This does not override the locale in case there are individual LC_<*>
variables like LC_CTYPE explicitly set.
Use LC_ALL=C instead which is the proper way of overriding all currently
set locales.
Use a big zone and the slow transfer mode. Initiate a retransfer, wait
several seconds, then initiate a retransfer using a '-force' argument,
which should cancel the previous transfer and start a new one.
Update the CNAME chain test to correspond to the changed behavior,
because now named returns SERVFAIL when hitting the maximum query
restarts limit (e.g. happening when following a long CNAME chain).
In the current test auth will hit the limit and return partial data
with a SERVFAIL code, while the resolver will return no data with
a SERVFAIL code after auth returns SERVFAIL to it.
Ensure all the variables are initialized when running the main function
of isctest module. This enables proper environment variables during test
script development when only conf.sh is sourced, rather than the script
being executed by the pytest runner.
Run the crypto support checks when initializing the isctest package and
save those results in environment variable. This removes the need to
repeatedly check for crypto operation support, as it's not something
that would change at test runtime.
Instead of invoking get_algorithms.py script repeatedly (which may yield
different results), move the algorithm configuration to an isctest
module. This ensures the variables are consistent across the entire test
run.
The updatecheck-kskonly.secure zone is being used to test dynamic
updates while the KSK is offline. It ensures that the DNSKEY RRset
will retain the RRSIG record, while the updated data is being signed
with the currently active ZSK.
When walking through ZSK rollovers, ensure that the newest ZSK (ZSK3)
is published before doing the dynamic update, preventing timing
related test failures.
Also fix the test log line ($ZSK_ID3 was not yet created at the time
of logging).
The key lifetime should no longer be adjusted if the key is being
retired earlier, for example because a manual rollover was started.
This would falsely be seen as a dnssec-policy lifetime reconfiguration,
and would adjust the retire/removed time again.
This also means we should update the status output, and the next
rollover scheduled is now calculated using (retire-active) instead of
key lifetime.
In order to protect from a malicious DNS client that sends many
queries with a SIG(0)-signed message, add a quota of simultaneously
running SIG(0) checks.
This protection can only help when named is using more than one worker
threads. For example, if named is running with the '-n 4' option, and
'sig0checks-quota 2;' is used, then named will make sure to not use
more than 2 workers for the SIG(0) signature checks in parallel, thus
leaving the other workers to serve the remaining clients which do not
use SIG(0)-signed messages.
That limitation is going to change when SIG(0) signature checks are
offloaded to "slow" threads in a future commit.
The 'sig0checks-quota-exempt' ACL option can be used to exempt certain
clients from the quota requirements using their IP or network addresses.
The 'sig0checks-quota-maxwait-ms' option is used to define a maximum
amount of time for named to wait for a quota to appear. If during that
time no new quota becomes available, named will answer to the client
with DNS_R_REFUSED.
kasp-max-types-per-name (named2.conf.in):
An unsigned zone with RR type count on a name right below the
configured limit. Then sign the zone using KASP. Adding a RRSIG would
push it over the RR type limit per name. Signing should fail, but
the server should not crash, nor end up in infinite resign-attempt loop.
kasp-max-records-per-type-dnskey (named1.conf.in):
Test with low max-record-per-rrset limit and a DNSSEC policy requiring
more than the limit. Signing should fail.
kasp-max-types-per-name (named1.conf.in):
Each RRSIG(covered type) is counted as an individual RR type. Test the
corner case where a signed zone, which is just below the limit-1,
adds a new type - doing so would trigger signing for the new type and
thus increase the number of "types" by 2, pushing it over the limit
again.
Add two new masterformat tests that use signing. In the case of
'under-limit-kasp', the signing will keep the number of records in the
RRset under the limit. In the case of 'on-limit-kasp', the signing
will push the number of records in the RRset over the limit, because
of the added RRSIG record.
Send a recursive query for a large number of RRsets, which should
fail when using the default max-types-per-name setting of 100, but
succeed when the cap is disabled.
Send a recursive query for a large (2500 record) RRset, which should
fail when using the default max-records-per-type setting of 100, but
succeed when the cap is disabled.
Add two new masterformat tests - the 'huge' zone fits within the ns1
limit and loads on the primary ns1 server, but must not transfer to the
ns2 secondary, and the 'uber' zone should not even load on the primary
ns1 server.
Previously, the number of RRs in the RRSets were internally unlimited.
As the data structure that holds the RRs is just a linked list, and
there are places where we just walk through all of the RRs, adding an
RRSet with huge number of RRs inside would slow down processing of said
RRSets.
Add a configurable limit to cap the number of the RRs in a single RRSet.
This is enforced at the database (rbtdb, qpzone, qpcache) level and
configured with new max-records-per-type configuration option that can
be configured globally, per-view and per-zone.
Changed the default value for 'allow-transfer' to 'none'; zone
transfers now require explicit authorization.
Updated all system tests to specify an allow-transfer ACL when needed.
Revised the ARM to specify that the default is 'none'.
When in -4 mode check that "IPv6 disabled and no IPv4 primaries"
is logged and when in -6 mode check that "IPv4 disabled and no IPv6
primaries" is logged.
The times it takes to run tests CI vary significantly enough
that it makes hypothesis test reach their deadlines and fail randomly
marking the tests as flaky.
This commit disables the deadlines when running in CI.
The most important being `dns_names` that generates dns.name.Name
objects based on given paramaters.
No guarantees are given when it comes the uniformity of generated
samples, however it plays nicely with the hypothesis' shrinking
algorithm.
Once we use hypothesis more widely (in at least one more test) this file
should be moved for it to be reused easily.
Check for more rcodes and various properties needed in the wildcard
test. Add a `name` module for various dns.name.Name operations (with
`prepend_label` function only now).
Expose `timeout` as a parameter of `query.tcp`/`query.udp`.
- duplicated question
- duplicated answer
- qtype as an answer
- two question types
- question names
- nsec3 bad owner name
- short record
- short question
- mismatching question class
- bad record owner name
- mismatched class in record
- mismatched KEY class
- OPT wrong owner name
- invalid RRSIG "covers" type
- UPDATE malformed delete type
- TSIG wrong class
- TSIG not the last record
An obscured DNSKEY RRset at a delegation was incorrectly added to
the NSEC/NSEC3 type bit map leading to zone verification failures.
This adds such a RRset to the test zone.
To avoid any escaping issues or messing with a language-specific format
when the variable has to be parsed, create a dedicated file for each
variable that is obtained from autoconf.
Make sure all initialization takes place in isctest.vars.__init__ and
export the initial env vars there. Remove the no longer needed env
fixture and use os.environ instead.
If OPENSSL_CONF is exported as an empty string, it will cause issues on
rhel9fips. Allow the environment variables to be set and exported, but
make sure to only export them if they have been set by the user.
The openssl config needs to be parsed for some tests that use SoftHSM2.
Rewrite the parsing to python and ensure the required variables are
properly set test-wide.
Provide a single point of access to all the variables used by tests. Use
a custom dict-like structure to access the underlying data without
making a copy. This allows the individual modules to update the contents
at runtime, which is used for some variables.
While this isn't required for pytest operation and execution of the
system test suite, it can be handy to allow test script development and
debugging. Especially setup scripts often source conf.sh and expect
environment variables to be loaded. If these scripts are executed
stand-alone, the environment variables need to be loaded from the python
package.
Remove conf.sh.in and move the environment variables into isctest/vars
python package. This enabled the removal of an ugly pytest hack which
loaded and parsed these variables from the environment.
qp and rbtdb produce stylistically different backup files. This
was causing the xferquota system test to fail. This has been
addressed by making the test independent of the stylistic differences.
This test is outdated because it tested the 'sig-validity-interval'
option that has been replaced by dnssec-policy's 'signatures-validity',
'signatures-refresh', and 'signatures-jitter' options.
Nevertheless, it tests if the jitter is spread correctly.
Update the test to make use of 'signatures-jitter', set the value
to 1 day (meaning resign in 499 days since 'signatures-validity' is
set to 500 days).
Note that this previously changed erroneously the refresh value to
449 days (should have been 499 days, but that is not allowed by
checkconf, since it is above 90% of 'signatures-validity').
After we have changed the maximum allowed iterations to 51 for signing,
the NSEC3 chain has changed and requires one more NSEC to be returned
in the answer (plus corresponding RRSIG). So the expected number or
records in the authority section is now 8.
If the key is offline and the keymgr runs, it will treat it as a missing key,
and generate a new key (according to the policy). Fix the test by putting
back the KSK temporarily when we run 'rndc loadkeys'.
1. When generating keys, don't set timing metadata. Otherwise keys
are considered to be in use and won't be selected when dnssec-policy
starts a new key rollover.
2. Add an extra check to make sure the new ZSK (zsk2) is prepublished.
Also add a check to make sure it has become active.
3. When using dnssec-settime, add -s to also write to key state files.
The config was recently modified to ensure ns4 won't leak any queries to
root servers. However, the test wasn't executed and it turns out the way
this was handled actually broke the test case. Add our custom root hint
to both of the views to ensure the test can still pass without leaking
any queries.
An RPZ response's SOA record TTL is set to 1 instead of the SOA TTL,
a boolean value is passed on to query_addsoa, which is supposed to be
a TTL value. I don't see what value is appropriate to be used for
overriding, so we will pass UINT32_MAX.
Creating the KSR happens on the "ZSK side". The KSK is offline and while
the public key and state file may be present, draft-icann-dnssec-keymgmt-01.txt
suggest that the KSR only contains ZSKs.
This is also what knot dns does, so it would also be in the spirit of
interoperability.
The final line in a KSR ";; KeySigningRequest generated at ..." was
missing the version number, that has now been fixed.
Thanks Libor Peltan for reporting.
the previous commit introduced a possible race in getsigningtime()
where the rdataset header could change between being found on the
heap and being bound.
getsigningtime() now looks at the first element of the heap, gathers the
locknum, locks the respective lock, and retrieves the header from the
heap again. If the locknum has changed, it will rinse and repeat.
Theoretically, this could spin forever, but practically, it almost never
will as the heap changes on the zone are very rare.
we simplify matters further by changing the dns_db_getsigningtime()
API call. instead of passing back a bound rdataset, we pass back the
information the caller actually needed: the resigning time, owner name
and type of the rdataset that was first on the heap.
Add test cases for the 'request' command. Reuse the earlier
pregenerated ZSKs. We also need to set up some KSK files, that can
be done with 'dnssec-keygen -k <policy> -fK' now.
The 'check_keys()' function is adjusted such that the expected active
time of the successor key is set to the inactive time of the
predecessor. Some additional information is saved to make 'request'
testing easier.
Add a system test for testing dnssec-ksr, initally for the keygen
command. This should be able to create or select key files given a
DNSSEC policy and a time window.
Introduce a new DNSSEC tool, dnssec-ksr, for creating signed key
response (SKR) files, given one or more key signing requests (KSRs).
For now it is just a dummy tool, but the future purpose of this utility
is to pregenerate ZSKs and signed RRsets for DNSKEY, CDNSKEY, and CDS
for a given period that a KSK is to be offline.
Check that RFC 1918 leak detection does not trigger an assertion
when nxdomain redirection is enabled in the server but not for the
RFC 1918 reverse namespace.
The condition in prereq.sh which attempts to match two string uses
integer equality operation. This results in an error, causing the
enginepkcs11 test to always be skipped. Use = operator for the string
comparison instead.
The autosign test uses sleep in many cases to wait for something to
happen. This inevitably leads to an instability that manifests in our
CI. Allow an automatic rerun of the test to improve its stability.
Variable assignment when calling subroutines might not be portable.
Notably, it doesn't work with FreeBSD shell, where the value of HOME
would be ignored in this case.
Since the commands are already executed in a subshell, export the HOME
variable to ensure it is properly handled in all shells.
Initializing the conftest logging upon importing the isctest package
isn't practical when there are standalone pieces which can be used
outside of the testing framework, such as the asyncdnsserver module.
Implement a new Python class, AsyncDnsServer, which can be used by
ans.py scripts placed in ansX/ system test subdirectories. This enables
conveniently starting a feature-limited, non-standards-compliant, custom
DNS server instance. It can read and serve zone files, but it is also
able to evaluate any user-provided query-processing logic, allowing
query responses to be changed, delayed, or dropped altogether. These
are all actions commonly taken by custom DNS servers written in Python
that are used in BIND 9 system tests. Having a single "base"
implementation of such a custom DNS server reduces code duplication,
improving test maintainability.
Co-authored-by: Tom Krizek <tkrizek@isc.org>
Be stricter in durations that are accepted. Basically we accept ISO 8601
formats, but fail to detect garbage after the integers in such strings.
For example, 'P7.5D' will be treated as 7 days. Pass 'endptr' to
'strtoll' and check if the endptr is at the correct suffix.
Add a regression test case for the scenario where a secure chain of
trust includes an inactive KSK, that is a KSK that is not signing the
DNSKEY RRset.
now that "qpzone" databases are available for use in zones, we no
longer need to retain the zone semantics in the "qp" database.
all zone-specific code has been removed from QPDB, and "configure
--with-zonedb" once again takes two values, rbt and qp.
some database API methods that are never used with a cache have
been removed from qpdb.c and qp-cachedb.c; these include newversion,
closeversion, subtractrdataset, and nodefullname.
add database API methods needed for loading rdatasets into memory
(currentversion, beginload, endload), plus the methods used by
zone_postload() for zone consistency checks (getoriginnode, find,
findnode, findrdataset, attachnode, detachnode, deletedata).
the QP trie doesn't support the find callback mechanism available
in dns_rbt_findnode() which allows examination of intermediate nodes
while searching, so the detection of wildcard and delegation nodes
is now done by scanning QP chains after calling dns_qp_lookup().
Note that the lookup in previous_closest_nsec() cannot return
ISC_R_NOTFOUND. In RBTDB, we checked for this return value and
ovewrote the result with ISC_R_NOMORE if it occurred. In the
qpzone implementation, we insist that this return value cannot happen.
dns_qp_lookup() would only return ISC_R_NOTFOUND if we asked for a
name outside the zone's authoritative domain, and we never do that
when looking up a predecessor NSEC record.
named-checkzone is now able to load a zone and check it for errors,
but cannot dump it.
by default, QPDB is the database used by named and all tools and
unit tests. the old default of RBTDB can now be restored by using
"configure --with-zonedb=rbt --with-cachedb=rbt".
some tests have been fixed so they will work correctly with either
database.
CHANGES and release notes have been updated to reflect this change.
The change from RBT to QP has changed the contents of generated zone
files slightly: node names are now always absolute, so instead of using
$ORIGIN and relative names, generated zone files use full names for all
records.
This caused a failure in the xferquota system test, which was looking
for a relative name in secondary zone files. Replace the string
matching with a regular expression to fix the test.
the dyndb test requires a mechanism to retrieve the name associated
with a database node, and since the database no longer uses RBT for
its underlying storage, dns_rbt_fullnamefromnode() doesn't work.
addressed this by adding dns_db_nodefullname() to the database API.
the change from RBT to QP has changed the contents of generated zone
files slightly: node names are now always absolute, so instead of using
$ORIGIN and relative names, generated zone files use full names for all
records.
this caused a failure in the stub system test, which was grepping for a
relative name in a dumped zone file. using "masterfile-style full" makes
the test pass regardless of the database being used.
as a side effect of the switch from RBT to QBDB, NSEC3 records
are no longer created for empty non-terminal nodes when the
node only contains insecure delegations in an opt-out range.
such NSEC3 records are optional according to RFC 5155 (and,
for example, they are not created by dnssec-signzone), but they were
previously created by named, as a harmless side effect of the RBT
structure, which contains empty internal nodes that can be reached
by a DB iterator. these nodes are not present in the QPDB, so
NSEC3 records are not created unless they're actually required.
the autosign system test contained a test case (added in commit
ad91a70d as part of GL #4027) that checked whether ENT NSEC3
records were deleted when the delegations under the ENT removed.
this test no longer passes, because the NSEC3's are not created
in the first place, and therefore cannot be removed.
rather than "fix" the QPDB to add unnecessary NSEC3 records, this
commit instead revises the test to check for removal of ENT NSEC3
records when *not* using opt-out.
replace the string "rbt" throughout BIND with "qp" so that
qpdb databases will be used by default instead of rbtdb.
rbtdb databases can still be used by specifying "database rbt;"
in a zone statement.
This is a regresssion test for GL #4621 where the NODATA responses
are SOA records that match the QNAME rather than the zone name. In
particular for NS queries.
the RRL test included a test case that tried to start named with
a broken configuration. the same error could be found with
named-checkconf, so it should have been tested in the checkconf
system test.
When the first parametrized test takes a bit longer than usual, the zone
transfer in ns3 may succeed before the second parametrized test is even
started, and then watch_log_from_here() won't find the "Transfer status:
success" message in the named log. Using watch_log_from_start() instead
makes sure the test is more stable.
This was mostly an artifact to tell which log lines belong to which test
from the time when the test output could be all mingled together. Now
this info is reduntant, because the pytest logger already includes both
the system test name, and the specific test.
Unify the different loggers (conftest, module, test) into a single
interface. Remove the need to select the proper logger by automatically
selecting the most-specific logger currently available.
This also removes the need to use the logger/mlogger fixtures manually
and pass these around. This was especially annoying and unwieldy when
splitting the test cases into functions, because logger had to always be
passed around. Instead, it is now possible to use the
isctest.log.(debug,info,warning,error) functions.
Preparation for further logging improvements - keep the watchlog
contents in a separate module inside isctest.log. Export the names in
the log package so the imports don't change for the users of these
classes.
This code has probably been accidentally added during some rebase. The
actual RNDCExecutor and related classes are in isctest/rndc.py. Remove
the duplicated and unused code from isctest/log.py, as it doesn't belong
there.
We were missing a test where a single owner name would have multiple
types with a different case. The generated RRSIGs and NSEC records will
then have different case than the signed records and message parser have
to cope with that and treat everything as the same owner.
When running `make check` on a platform which has older (but still
supported) pytest, e.g. 3.4.2 on EL8, the junit to trs conversion would
fail because the junit format has different structure. Make the junit
XML processing more lenient to support both the older and newer junit
XML formats.
The ditch.pl script is used to generate burst traffic without waiting
for the responses. When running other tests in parallel, this can result
in a ephemeral port clash, since the ditch.pl process closes the socket
immediately. In rare occasions when the message ID also clashes with
other tests' queries, it might result in an UnexpectedSource error from
dnspython.
Use a dedicated port EXTRAPORT8 which is reserved for each test as a
source port for the burst traffic.
Stop the cname_and_other_data processing if we already know that the
result is true. Also, we know that CNAME will be placed in the priority
headers, so we can stop looking for CNAME if we haven't found CNAME and
we are past the priority headers.
Currently we test the incoming zone transfers data in the statistics
channel by retransfering the zones in slow mode and capturing the XML
and JSON outputs in the meantime to check their validity. Add a new
transfer to the test, and check that the XML and JSON files correctly
indicate that we have 3 retransfers and 1 new (first time) transfer.
Explicitly use an empty 'trust-anchors' statement in the system
tests where it was used implicitly before.
In resolver/ns5/named.conf.in use the trust anchor in 'trusted.conf',
which was supposed to be used there.
Add checks into the 'checkconf' system test to make sure that the
'dnssec-validation yes' option fails without configured trusted
anchors, and succeeds with configured non-empty, as well as empty
trusted anchors.
the 'low', 'high' and 'discount' parameters to 'fetch-quota-param'
are meant to be ratios with values between zero and one, but higher
values can be assigned. this could potentially lead to an assertion
in maybe_adjust_quota().
In the second test we are looking for key files and extract the key
id numbers. Because keys can be in different directories, we needed
to change the maxdepth when searching for keys.
For the second kasp system test, check that 'dnssec-keygen -k' (default
policy) creates valid files, the 'get_keyids' returned more than one
keytag, namely the ones that are inside the keys/ directory, that were
created for the predecessor test, check that 'dnssec-keygen -k'
(configuredd policy) creates valid files.
This caused the system test to spew out errors that key files were
missing (we were looking for key files in the current directory, but
when looking for key id numbers we included the keys/ directory). It
could also cause the next test to fail, check that 'dnssec-settime' by
default does not edit key state file, because the STATE_FILE environment
variable was overwritten with the key file path of one of the keys that
were created with the configured policy.
We fix this by adjusting the maxdepth for the test in question. Other
tests don't need adjusting because they use unique zone names.
The name "uri" was considered to be too generic and could potentially
clash with a future URI configuration option. Renamed to "pkcs11-uri".
Note that this option name was also preferred over "pkcs11uri", the
dash is considered to be the more clearer form.
Add test cases for zones in different views that are using PKCS#11
tokens to store its keys.
If it is using the same DNSSEC policy, only one PKCS#11 token should be
created and the same key should be used for the zone in both views.
If it is using a different DNSSEC policy, multiple PKCS#11 token should
be created and each view should use their respective key.
This log may still occur if there is a DNSKEY in the unsigned zone.
This may happen in a multi-signer setup for example.
Ideally this should not log a warning, but that requires looking up
keys a different way (by searching for key files only). However, that
requires adapting a bunch of system tests, and is out of scope for now.
- Shell function body should be in between curly braces.
- Some erroneous '|| return 1' are replaced with '|| ret=1'.
- Fix a variable name (was 'ret', should be '_ret').
- Clean up when setting up a new test.
The bullseye and bookworm images are not set up with pkcs11-provider,
so we need to add an additional prerequisite for running the
pkcs11engine test. Check the path of OPENSSL_CONF.
Move dns_dnssec_findzonekeys from the dnssec.{c,h} source code to
zone.{c,h} (the header file already commented that this should be done
inside dns_zone_t).
Alter the function in such a way, that keys are searched for in the
key stores if a 'dnssec-policy' (kasp) is attached to the zone,
otherwise keep using the zone's key-directory.
When using dnssec-policy with dnssec-keygen in combination with setting
the key-directory on the command line, the commandline argument takes
priority over the key-directory from the default named.conf.
Add a test case where dnssec-policy uses key stores with a directory
other than the zone's key-directory.
This requires changing the kasp shell script to take into account that
keys can be in different directories. When looking for keys, the
'find' command now takes a maxdepth of 3 to also look for keys in
subdirectories. Note this maxdepth value is arbitrary, the added
'keystore.kasp' test only requires a maxdepth of 2.
Because of this change, the dnssec-keygen tests no longer work because
they are for the same zone (although different directories). Change
the test to use a different zone ('kasp2' instead of 'kasp').
Add cases for each algorithm to test the interaction between
dnssec-policy and engine_pkcs11. Ensure that named creates keys on
startup.
Also test dnssec-keygen when using a dnssec-policy with a PKCS#11
based key-store.
The check for printing zone list failed because of these additional
lines in the output:
good.conf:22: dnssec-policy: key algorithm 13 has predefined length; \
ignoring length value 256
I am not sure why this failure hasn't happened before already.
Similar to key-directory, check for zones in different views and
different key and signing policies. Zones must be using different key
directories to store key files on disk.
Now that a key directory can be linked with a dnssec-policy key, the
'keydirexist' checking needs to be reshuffled.
Add tests for bad configuration examples, named-checkconf should catch
those. Also add test cases for a mix of key-directory and key-store
directory.
Similar to key-directory, check if the key-store directory exists and
if it is an actual directory.
This commit fixes an accidental test bug in checkconf where if
the "warn key-dir" test failed, the result was ignored.
Add checkconf check to ensure that the used key-store in the keys
section exists. Error if that is not the case. We also don't allow
the special keyword 'key-directory' as that is internally used to
signal that the zone's key-directory should be used.
Add new configuration for setting key stores. The new 'key-store'
statement allows users to configure key store backends. These can be
of type 'file' (that works the same as 'key-directory') or of type
'pkcs11'. In the latter case, keys should be stored in a HSM that is
accessible through a PKCS#11 interface.
Keys configured within 'dnssec-policy' can now also use the 'key-store'
option to set a specific key store.
Update the checkconf test to accomodate for the new configuration.
These tests have ns1 configured as a mock root server. Make sure it is
used in all config files of those tests, otherwise some queries could
leak to root nameservers.
Some tests don't have a mock root server configured, because they don't
need one. However, these tests might still leak queries to actual name
servers. Add a shared root hints file which can serve as a blackhole for
these queries.
As dnsrps and native test cases have been properly split up, the
ckdnsrps.sh script is no longer used anywhere, as the logic for
selecting these test cases is handled by pytest.
Previously, dnsrps test was executed as an optional part of the rpz and
rpzrecurse system tests. This was conceptually problematic, as the test
took the responsibility of running parts of the test framework -
cleaning files and setting up servers again.
Instead, allow these tests to execute either the native variant, or the
dnsrps one. To ensure the same test coverage, trigger both of these
variants as separate test cases from pytest.
We need to skip some portions the system test in FIPS mode as some of
the algorithms used in the test are not available when using the FIPS
mode (e.g. TLS_CHACHA20_POLY1305_SHA256)
The older version of the code was reporting that listeners are going
to be of the same type after reconfiguration when switching from DoT
to HTTPS listener, making BIND abort its executions.
That was happening due to the flaw in logic due to which the code
could consider a current listener and a configuration for the new one
to be of the same type (DoT) even when the new listener entry is
explicitly marked as HTTP.
The checks for PROXY in between the configuration were masking that
behaviour, but when porting it to 9.18 (when there is no PROXY
support), the behaviour was exposed.
Now the code mirrors the logic in 'interface_setup()' closely (as it
was meant to).
This commit adds a system test that helps to verify that changing a
listener transport by editing "listen-on" statements before
reconfiguration works as expected.
This commit adds a new system test which verifies that using the
'cipher-suites' option actually works as expected (as well as adds
first TLSv1.3 specific tests).
The "exceeded time limit waiting for literal 'too many DNS UPDATEs
queued' in ns1/named.run" is prone to fail due to a timing issue.
Despite out efforts to stabilize it, the check still often fails on
FreeBSD in our CI. Allow the test to be re-run on this platform.
This file was missing explicit dnssec-validation. Seems like it was
missed in our previous efforts, probably because of the different
filename / extension. Rename it to end with *.in to reflect that it is a
template file used by copy_setports.
If the dnskey-ttl in the dnssec-policy doesn't match the DNSKEY's
ttl then the DNSKEY, CDNSKEY and CDS rrset should be updated by
named to reflect the expressed policy. Check that named does this
by creating a zone with a TTL that does not match the policy's TTL
and check that it is correctly updated.
In Net::DNS 1.42 $ns->main_loop no longer loops. Use current methods
for starting the server, wait for SIGTERM then cleanup child processes
using $ns->stop_server(), then remove the pid file.
Reconfiguring named using RNDC is a common action in BIND 9 system
tests. It involves sending the "reconfig" RNDC command to a named
instance and waiting until it is fully processed. Add a reconfigure()
method to the NamedInstance class in order to simplify and standardize
named reconfiguration using RNDC in Python-based system tests.
TODO:
- full reconfiguration support (w/templating *.in files)
- add an "rndc null" before every reconfiguration to show which file
is used (NamedInstance.add_mark_to_log() as it may be generically
useful?)
The "checkds" system test contains a lot of duplicated code despite
carrying out the same set of actions for every tested scenario
(zone_check() → wait for logs to appear → keystate_check()). Extract
the parts of the code shared between all tests into a new function,
test_checkds(), and use pytest's test parametrization capabilities to
pass distinct sets of test parameters to this new function, in an
attempt to cleanly separate the fixed parts of this system test from the
variable ones. Replace format() calls with f-strings.
The "checkds" system test only uses dns.resolver.Resolver objects to
access their 'nameservers' and 'port' attributes. Instances of the
NamedInstance class also expose that information via their attributes,
so only pass NamedInstance objects around instead of needlessly
depending on dns.resolver.Resolver.
Make log file watching in Python-based system tests consistent by
employing the helper Python classes designed for that purpose. Drop the
custom code currently used.
Waiting for a specific log line to appear in a named.run file is a
common action in BIND 9 system tests. Implement a set of Python classes
which intend to simplify and standardize this task in Python-based
system tests.
Co-authored-by: Štěpán Balážik <stepan@isc.org>
The "addzone" and "shutdown" system tests currently invoke rndc using
test-specific helper code. Rework the relevant bits of those tests so
that they use the helper classes from bin/tests/system/isctest.py.
Controlling named instances using RNDC is a common action in BIND 9
system tests. However, there is currently no standardized way of doing
that from Python-based system tests, which leads to code duplication.
Add a set of Python classes and pytest fixtures which intend to simplify
and standardize use of RNDC in Python-based system tests.
For now, RNDC commands are sent to servers by invoking the rndc binary.
However, a switch to a native Python module able to send RNDC commands
without executing external binaries is expected to happen soon. Even
when that happens, though, having the capability to invoke the rndc
binary (in order to test it) will remain useful. Define a common Python
interface that such "RNDC executors" should implement (RNDCExecutor), in
order to make switching between them convenient.
Co-authored-by: Štěpán Balážik <stepan@isc.org>
When changing the NSEC3 chain, the new NSEC3 chain must be built before
the old NSEC3PARAM is removed. Check each delta in the conversion to
ensure this ordering is met.
When transitioning from NSEC3 to NSEC the NSEC3 must be built before
the NSEC3PARAM is removed. Check each delta in the conversion to
ensure this ordering is met.
When transitioning from NSEC3 to NSEC the added records where not
being signed because the wrong time was being used to determine if
a key should be used or not. Check that these records are actually
signed.
The check_loaded() function compares the zone's loadtime value and
an expected loadtime value, which is based on the zone file's mtime
extracted from the filesystem.
For the secondary zones there may be cases, when the zone file isn't
ready yet before the zone transfer is complete and the zone file is
dumped to the disk, so a so zero value mtime is retrieved.
In such cases wait one second and retry until timeout. Also modify
the affected check to allow a possible difference of the same amount
of seconds as the chosen timeout value.
certain dig options which were deprecated and became nonoperational
several releases ago still had documentation in the dig man page and
warnings printed when they were used: these included +mapped,
+sigchase, +topdown, +unexpected, +trusted-key, and the -i and -n
options. these are now all fatal errors.
another option was described as deprecated in the man page, but
the code to print a warning was never added. it has been added now.
This commit extends the 'doth' system tests with additional secondary
NS instance that reuses the same 'tls' entry for connecting the the
primary to download zones. This configurations were known to crash
secondaries in some cases.
This commit adds a system test suite for PROXYv2. The idea on which it
is based is simple:
1. Firstly we check that 'allow-proxy' and 'allow-proxy-on' (whatever
is using the new 'isc_nmhandle_real_localaddr/peeraddr()') do what
they intended to do.
2. Anything else that needs an interface or peer address (ACL
functionality, for example) is using the old
'isc_nmhandle_localaddr/peeraddr()' - which are now returning
addresses received via PROXY (if any) instead of the real connection
addresses. The beauty of it that we DO NOT need to verify every bit of
the code relying on these functions: whatever works in one place will
work everywhere else, as these were the only functions that allowed
any higher level code to get peer and interface addresses.
This way it is relatively easy to see if PROXYv2 works as intended.
The system tests need to be updated because non-zero iterations are no
longer accepted.
The autosign system test changes its iterations from 1 to 0 in one
test case. This requires the hash to be updated.
The checkconf system test needs to change the iterations in the good
configuration files to 0, and in the bad ones to 1 (any non-zero value
would suffice, but we test the corner case here). Also, the expected
failure message is change, so needs to be adjusted.
The nsec3 system test also needs iteration configuration adjustments.
In addition, the test script no longer needs the ITERATIONS environment
variable.
In the process of updating the system tests, I noticed an error
in the dnssec-policy "nsec3-other", where the salt length in one
configuration file is different than in the other (they need to be
the same). Furthermore, the 'rndc signing -nsec3param' test case
is operated on the zone 'nsec-change.kasp', so is moved so that the
tests on the same zone are grouped together.
Create a utility package for code shared by the python tests. The
utility functions should use reasonable defaults and be split up into
modules according to their functionality.
Ensure assert rewriting is enabled for the modules to get the most
useful output from pytest.
By default, the useful assertion message rewrite is used by pytest for
test modules only. Since another module is imported with shared
functionality, ensure it has pytest's assertion message rewriting
enabled to obtain more debug information in case it fails.
This file is executed outside of pytest with pure python, which doesn't
do any AssertionError message rewriting like pytest. Ensure the assert
messages in this file provide a useful debug message.
Rewrite and reorganize the test documentation to focus on the pytest
runner, omit any mentions of the legacy runner which are no longer
relevant, and mention a few pytest tricks.
The ns2/named-alt3.conf.in config file was removed in
f8e264ba6d. From then on, system test
reports:
sed: can't read ns2/named-alt3.conf.in: No such file or directory"
Drop the last remnant of ns2/named-alt3.conf.in.
when transferring in a non-inline-signing secondary for the first time,
we previously never set the value of zone->loadtime, so it remained
zero. this caused a test failure in the statschannel system test,
and that test case was temporarily disabled. the value is now set
correctly and the test case has been reinstated.
The AES algorithm for DNS cookies was being kept for legacy reasons, and
it can be safely removed in the next major release. Remove both the AES
usage for DNS cookies and the AES implementation itself.
Not every element tagged `skipped` in the JUnitXML tree has to contain
the `type` attribute. An example of that is a test that results in
xpass.
This has been verified with pytest version 7.4.2 and prior.
Add a test case where serve-stale is enabled on a server that also
servers a local authoritative zone.
The particular case tests a lame delegation and checks if falling
back to serving stale data does not attempt to retrieve the query
by recursing from the root down.
The lock-file configuration (both from configuration file and -X
argument to named) has better alternatives nowadays. Modern process
supervisor should be used to ensure that a single named process is
running on a given configuration.
Alternatively, it's possible to wrap the named with flock(1).
All changes in this commit were automated using the command:
shfmt -w -i 2 -ci -bn . $(find . -name "*.sh.in")
By default, only *.sh and files without extension are checked, so
*.sh.in files have to be added additionally. (See mvdan/sh#944)
When named fails to starts due to not being able to obtain
a lock on the lock file that lock file should remain. Check
that the lock file exists before and after the attempt to
start a second instance of named.
We test EDNS requests returning FORMERR where named is expected
to retry without EDNS.
We test EDNS requests returning NOTIMP where named is expected
to fail the transfer as the remote end is not protocol compliant.
The server is expected to retry the transfer using SOA and if
the returned serial is greater than the current serial AXFR.
Check the log that IXFR is request.
Now that inline-signing is ignored when there is no dnssec-policy,
add 'dnssec-policy default;' to the zones when attempting to add them
via 'rndc addzone'.
The 'dynamic-signed-inline-signing.kasp' zone was set up with
the environment variable 'ksktimes', but that should be 'csktimes'
which is set one line above. Since the values are currently the same
the behavior is identical, but of course it should use the correct
variable.
The 'step4.enable-dnssec.autosign' zone was set up twice. This is
unnecessary.
Add a test scenario for a dynamic zone that uses inline-signing which
accidentally has signed the raw version of the zone.
This should not trigger resign scheduling on the raw version of the
zone.
The usage of the newline in the replacement part of the 'sed' call
works in GNU systems, but not in OpenBSD. Use 'awk' instead.
Also use the extended syntax of regular expressions for 'grep', which
is similarly more portable across the supported systems.
'rndc thaw' initiates asynchrous loading of all the zones
similar to 'rndc load'. Wait for the test zone's load to
complete before testing that it is updatable again.
Instead of creating new memory pools for each new dns_message, change
dns_message_create() method to optionally accept externally created
dns_fixedname_t and dns_rdataset_t memory pools. This allows us to
preallocate the memory pools in ns_client and dns_resolver units for the
lifetime of dns_resolver_t and ns_clientmgr_t.
The XFRST_INITIALSOA state in the xfrin module is named like that,
because the first RR in a zone transfer must be SOA. However, the
name of the state is a bit confusing (especially when exposed to
the users with statistics channel), because it can be mistaken with
the refresh SOA request step, which takes place before the zone
transfer starts.
Rename the state to XFRST_ZONEXFRREQUEST (i.e. Zone Transfer Request).
During that step the state machine performs several operations -
establishing a connection, sending a request, and receiving/parsing
the first RR in the answer.
Add two more secondary zones to ns3 to be transferred from ns1,
using its IPv6 address for which the 'tcp-only' is set to 'yes'.
Check the statistics channel's incoming zone transfers information
to confirm that the expected transports were used for each of the
SOA query cases (UDP, TCP, TLS), and also for zone transfers (TCP,
TLS).
This allows the statistics channel to be viewed in a browser while
the transfer is in progress. Also set the transfer format to
one-answer to extend the amount of time the re-transfer takes.
When running the statschannel test on its own, use
<http://10.53.0.3:5304/xml/v3/xfrins> to see the output.
Note: the port is subject to future change.
Use the named -T transferslowly test options to slow down a zone
transfer from the primary server, and test that it's correctly
exposed in the statistics channel of the secondary server, while
it's in-progress.
Apply the semantic patch to catch all the places where we pass 'char' to
the <ctype.h> family of functions (isalpha() and friends, toupper(),
tolower()).
The Unix Domain Sockets support in BIND 9 has been completely disabled
since BIND 9.18 and it has been a fatal error since then. Cleanup the
code and the documentation that suggest that Unix Domain Sockets are
supported.
The previous symlink name convention was prone to name collisions If a
system test contained both a shell test and a pytest module of the same
name (e.g. dnstap test has both tests.sh and tests_dnstap.py), then
these would have the same convenience symlink, which could cause test
setup issues as well as confusion when examining test artifacts.
Update the naming convention to include the full pytest module name.
This results in a slightly more verbose names for shell tests (e.g.
dnstap_sh_dnstap instead of the previous dnstap_dnstap), but it removes
the chance of a collision.
Reorganize individual port fixtures and re-use the ports fixture to
obtain their number. Store it as integer and only cast it to string when
setting it as environment variable.
Remove code fork for legacy runner, reorganize imports and move a
pylint-silencing snippet to the top of the file. The rest of the code
was just unindented.
In order to python system tests, pytest (runner) has to be used
directly. This makes it possible to simplify the pytest runner and make
its behavior simpler and easier to extend.
The legacy runner can still be used to run shell system tests.
The legacy runner no longer uses make check. Ensure the legacy runner
script doesn't interact with that automake target in any way. The legacy
runner script remains available to execute the legacy runner, but there
is no out-of-the box support for running tests in parallel. Other tools
such as xargs can be utilized for that.
