Commit graph

20 commits

Author SHA1 Message Date
Evan Hunt
b234c6d954 convert filter-aaaa test to python
use pytest for the filter-aaaa test.

In order to implement this, isctest.mark can now test whether IPv6 is
configured by calling testsock6.pl.
2025-08-05 23:01:03 -07:00
Evan Hunt
950df056b3 convert dnssec validation tests to python
begin converting DNSSEC validation tests from shell to python,
and simplify the name servers used in the test.

ns4, the name server used for validation tests, is now configured
using jinja2 templates.

ns8, which was previously used for testing unsupported, disabled
and revoked keys and trust anchors, has been removed.  we now
use a jinja2 configuration in ns5 for this purpose.

the configurations in ns7 and ns6 didn't conflict with one another,
so the two servers have been merged into one.
2025-07-31 12:55:40 -07:00
Nicki Křížek
d737986ea2 Turn on doctest in CI
Run doctests for the isctest module in a dedicated CI job.
2025-07-18 11:32:41 +02:00
Nicki Křížek
4c487c811d Use pytest.mark.flaky as the flaky marker
It's possible to use pytest.mark.flaky, which achieves the exact same
thing as our custom-defined isctest.mark.flaky -- attempts to rerun the
test on failure, but only is flaky package is available.
2025-07-07 13:29:15 +02:00
Nicki Křížek
34867e1693 Allow dnstap system test rerun on freebsd13
The check "unix socket message counts" sometimes fails with "dnstap
output file smaller than expected". This only happens on freebsd13 and
can't be reproduced easily. There was an attempt to decrease the
required file size in the past, but apparently, the issue can still
occur.
2025-07-07 13:29:15 +02:00
Nicki Křížek
66f6f4bba9 Allow reruns for test_json and test_xml tests
These tests have been unstable under TSAN in the past, but it appears
that the same failure mode can happen outside of TSAN tests as well.
These tests have produced 12 failures combined in the past three weeks
in nightlies.
2025-07-07 13:29:02 +02:00
Štěpán Balážik
3a8ffc74df Add a pytest mark for system tests that require Internet connection
We skip those by default as:

a) we don't want to stress the upstream servers in every CI pipeline
b) system tests need to be runnable in a isolated environment by default
2025-06-27 07:24:48 +00:00
Štěpán Balážik
c7a8f61055 Rename CI_ENABLE_ALL_TESTS to CI_ENABLE_LONG_TESTS
New name is more descriptive and allows us to created more optional
test sets.
2025-06-27 07:24:48 +00:00
Petr Špaček
313a985dfc Fix DNSTAP feature detection for pytest 2025-05-28 10:45:32 +00:00
Matthijs Mekking
7d670b7fe7 Parametrize the default kasp test cases
Make use of pytest.mark.parametrize to split up the many default kasp
test cases into separate tests.
2025-04-23 15:22:04 +00:00
Michal Nowak
409f394d6e Rewrite keyfromlabel system test to pytest 2025-02-17 19:18:04 +00:00
Michal Nowak
7c499d1689
Rewrite nzd2nzf system test to pytest 2025-02-13 18:43:47 +01:00
Michal Nowak
df7e9f4ac3 Rename have_* marks to with_*
Marks starting with "with" or "without" make more sense linguistically
than those starting with "have" or "have_not".
2025-01-24 08:45:51 +00:00
Michal Nowak
feecbd8e77 Add "without_fips" mark
The "without_fips" mark disables test function when BIND 9 was built
with the FIPS mode enabled as not everything works in FIPS-enabled
builds.
2025-01-24 08:45:51 +00:00
Michal Nowak
a8b70abca1
Add with_dnstap mark 2024-10-21 12:04:56 +02:00
Ondřej Surý
62d59766d6
Remove DNSRPS implementation
DNSRPS was the API for a commercial implementation of Response-Policy
Zones that was supposedly better.  However, it was never open-sourced
and has only ever been available from a single vendor.  This goes against
the principle that the open-source edition of BIND 9 should contain only
features that are generally available and universal.

This commit removes the DNSRPS implementation from BIND 9.  It may be
reinstated in the subscription edition if there's enough interest from
customers, but it would have to be rewritten as a plugin (hook) instead
of hard-wiring it again in so many places.
2024-09-18 17:39:14 +02:00
Nicki Křížek
ec2fc7680a
Allow rerun of unstable statschannel tests under TSAN
The test_traffic_json and test_traffic_xml occasionally fail when
running under TSAN. This happens in CI and is most likely a result of
some instability that doesn't seem to be easily reproduced.
2024-08-19 16:08:48 +02:00
Tom Krizek
cb55fb2cae
Split up the dnsrps and native variants of rpz system tests
Previously, dnsrps test was executed as an optional part of the rpz and
rpzrecurse system tests. This was conceptually problematic, as the test
took the responsibility of running parts of the test framework -
cleaning files and setting up servers again.

Instead, allow these tests to execute either the native variant, or the
dnsrps one. To ensure the same test coverage, trigger both of these
variants as separate test cases from pytest.
2024-01-18 15:28:28 +01:00
Tom Krizek
a65b654a23
Don't type-check the flaky plugin with mypy
Since we execute mypy for bin/tests/system/isctest package, this is now
needed because the flaky package doesn't have type hints.
2024-01-10 14:57:20 +01:00
Tom Krizek
f314f1b432
Move custom pytest markers into isctest module
Keep our pytest code more organized by moving the shared code for custom
pytest markers into a dedicated isctest/mark.py module.
2024-01-10 14:57:13 +01:00
Renamed from bin/tests/system/pytest_custom_markers.py (Browse further)