opnsense-src/tests
John Baldwin ecb3a7d43d netmap: Disable a buggy and unsafe test (sync_kloop_conflict)
This test starts two threads to verify that two concurrent threads
cannot enter the kernel loop on the same netmap context.  The test
even has a comment about a potential race condition where the first
thread enters the loop and is stopped before the second thread tries
to enter the loop.  It claims it is fixed by the use of a semaphore.
Unfortunately, the semaphore doesn't close the race.

In the CI setup for CHERI, we run the testsuite once a week against
various architectures using single CPU QEMU instances.  Across
multiple recent runs of the plain "aarch64" test the job ran for an
entire day before QEMU was killed by a timeout.  The last messages
logged were from this test:

734.881045 [1182] generic_netmap_attach     Emulated adapter for tap3312 created (prev was NULL)
734.882340 [ 321] generic_netmap_register   Emulated adapter for tap3312 activated
734.882675 [2224] netmap_csb_validate       csb_init for kring tap3312 RX0: head 0, cur 0, hwcur 0, hwtail 0
734.883042 [2224] netmap_csb_validate       csb_init for kring tap3312 TX0: head 0, cur 0, hwcur 0, hwtail 1023
734.915397 [ 820] netmap_sync_kloop         kloop busy_wait 1, direct_tx 0, direct_rx 0, na_could_sleep 0
736.901945 [ 820] netmap_sync_kloop         kloop busy_wait 1, direct_tx 0, direct_rx 0, na_could_sleep 0

From the timestamps, the synchronous kloop was entered twice 2 seconds
apart.  This corresponds to the 2 second timeout on the semaphore in
the test.  What appears to have happened is that th1 started and
entered the kernel where it spun in an endless busy loop.  This
starves th2 so it _never_ runs.  Once the semaphore times out, th1 is
preempted to run the main thread which invokes the ioctl to stop the
busy loop.  th1 then exits the loop and returns to userland to exit.
Only after this point does th2 actually run and execute the ioctl to
enter the kernel.  Since th1 has already exited, th2 doesn't error and
enters its own happy spin loop.  The main thread hangs forever in
pthread_join, and the process is unkillable (the busy loop in the
kernel doesn't check for any pending signals so kill -9 is ignored and
ineffective).

I don't see a way to fix this test, so I've just disabled it.  There
is no good way to ensurce concurrency on a single CPU system when one
thread wants to sit in a spin loop.  Someone should fix the netmap
kloop to respond to kill -9 in which case kyua could perhaps at least
timeout the individual test process and kill it.

Reviewed by:	vmaffione
Obtained from:	CheriBSD
Sponsored by:	AFRL, DARPA
Differential Revision:	https://reviews.freebsd.org/D49220
2025-03-06 13:22:25 -05:00
..
atf_python vnet tests: verify that we can load if_epair and if_bridge 2024-07-23 15:57:25 +02:00
ci ci: Redirect output for builds. 2024-05-23 11:59:40 -06:00
etc Remove residual blank line at start of Makefile 2024-07-15 16:43:39 -06:00
examples Remove residual blank line at start of Makefile 2024-07-15 16:43:39 -06:00
freebsd_test_suite Remove $FreeBSD$: two-line .h pattern 2023-08-16 11:54:16 -06:00
include tests: Test endian.h, byteswap.h, sys/endian.h and both endian.h and byteswap.h together 2024-10-15 17:14:42 -06:00
sys netmap: Disable a buggy and unsafe test (sync_kloop_conflict) 2025-03-06 13:22:25 -05:00
__init__.py testing: Add basic atf support to pytest. 2022-06-25 19:25:15 +00:00
conftest.py Testing: add framework for the kernel unit tests. 2023-04-14 15:47:55 +00:00
Kyuafile Remove $FreeBSD$: one-line lua tag 2023-08-16 11:55:34 -06:00
Makefile Remove residual blank line at start of Makefile 2024-07-15 16:43:39 -06:00
Makefile.depend Remove $FreeBSD$: one-line sh pattern 2023-08-16 11:55:03 -06:00
Makefile.inc0 Remove residual blank line at start of Makefile 2024-07-15 16:43:39 -06:00
README Remove $FreeBSD$: one-line bare tag 2023-08-16 11:55:20 -06:00

src/tests: The FreeBSD test suite
=================================

Usage of the FreeBSD test suite:
(1)  Run the tests:
       kyua test -k /usr/tests/Kyuafile
(2)  See the test results:
       kyua report

For further information on using the test suite, read tests(7):
       man tests

Description of FreeBSD test suite
=================================
The build of the test suite is organized in the following manner:

* The build of all test artifacts is protected by the MK_TESTS knob.
  The user can disable these with the WITHOUT_TESTS setting in
  src.conf(5).

* The goal for /usr/tests/ (the installed test programs) is to follow
  the same hierarchy as /usr/src/ wherever possible, which in turn drives
  several of the design decisions described below.  This simplifies the
  discoverability of tests.  We want a mapping such as:

    /usr/src/bin/cp/      -> /usr/tests/bin/cp/
    /usr/src/lib/libc/    -> /usr/tests/lib/libc/
    /usr/src/usr.bin/cut/ -> /usr/tests/usr.bin/cut/
    ... and many more ...

* Test programs for specific utilities and libraries are located next
  to the source code of such programs.  For example, the tests for the
  src/lib/libcrypt/ library live in src/lib/libcrypt/tests/.  The tests/
  subdirectory is optional and should, in general, be avoided.

* The src/tests/ hierarchy (this directory) provides generic test
  infrastructure and glue code to join all test programs together into
  a single test suite definition.

* The src/tests/ hierarchy also includes cross-functional test programs:
  i.e. test programs that cover more than a single utility or library
  and thus don't fit anywhere else in the tree.  Consider this to follow
  the same rationale as src/share/man/: this directory contains generic
  manual pages while the manual pages that are specific to individual
  tools or libraries live next to the source code.

In order to keep the src/tests/ hierarchy decoupled from the actual test
programs being installed --which is a worthy goal because it simplifies
the addition of new test programs and simplifies the maintenance of the
tree-- the top-level Kyuafile does not know which subdirectories may
exist upfront.  Instead, such Kyuafile automatically detects, at
run-time, which */Kyuafile files exist and uses those directly.

Similarly, every directory in src/ that wants to install a Kyuafile to
just recurse into other subdirectories reuses this Kyuafile with
auto-discovery features.  As an example, take a look at src/lib/tests/
whose sole purpose is to install a Kyuafile into /usr/tests/lib/.
The goal in this specific case is for /usr/tests/lib/ to be generated
entirely from src/lib/.

--