Commit graph

37254 commits

Author SHA1 Message Date
Mark Andrews
ee5da17e13 Add CHANGES for [GL #3799]
(cherry picked from commit 5c471c5d2c)
2023-01-20 00:47:51 +11:00
Mark Andrews
4508b0a1e2 Add missing node lock when setting node->wild in rbtdb.c
The write node lock needs to be held when setting node->wild in
add_wildcard_magic except when being called from loading_addrdataset
which is used to load the zone without locking during its initial
load.

(cherry picked from commit 81c24b8da2)
2023-01-20 00:46:24 +11:00
Arаm Sаrgsyаn
feed44fd51 Merge branch '3808-refactor-isc_nm_xfr_allowed-v9_18' into 'v9_18'
[9.18] Refactor isc_nm_xfr_allowed()

See merge request isc-projects/bind9!7413
2023-01-19 13:37:24 +00:00
Aram Sargsyan
969811fc77 Add a CHANGES note for [GL #3808]
(cherry picked from commit 5a1707aa8d)
2023-01-19 12:20:10 +00:00
Aram Sargsyan
8f209c7dcf Refactor isc_nm_xfr_allowed()
Return 'isc_result_t' type value instead of 'bool' to indicate
the actual failure. Rename the function to something not suggesting
a boolean type result. Make changes in the places where the API
function is being used to check for the result code instead of
a boolean value.

(cherry picked from commit 41dc48bfd7)
2023-01-19 12:20:10 +00:00
Ondřej Surý
1f6d54d1fb Merge branch '3801-reduce-memory-bloat-caused-by-delayed-view-detach-lock-order-inversion-v9_18' into 'v9_18'
[9.18] Detach the zone views outside of the zone lock

See merge request isc-projects/bind9!7408
2023-01-19 10:00:49 +00:00
Ondřej Surý
3d3c8a935a
Detach the zone views outside of the zone lock
Detaching the views in the zone_shutdown() could lead to
lock-order-inversion between adb->namelocks[bucket], adb->lock,
view->lock and zone->lock.  Detach the views outside of the section that
zone-locked.

(cherry picked from commit 978a0ef84c)
2023-01-19 10:21:19 +01:00
Ondřej Surý
579f3b66f1 Merge branch 'ondrej/replace-lgtm-with-codeql-add-python-ply-v9_18' into 'v9_18'
[9.18] Add python3-ply for ./configure to succeed on BIND 9.16 branch

See merge request isc-projects/bind9!7406
2023-01-19 09:16:37 +00:00
Ondřej Surý
a393573300
Add python3-ply to GitHub CodeQL configuration
BIND 9.16 needs Python and PLY packages for configure to succeed.
Unless we want to tweak the build script to exclude python, we need to
add python3-ply package to the CodeQL configuration.

(cherry picked from commit 0622d11366)
2023-01-19 10:12:04 +01:00
Ondřej Surý
ba1460407d Merge branch 'ondrej/replace-lgtm-with-codeql-v9_18' into 'v9_18'
[9.18] Add CodeQL GitHub Action

See merge request isc-projects/bind9!7403
2023-01-19 08:46:43 +00:00
Ondřej Surý
fd8fe4ba4e Add CodeQL GitHub Action
Replace the decommissioned LGTM service with CodeQL GitHub action.

(cherry picked from commit 77abe612a1)
2023-01-19 08:46:36 +00:00
Ondřej Surý
adfa48ed39 Merge branch '3795-speed-up-EVP_DigestInit_ex-v9_18' into 'v9_18'
[9.18] Avoid implicit algorithm fetch for OpenSSL EVP_MD family

See merge request isc-projects/bind9!7398
2023-01-18 23:33:53 +00:00
Ondřej Surý
9679417cfa Add CHANGES and release note for [GL #3795]
(cherry picked from commit 401294cf60)
2023-01-19 00:33:37 +01:00
Ondřej Surý
bf1a29e9e1 Use OpenSSL 1.x SHA_CTX API in isc_iterated_hash()
If the OpenSSL SHA1_{Init,Update,Final} API is still available, use it.
The API has been deprecated in OpenSSL 3.0, but it is significantly
faster than EVP_MD API, so make an exception here and keep using it
until we can't.

(cherry picked from commit 25db8d0103)
2023-01-19 00:33:37 +01:00
Ondřej Surý
a1dcbcab8d Use OpenSSL EVP_MD API directly in isc_iterated_hash()
Instead of going through another layer, use OpenSSL EVP_MD API directly
in the isc_iterated_hash() implementation.  This shaves off couple of
microseconds in the microbenchmark.

(cherry picked from commit 36654df732)
2023-01-19 00:32:51 +01:00
Ondřej Surý
cb4453faef Merge branch '3252-repair-isc_task_purgeevent-v9_18' into 'v9_18'
[9.18] Keep the list of scheduled events on the timer

See merge request isc-projects/bind9!7396
2023-01-18 23:32:35 +00:00
Ondřej Surý
a7e8e43829
Add CHANGES note for [GL #3252]
(cherry picked from commit a7cd0868a2)
2023-01-18 22:39:26 +01:00
Mark Andrews
80a052aaf6
Unlink the timer event before trying to purge it
as far as I can determine the order of operations is not important.

    *** CID 351372:  Concurrent data access violations  (ATOMICITY)
    /lib/isc/timer.c: 227 in timer_purge()
    221     		LOCK(&timer->lock);
    222     		if (!purged) {
    223     			/*
    224     			 * The event has already been executed, but not
    225     			 * yet destroyed.
    226     			 */
    >>>     CID 351372:  Concurrent data access violations  (ATOMICITY)
    >>>     Using an unreliable value of "event" inside the second locked section. If the data that "event" depends on was changed by another thread, this use might be incorrect.
    227     			timerevent_unlink(timer, event);
    228     		}
    229     	}
    230     }
    231
    232     void

(cherry picked from commit 98718b3b4b)
2023-01-18 22:39:26 +01:00
Ondřej Surý
e26aa4cbb1
Don't use reference counting in isc_timer unit
The reference counting and isc_timer_attach()/isc_timer_detach()
semantic are actually misleading because it cannot be used under normal
conditions.  The usual conditions under which is timer used uses the
object where timer is used as argument to the "timer" itself.  This
means that when the caller is using `isc_timer_detach()` it needs the
timer to stop and the isc_timer_detach() does that only if this would be
the last reference.  Unfortunately, this also means that if the timer is
attached elsewhere and the timer is fired it will most likely be
use-after-free, because the object used in the timer no longer exists.

Remove the reference counting from the isc_timer unit, remove
isc_timer_attach() function and rename isc_timer_detach() to
isc_timer_destroy() to better reflect how the API needs to be used.

The only caveat is that the already executed event must be destroyed
before the isc_timer_destroy() is called because the timer is no longet
attached to .ev_destroy_arg.

(cherry picked from commit ae01ec2823)
2023-01-18 22:39:26 +01:00
Ondřej Surý
7197cf2b7e
Remove isc_task_purge() and isc_task_purgerange()
The isc_task_purge() and isc_task_purgerange() were now unused, so sweep
the task.c file.  Additionally remove unused ISC_EVENTATTR_NOPURGE event
attribute.

(cherry picked from commit c17eee034b)
2023-01-18 22:06:24 +01:00
Ondřej Surý
a7055b01af
Set quantum to infinity for the zone loading task
When we are loading the zones, set the quantum to UINT_MAX, which makes
task_run process all tasks at once.  After the zone loading is finished
the quantum will be dropped to 1 to not block server when we are loading
new zones after reconfiguration.

(cherry picked from commit 87c4c24cde)
2023-01-18 18:04:41 +01:00
Ondřej Surý
68abe3fa06
Add isc_task_setquantum() and use it for post-init zone loading
Add isc_task_setquantum() function that modifies quantum for the future
isc_task_run() invocations.

NOTE: The current isc_task_run() caches the task->quantum into a local
variable and therefore the current event loop is not affected by any
quantum change.

(cherry picked from commit 15ea6f002f)
2023-01-18 18:04:41 +01:00
Ondřej Surý
5f141e2c7f
Keep the list of scheduled events on the timer
Instead of searching for the events to purge, keep the list of scheduled
events on the timer list and purge the events that we have scheduled.

(cherry picked from commit 3f8024b4a2f12fcd28a9dd813b6f1f3f11d506f2)
2023-01-18 18:04:41 +01:00
Ondřej Surý
be99507488
Repair isc_task_purgeevent(), clean isc_task_unsend{,range}()
The isc_task_purgerange() was walking through all events on the task to
find a matching task.  Instead use the ISC_LINK_LINKED to find whether
the event is active.

Cleanup the related isc_task_unsend() and isc_task_unsendrange()
functions that were not used anywhere.

(cherry picked from commit 17aed2f895)
2023-01-18 18:04:41 +01:00
Ondřej Surý
29bc0daed1 Merge branch '3801-reduce-memory-bloat-caused-by-delayed-view-detach-v9_18' into 'v9_18'
[9.18] Detach the views in zone_shutdown(), not in zone_free()

See merge request isc-projects/bind9!7388
2023-01-17 22:17:29 +00:00
Ondřej Surý
b4357fab6f
Add CHANGES and release note for [GL #3801]
(cherry picked from commit b049e329ef)
2023-01-17 22:47:44 +01:00
Ondřej Surý
fe0e9cf301
Commit the change of view for view->managed_keys
When we change the view in the view->managed_keys, we never commit the
change, keeping the previous view possibly attached forever.

Call the dns_zone_setviewcommit() immediately after changing the view as
we are detaching the previous view anyway and there's no way to recover
from that.

(cherry picked from commit 7e8b53720d)
2023-01-17 22:47:44 +01:00
Ondřej Surý
f8b7188a42
Detach the views in zone_shutdown(), not in zone_free()
The .view (and possibly .prev_view) would be kept attached to the
removed zone until the zone is fully removed from the memory in
zone_free().  If this process is delayed because server is busy
something else like doing constant `rndc reconfig`, it could take
seconds to detach the view, possibly keeping multiple dead views in the
memory.  This could quickly lead to a massive memory bloat.

Release the views early in the zone_shutdown() call, and don't wait
until the zone is freed.

(cherry picked from commit 13bb821280)
2023-01-17 22:47:43 +01:00
Artem Boldariev
5d41338e9c Merge branch '3767-dot-no-alpn-transfer-shutdown-crash-v9-18' into 'v9_18'
[9.18](backport )XoT: properly handle the case when checking for ALPN failed

See merge request isc-projects/bind9!7387
2023-01-17 19:53:54 +00:00
Artem Boldariev
26c0dd96eb Modify CHANGES [GL #3767]
Mention that downloading a zone via TLS- from a server which does not
negotiate "dot" ALPN tag could have crashed BIND on shutdown.

(cherry picked from commit 16fa692b56)
2023-01-17 21:29:38 +02:00
Artem Boldariev
dbd15f962e XoT: properly handle the case when checking for ALPN failed
During XoT it is important to check for "dot" ALPN tag to be
negotiated (according to the RFC 9103). We were doing that, however, the
situation was not handled properly, leading to non-cancelled zone
transfers that would crash (abort()) BIND on shutdown.

In this particular case 'result' might equal 'ISC_R_SUCCESS'. When
this is the case, the part of the code supposed to handle failures
will not cancel the zone transfer.

This situation cannot happen when BIND is a secondary of other BIND
instance. Only primaries following the RFC not closely enough could
trigger such a behaviour.

(cherry picked from commit 34a1aab1cb)
2023-01-17 21:27:44 +02:00
Tom Krizek
ed23fd4b1f Merge branch '3804-fix-feature-detection-in-pytests-v9_18' into 'v9_18'
[9.18] Fix feature detection for pytest markers in tests

See merge request isc-projects/bind9!7385
2023-01-17 14:29:50 +00:00
Tom Krizek
58ff3b4fc0
Fix feature detection for pytest markers in tests
The condition was accidentally reversed during refactoring in
9730ac4c56 . It would result in skipped
tests on builds with proper support and false negatives on builds
without proper feature support.

Credit for reporting the issue and the fix goes to Stanislav Levin.

(cherry picked from commit 473cb530f4)
2023-01-17 14:50:23 +01:00
Ondřej Surý
aee89c3454 Merge branch 'cherry-pick-d7bcdf8b' into 'v9_18'
[9.18] Merge branch 'feature/main/zt-rwlock.h' into 'main'

See merge request isc-projects/bind9!7377
2023-01-16 11:09:55 +00:00
Ondřej Surý
c5c432a36a Merge branch 'feature/main/zt-rwlock.h' into 'main'
Include isc_rwlocktype_t type definition in zt.h

See merge request isc-projects/bind9!7376

(cherry picked from commit d7bcdf8bd6)

395d6fca Include isc_rwlocktype_t type definition in zt.h
2023-01-16 11:07:19 +00:00
Tom Krizek
0d0c2e466b Merge branch 'tkrizek/system-tests-pytest-prep-v9_18' into 'v9_18'
[9.18] Various tweaks of system test framework

See merge request isc-projects/bind9!7374
2023-01-16 09:42:30 +00:00
Tom Krizek
f3c6217046
Update the TEST_PARALLEL_JOBS value in CI
The authoritative source for this value is in the project's CI/CD
Variables Setting. The reason to keep it in .gitlab-ci.yaml as well is
to have functional testing in forks without the need to manually specify
this variable in Settings.

The tests have been executed with 4 jobs for some time now. This
"change" only brings .gitlab-ci.yaml file up to date, it doesn't
actually change the number of jobs we currently use to test.

(cherry picked from commit 03d7b45d81)
2023-01-16 10:13:53 +01:00
Tom Krizek
dd5f0d2c4c
Tweak dupsigs test output
Use a different visual separator, since "====", "----" and "____" is
used by pytest to separate the log output.

(cherry picked from commit 9593ff9347)
2023-01-16 10:13:53 +01:00
Tom Krizek
8033857295
Look for ifconfig.sh.in in testsock.pl parent dir
Instead of using the current working directory to find the ifconfig.sh
script, look for the ifconfig.sh.in template in the directory where the
testsock.pl script is located. This enables the testsock.pl script to be
called from any working directory.

Using the ifconfig.sh.in template is sufficient, since it contains
the necessary information to be extracted: the max= value (which is
hard-coded in the template).

(cherry picked from commit e24d3b21d0)
2023-01-16 10:13:53 +01:00
Tom Krizek
bcf1f5c808
Factor out script to handle system test core dumps
Move the core dump detection functionality for system test runs into a
separate script. This enables reuse by the pytest runner. The
functionality remains the same.

(cherry picked from commit d9a97200d5)
2023-01-16 10:13:53 +01:00
Tom Krizek
db03296683
testcrypto.sh: run in TMPDIR if possible
Avoid creating any temporary files in the current workdir.

Additional/changing files in the bin/tests/system directory are
problematic for pytest/xdist collection phase, which assumes the list of
files doesn't change between the collection phase of the main pytest
thread and the subsequent collection phase of the xdist worker threads.

Since the testcrypto.sh is also called during pytest initialization
through conf.sh.common (to detect feature support), this could
occasionally cause a race condition when the list of files would be
different for the main pytest thread and the xdist worker.

(cherry picked from commit 61330a7863)
2023-01-16 10:13:51 +01:00
Michał Kępień
8b06de031e Merge branch 'michal/set-up-version-and-release-notes-for-bind-9.18.12' into 'v9_18'
Set up version and release notes for BIND 9.18.12

See merge request isc-projects/bind9!7363
2023-01-13 14:47:07 +00:00
Michał Kępień
b6d271a1af Set up release notes for BIND 9.18.12 2023-01-13 15:40:27 +01:00
Michał Kępień
6d89ad302b Update BIND version to 9.18.12-dev 2023-01-13 15:40:27 +01:00
Ondřej Surý
dec1bf29fd Merge branch '3212-implement-incremental-rehashing-for-isc_ht-hashtables-v9_18' into 'v9_18'
[9.18] Implement incremental hash table resizing in isc_ht

See merge request isc-projects/bind9!7355
2023-01-11 17:01:00 +00:00
Ondřej Surý
68c094c59a
Add CHANGES note for [GL #3212]
(cherry picked from commit 5ccb28d6d8)
2023-01-11 17:16:35 +01:00
Ondřej Surý
8c31a939c9
Implement incremental hash table resizing in isc_ht
Previously, an incremental hash table resizing was implemented for the
dns_rbt_t hash table implementation.  Using that as a base, also
implement the incremental hash table resizing also for isc_ht API
hashtables:

 1. During the resize, allocate the new hash table, but keep the old
    table unchanged.
 2. In each lookup, delete, or iterator operation, check both tables.
 3. Perform insertion operations only in the new table.
 4. At each insertion also move <r> elements from the old table to
    the new table.
 5. When all elements are removed from the old table, deallocate it.

To ensure that the old table is completely copied over before the new
table itself needs to be enlarged, it is necessary to increase the
size of the table by a factor of at least (<r> + 1)/<r> during resizing.

In our implementation <r> is equal to 1.

The downside of this approach is that the old table and the new table
could stay in memory for longer when there are no new insertions into
the hash table for prolonged periods of time as the incremental
rehashing happens only during the insertions.

(cherry picked from commit e42cb1f198)
2023-01-11 17:15:33 +01:00
Arаm Sаrgsyаn
c120cc40f5 Merge branch 'aram/dns_fwdtable_addfwd-cleanup-bugfix-v9_18' into 'v9_18'
[9.18] Fix dns_fwdtable_addfwd() error path cleanup bug

See merge request isc-projects/bind9!7354
2023-01-11 14:25:44 +00:00
Aram Sargsyan
39cc597022 Use sizeof(*ptr) for allocating/freeing memory in forward.c
As shown in the previous commit, using sizeof(type_t) is a little
bit more error-prone when copy-pasting code, so extracting the
size information from the pointer which is being dealt with seems
like a better alternative.

(cherry picked from commit cf4003fa58)
2023-01-11 13:42:09 +00:00
Aram Sargsyan
64c8e63942 Fix dns_fwdtable_addfwd() error path cleanup bug
Free 'sizeof(dns_forwarder_t)' bytes of memory instead of
'sizeof(dns_sockaddr_t)' bytes, because `fwd` is a pointer
to a 'dns_forwarder_t' type structure.

(cherry picked from commit 0cc1b06d98)
2023-01-11 13:42:09 +00:00