These test cases follow the same pattern as many other, but all require
some additional checks. These are set in "additional-tests".
The "zsk-missing.autosign" zone is special handled, as it expects the
KSK to sign the SOA RRset (because the ZSK is unavailable).
The kasp/ns3/setup.sh script is updated so the SyncPublish is not set
(named will initialize it correctly). For the test zones that have
missing private key files we do need to set the expected key timing
metadata.
Remove the counterparts for the newly added test from the kasp shell
tests script.
(cherry picked from commit 5f23f750c24ea734e52798276bbeb270cec2aed2)
The check_signatures code was initially created to be suitable for
the ksr system test, to test the Offline KSK feature. For that, a
key is expected to be signing if the current time is between
the timing metadata Active and Retired.
With dnssec-policy, the key timing metadata is indicative, the key
states determine the actual signing behavior.
Update the check_signatures function so that by default the signing
is derived from the key states (ksigning and zsigning). Add an
argument 'offline_ksk', if set the make sure that the zsigning is set
if the current time is between the Active and Retired timing metadata,
and for ksigning we just use the timing metadata (as the key is offline,
we cannot check the key states).
Another (upcoming) test case is where key files are missing. When the
ZSK private key file is missing, the KSK takes over. Add an argument
'zsk_missing', when set to True the expected zone signing (zsigning)
is reversed.
(cherry picked from commit fddf9f778b49f454e68cbefea8c897ac3bd0ea44)
The zone 'pregenerated.kasp' is a case where there already exist more
keys than required. For this we set the 'pregenerated' setting. This
will change the 'keydir_to_keylist' function behavior: Only keys in use
are considered. A key is in use if all of the states are either
undefined, or set to 'hidden'.
The 'some-keys.kasp' zone is similar to 'pregenerated.kasp', except
only some keys have been pregenerated.
(cherry picked from commit 43ded45ae9af1b5ad93a68444ac289574ae703a2)
Write python-based tests for the many test cases from the kasp system
test. These test cases all follow the same pattern:
- Wait until the zone is signed.
- Check the keys from the key-directory against expected properties.
- Set the expected key timings derived from when the key was created.
- Check the key timing metadata against expected timings.
- Check the 'rndc dnssec -status' output.
- Check the apex is signed correctly.
- Check a subdomain is signed correctly.
- Verify that the zone is DNSSEC correct.
Remove the counterparts for the newly added test from the kasp shell
tests script.
(cherry picked from commit 41481af1446ad3b0e319214b3ceee8805fd67e72)
When ``stale-answer-client-timeout 0`` option was enabled, it could be ignored
when resolving a zone which is a delegation of an authoritative zone belonging
to the resolver. This has been fixed.
Closes#5275
Backport of MR !10381
Merge branch 'backport-5275-stale-answer-client-timeout-0-and-delegation-fix-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10420
Add a new test which gets an answer for a delegated zone, then
checks whether the 'stale-answer-client-timeout 0' mode (i.e. the
'stalefirst' mode) works for it.
(cherry picked from commit 441b7d53f4c023f891f7404130b191bb575501c2)
When 'stale-answer-client-timeout' is 0, named is allowed to return
a stale answer immediately, while also initiating a new query to get
the real answer. This mode is activated in ns__query_start() by setting
the 'qctx->options.stalefirst' optoin to 'true' before calling the
query_lookup() function, but not when the zone is known to be
authoritative to the server. When the zone is authoritative, and
query_looup() finds out that the requested name is a delegation,
then before proceeding with the query, named tries to look it up
in the cache first. Here comes the issue that it doesn't consider
enabling 'qctx->options.stalefirst' in this case, and so the
'stale-answer-client-timeout 0' setting doesn't work for those
delegated zones - instead of immediately returning the stale answer
(if it exists), named tries to resolve it.
Fix this issue by enabling 'qctx->options.stalefirst' in the
query_zone_delegation() function just before named looks up the name
in the cache using a new query_lookup() call. Also, if nothing was
found in the cache, don't initiate another query_lookup() from inside
query_notfound(), and let query_notfound() do its work, i.e. it will
call query_delegation() for further processing.
(cherry picked from commit 412aa881f2ce0e9d07b9ab46d2d4863ba388b898)
With `dnssec-policy` you can pregenerate keys and if they are eligible, rather than creating a new key, a key is selected from the pregenerated keys. A key is eligible if it is unused, i.e it has no key timing metadata set.
Backport of MR !10385
Merge branch 'backport-matthijs-clarify-pregenerating-keys-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10388
With dnssec-policy you can pregenerate keys and if they are eligible,
rather than creating a new key, a key is selected from the pregenerated
keys. A key is eligible if it is unused, i.e it has no key timing
metadata set.
(cherry picked from commit 9880bfff63d853629afb85394a65dd4eaf9f90b0)
Apple broke custom memory allocation functions in the system-wide libxml2 starting with macOS Sequoia 15.4. Usage of the custom memory allocation functions has been disabled on macOS.
Closes#5268
Backport of MR !10374
Merge branch 'backport-5268-disable-libxml2-memory-management-on-macos-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10411
The custom allocation API for libxml2 is deprecated starting in macOS
Sequoia 15.4, iOS 18.4, tvOS 18.4, visionOS 2.4, and tvOS 18.4.
Disable the memory function override for libxml2 when
LIBXML_HAS_DEPRECATED_MEMORY_ALLOCATION_FUNCTIONS is defined as Apple
broke the system-wide libxml2 starting with macOS Sequoia 15.4.
(cherry picked from commit bf1b8824ac739c172f4b84a1de8b372a4da3509f)
Convert the first batch of tests from `kasp/tests.sh` to `kasp/tests_kasp.py`.
Backport of MR !10253
Merge branch 'backport-matthijs-pytest-rewrite-kasp-system-test-2-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10408
isctest.util was not imported so file_contents_contain could not be
found. And rename verify_keys to check_keys because it asserts in
isctest.run.retry_with_timeout.
(cherry picked from commit 62a6b9faa7ccf85b2b628fb991c8f72d2976eee3)
The kasp tests make use of dns.update.UpdateMessage and dns.tsig.Key,
that are introduced in dnspython 2.0.0.
(cherry picked from commit 46aead5a6df2337fa3dc758f2bd839c88efe611c)
This converts a special characters test case, a max-zone-ttl error
check, and two cases of insecure zones.
We no longer assert for having more than one DNSKEY and/or RRSIG
records. If the zone is insecure, this is no longer always true. And
we already check for the expected number of records in the
check_dnskeys/check_signatures functions.
(cherry picked from commit 07ac0e603633a701b35c92de2e3b27dc2643dcba)
This commit deals with converting the dynamic zone test cases to
pytest. The tests for 'inline-signing.kasp' are similar to the default
case, so these are added to 'test_kasp_default'.
Unfortunately I need to add sleep calls in between freezing, updating,
and thawing a zone. Without it the intermittent failures are too
frequent.
(cherry picked from commit 0b41afbd15da6d9564952b63ebe3df5f56ad7d4e)
This commit deals with converting the test cases related to the default
dnssec-policy.
This requires a new method 'check_update_is_signed'. This method will
be used in future tests as well, and checks if an expected record is
in the zone and is properly signed.
Remove the counterparts for the newly added test from the kasp shell
tests script.
(cherry picked from commit 4e22b019f5123c27bda7574ae1a5750f27d5bd4b)
Convert the first couple of tests from 'kasp/tests.sh' to
'kasp/tests_kasp.py', those are test cases related to 'dnssec-keygen'
and 'dnssec-settime'.
For this, we also add a new KeyProperties method,
'policy_to_properties', that takes a list of strings which represent
the keys according to the dnssec-policy and the expected key states.
(cherry picked from commit 00ea2c256482f334e3bd9ed52f20cc3a6a53660f)
The pthread-based implementation of the isc_rwlock_tryupgrade()
function always returns ISC_R_LOCKBUSY. Fix the test by adding
conditional checks.
Closes#5287
Backport of MR !10398
Merge branch 'backport-5287-pthread-rwlock-tryupgrade-test-fix-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10403
The pthread-based implementation of the isc_rwlock_tryupgrade()
function always returns ISC_R_LOCKBUSY. Fix the test by adding
conditional checks.
(cherry picked from commit c27659bc623f2d58bccc9e0d99ca1ad180e8180b)
`python-jinja2` is now required to run system tests.
Related #4938
Backport of MR !9588
Merge branch 'backport-nicki/replace-setup-sh-files-with-jinja2-templates-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10396
Many of the system tests now use jinja2 template engine. Adding jinja2
as a hard dependency is preferable than potentially silently skipping
many system tests.
(cherry picked from commit 543ba8da5a9ff45f2595a617d08d56dd281865a4)
These setup.sh scripts only do templating and copying files. Both of
these can be replaced with either jinja templates, or using plain files.
Since each test invocation creates its own temporary directory, copying
files to ensure a "clean" state is no longer necessary.
In cases where named writes some content to the files, a jinja template
can be used instead of a plain file to avoid an artifact check which
would detect a change to a git-tracked file.
(cherry picked from commit e0f0c557a0b305a83967a0e75c9f955f410a2cc9)
All these setup files only use copy_setports function which can be done
with jinja2 templates instead -- simply by renaming the .in files to
.j2, without any other changes. The pytest runner will render these
templates during test setup without any need for an additional script.
(cherry picked from commit a9f1b8d6a64c956f9f122b4459d24b8bccab3429)
This change allows the client to identify the server that returns the
BADVERS and to provide a DNS SERVER COOKIE to be included in the
resend of the request.
Closes#5235
Backport of MR !10334
Merge branch 'backport-5235-return-the-server-cookie-when-returning-badvers-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10392
DNS COOKIE and NSID should also be being processed when returning
BADVERS. Check that this has actually occured by looking for the
cookie and nsid in the response.
(cherry picked from commit f69b4bc5e0472035259dfc9df03ee0a9c02925dc)
A BADVERS response can also include EDNS SERVER COOKIE. Extract
that from the OPT record and use it when resending the request.
(cherry picked from commit 44140cad3b8e6567ec77175332f9da58da9d0617)
This will help identify the broken server if we happen to break
EDNS version negotiation. It will also help protect the client
from spoofed BADVERSION responses.
(cherry picked from commit 0d9cab15551be8cdc06969c39eaf851732cda51d)
The original check_pid() always returned 0 on FreeBSD, even if the
process was still running. This makes the "verifying that named checks
for conflicting named processes" check fail on FreeBSD with TSAN.
Backport of MR !10373
Merge branch 'backport-mnowak/fix-runtime-pid-check-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10386
The original check_pid() always returned 0 on FreeBSD, even if the
process was still running. This makes the "verifying that named checks
for conflicting named processes" check fail on FreeBSD with TSAN.
(cherry picked from commit 6acaca963d0a8b1715c1e6c6b38eec45b1c5d63c)
Replace the custom DNS servers used in the "forward" system test with new
code based on the isctest.asyncserver module.
For ans6, instead of configuring the responses to send at runtime, set
them up when the server is started. Make sure the server supports
toggling response sending at runtime to enable simulating forwarder
timeouts as required by one of the checks.
For ans11, put most of the responses to be provided by that server into
a zone file, only retaining code modifying zone-based answers in the
form of a response handler, to improve code readability. Use explicit
domain names instead of variables as that server only handles a single
domain and fixed strings improve readability in this case. Make sure
the server supports toggling response sending at runtime to enable
simulating forwarder timeouts as required by one of the checks.
Migrate sendcmd() and its uses to the new way of sending control queries
to custom servers used in system tests.
Depends on !10339
Backport of MR !10340
Merge branch 'backport-michal/forward-asyncserver-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10384
Replace the custom DNS servers used in the "forward" system test with
new code based on the isctest.asyncserver module.
For ans6, instead of configuring the responses to send at runtime, set
them up when the server is started. Make sure the server supports
toggling response sending at runtime to enable simulating forwarder
timeouts as required by one of the checks.
For ans11, put most of the responses to be provided by that server into
a zone file, only retaining code modifying zone-based answers in the
form of a response handler, to improve code readability. Use explicit
domain names instead of variables as that server only handles a single
domain and fixed strings improve readability in this case. Make sure
the server supports toggling response sending at runtime to enable
simulating forwarder timeouts as required by one of the checks.
Migrate sendcmd() and its uses to the new way of sending control queries
to custom servers used in system tests.
(cherry picked from commit 23bc8430d9fb02a8e977bc2569dbf972765e1a9c)
Some BIND 9 system tests need to dynamically change custom server
behavior at runtime. Existing custom servers typically use a separate
TCP socket for listening to control commands, which mimics what `named`
does, but adds extra complexity to the custom server's networking code
for no gain (given the purpose at hand). There is also no common way of
performing typical runtime actions (like toggling response dropping)
across all custom servers.
Instead of listening on a separate TCP socket in `asyncserver.py`, make
it detect DNS queries to a "magic" domain (`_control.`) on the same port
as the one it uses for receiving "production" DNS traffic. This enables
query/response logging code to be reused for control traffic, clearly
denotes behavior changes in packet captures, facilitates implementing
commonly used features as reusable chunks of code (by making them "own"
distinct subdomains of the control domain), voids the need for separate
tools sending control commands, and enables using DNS facilities for
returning information to the user (e.g. RCODE for status codes, TXT
records for additional information, etc.).
Backport of MR !10339
Merge branch 'backport-michal/asyncserver-control-commands-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10383
Implement a reusable control command that makes it possible to
dynamically disable/enable sending responses to clients. This is a
typical use case for custom DNS servers employed in various BIND 9
system tests.
(cherry picked from commit 92b39f8352661eb0999c09a660333fb134436782)
Some BIND 9 system tests need to dynamically change custom server
behavior at runtime. Existing custom servers typically use a separate
TCP socket for listening to control commands, which mimics what named
does, but adds extra complexity to the custom server's networking code
for no gain (given the purpose at hand). There is also no common way of
performing typical runtime actions (like toggling response dropping)
across all custom servers.
Instead of listening on a separate TCP socket in asyncserver.py, make it
detect DNS queries to a "magic" domain ("_control.") on the same port as
the one it uses for receiving "production" DNS traffic. This enables
query/response logging code to be reused for control traffic, clearly
denotes behavior changes in packet captures, facilitates implementing
commonly used features as reusable chunks of code (by making them "own"
distinct subdomains of the control domain), voids the need for separate
tools sending control commands, and enables using DNS facilities for
returning information to the user (e.g. RCODE for status codes, TXT
records for additional information, etc.).
(cherry picked from commit a7e1de716b8fff724b74e28c1087e5d0c1244c00)
With multiple and/or dynamically managed response handlers at play, it
becomes useful for debugging purposes to know which handler (if any) was
used for preparing each response sent by the server. Add debug logs
providing that information. Make class name the default string
representation of each response handler to prettify logs.
(cherry picked from commit 5e71fd081e36c2dc27b258b5c3b90f668e59bf74)
Extend AsyncDnsServer.install_response_handler() so that the provided
response handler can be inserted at the beginning of the handler list.
This enables installing a response handler that takes priority over all
previously installed handlers.
Add a new method, AsyncDnsServer.uninstall_response_handler(), which
enables removing a previously installed response handler.
Together, these two methods provide full control over the response
handler list at runtime.
(cherry picked from commit 92b072bff4376e02bf1d0cf8bd01b179fbea5358)
Add a main() function to all custom servers based on isctest.asyncserver
and move server startup code there. This prevents redefining variables
from outer scope in custom server code as it evolves.
(cherry picked from commit 8cb51d4c2b79795124b1a9e9e1a08e3008d65260)
Prevent custom servers based on asyncserver.py from exiting prematurely
due to unhandled exceptions raised as a result of attempting to parse
invalid queries sent by clients.
(cherry picked from commit fd0290c9192da1942628c0b556d8faecac3958b1)
The StreamWriter.wait_closed() method was introduced in Python 3.7, so
attempting to use it with Python 3.6 raises an exception. This has not
been noticed before because awaiting StreamWriter.wait_closed() is the
last action taken for each TCP connection and unhandled exceptions were
not causing the scripts based on AsyncServer to exit prematurely until
the previous commit.
As per Python documentation [1], awaiting StreamWriter.wait_closed()
after calling StreamWriter.close() is recommended, but not mandatory, so
try to use it if it is available, without taking any fallback action in
case it isn't.
[1] https://docs.python.org/3.13/library/asyncio-stream.html#asyncio.StreamWriter.close
(cherry picked from commit 715bd1b6678ae591b6edf34c1d5d748f04ad22d0)
Uncaught exceptions raised by tasks running on event loops are not
handled by Python's default exception handler, so they do not cause
scripts to die immediately with a non-zero exit code. Set up an
exception handler for AsyncServer code that makes any uncaught exception
the result of the Future that the top-level coroutine awaits. This
ensures that any uncaught exceptions cause scripts based on AsyncServer
to immediately exit with an error, enabling the system test framework to
fail tests in which custom servers encounter unforeseen problems.
(cherry picked from commit ec4c92d9d59d1ba0ee3242485965afbaeb62c847)
Move test code that can be reused for the kasp pytest-based system test.
Backport of MR !10252
Merge branch 'backport-matthijs-pytest-rewrite-kasp-system-test-1-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10382
In the kasp system test there are cases that the SyncPublish is not
set, nor it is required to do so. Update the _check_dnskeys function
accordingly.
(cherry picked from commit 0a6cc42914964cd83fe4973456fc54f8bff90c30)
For some kasp test we are going to need TSIG based queries to
differentiate between views.
(cherry picked from commit 9cb287afa0d428820d41577601cb2a763206a48b)
For the kasp tests we need a new utility that can retrieve a list of
Keys from a given directory, belonging to a specific zone. This is
'keydir_to_keylist' and is the replacement of 'kasp.sh:get_keyids()'.
'next_key_event_eqauls' is a method to check when the next key event is
scheduled, needed for the rollover tests, and is the equivalent of shell
script 'check_next_key_event'.
(cherry picked from commit 12e57eb222c3e4e721d5978d41c84efe0caadd77)
This commit introduces replacements for the 'check_keys' and
'check_keytimes' from the shell test library. 'check_keys' is renamed
to 'verify_keys' because it does not assert.
For that, we introduce more functions for the class Key. The
'match_properties' function is used in 'verify_keys' to see if a set of
KeyProperties match the Key. This speficially ignores timing metadata.
The function resembles what is in 'kasp.sh:check_key()'.
The 'match_timingmetadata' function is used in 'check_keytimes' to see
if the timing metadata of a set of KeyProperties match the Key. The
values are checked in all three key files (except if the private key is
not available (set with properties["private"]), or if it is a legacy key
(set with properties["legacy"]).
An additional check function is added, to check if the key relationships
are set correctly. It follows a similar pattern as 'check_keytimes'. If
"Predecessor" and/or "Successor" are expected to be set in the state
file, this function checks so, and also verifies that they are not set
if they should not be.
(cherry picked from commit 44ff63a50d660fd5e836b9bb0365d1ae6bdbf60a)
Because we want to check the metadata in all three files, a new
value in the Key class is added: 'privatefile'. The 'get_metadata'
function is adapted so that we can also check metadata in other files.
Introduce methods to easily retrieve the TTL and public DNSKEY record
from the keyfile.
When checking if the CDS is equal to the expected value, use the DNSKEY
TTL instead of hardcoded 3600.
(cherry picked from commit 97f6b7ad11c3c1d6c1ceeaada5776759649afa49)
