Cleanup the pattern in the decref() functions in both qpcache.c and
qpzone.c, so it follows the similar patter as we already have in
newref() function.
(cherry picked from commit 814b87da64)
Extended DNS error 22 (No reachable authority) was previously detected when `fctx_expired` fired. It turns out this function is used as a "safety net" and the timeout detection should be caught earlier.
It was working though, because of another issue fixed by !9927. But then, the recursive request timed out detection occurs before `fctx_expired` making impossible to raise the EDE 22 error.
This fixes the problem by triggering the EDE 22 in the part of the code detecting the (TCP or UDP) time out and taking the decision to cancel the whole fetch (i.e. There is no other server to attempt to contact).
Note this is not targeting users (no release note) because there is no release versions of BIND between !9927 and this changes. Thus a release note would be confusing.
Closes#5137
Backport of MR !9985
Merge branch 'backport-5137-ede22-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10001
When EDE 3 (stale answer) was added the serve-stale tests were checking
for those exclusively, i.e. grepping for no "EDE" in the dig output when
no stale answer was expected.
However, some stale tests disable stale answers and make the
authoritative server unresponsive, effectively triggering a timed out
request thus an EDE 22. Update those tests so they still tests the
absence of EDE 3 error, but also the presence of EDE 22.
(cherry picked from commit 27f3b8950a)
This re-do a previously existing EDE 22 system test as well as add
another one making sure the timed out flow detection works also on UDP
when the resolver is contacting the authoritative server. (the existing
test was using TCP to contact the authoritative servers).
(cherry picked from commit 7cb8a028fe)
Extended DNS error 22 (No reachable authority) was previously detected
when `fctx_expired` fired. It turns out this function is used as a
"safety net" and the timeout detection should be caught earlier.
It was working though, because of another issue fixed by !9927. Since
this change, the recursive request timed out detection occurs before
`fctx_expired` so EDE 22 is not added to the response message anymore.
The fix of the problem is to add the EDE 22 code in two situations:
- When the dispatch code timed out (rctx_timedout) the resolver code
checks various properties to figure out if it needs to make another
fetch attempt. One of the paramters if the fetch expiration time. If
it expires, the whole recursion is canceled, so it now adds the EDE 22
code.
- If the fetch expiration time doesn't expires in the case above (and
other parameters allows it) a new fetch attempt is made (fctx_query).
But before the new request is actually made, the fetch expiration time
is re-checked. It might then has elapsed, and the whole recursion is
canceled. So it now also adds the EDE 22 code here as well.
(cherry picked from commit 78274ec2b1)
Changes introducing the support of extended DNS error code 1 and 2 uses
SHA-1 digest for some tests which break FIPS platform. The digest itself
was irrelevant, another digest is used.
Backport of MR !10002
Merge branch 'backport-colin/fix-fips-9807-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10031
Changes !9948 introducing the support of extended DNS error code 1 and 2
uses SHA-1 digest for some tests which break FIPS platform. The digest
itself was irrelevant, another digest is used.
(cherry picked from commit d82262d293)
Instead of mixing the dns_resolver and dns_validator units directly with
the EDE code, split-out the dns_ede functionality into own separate
compilation unit and hide the implementation details behind abstraction.
Additionally, the new dns_edelist_t doesn't have to be copied into all
responses as those are attached to the fetch context, but it could be
only passed by reference.
This makes the dns_ede implementation simpler to use, although sligtly
more complicated on the inside.
Closes#5141
Backport of MR !10016
Merge branch 'backport-5141-ede-mem-fix-v2-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10030
Because the new introduced code in main doesn't use the log context
anymore, manually add the log context for isc_log_write usages in the
new ede.c file.
In order to avoid to loop to find the next position to store an EDE in
a dns_edectx_t, add a "nextede" state which holds the next available
position.
Also, in order ot avoid to loop to find if an EDE is already existing in
a dns_edectx_t, and avoid a duplicate, use a bitmap to immediately know
if the EDE is there or not.
Those both changes applies for adding or copying EDE.
Also make the direction of dns_ede_copy more explicit/avoid errors by
making "edectx_from" a const pointer.
(cherry picked from commit 9021f9d802)
Add documentation usage of EDE compilation unit as well as centralize
all EDE-related macros in the same lib/dns/include/dns/ede.h header.
(cherry picked from commit 7b01cbfb04)
Migrate tests cases in client_test code which were exclusively testing
code which is now all wrapped inside ede compilation unit. Those are
testing maximum number of EDE, duplicate EDE as well as truncation of
text of an EDE.
Also add coverage for the copy of EDE from an edectx to another one, as
well as checking the assertion of the maximum EDE info code which can be
used.
(cherry picked from commit f9f41190b3)
Instead of mixing the dns_resolver and dns_validator units directly with
the EDE code, split-out the dns_ede functionality into own separate
compilation unit and hide the implementation details behind abstraction.
Additionally, the EDE codes are directly copied into the ns_client
buffers by passing the EDE context to dns_resolver_createfetch().
This makes the dns_ede implementation simpler to use, although sligtly
more complicated on the inside.
Co-authored-by: Colin Vidal <colin@isc.org>
Co-authored-by: Ondřej Surý <ondrej@isc.org>
(cherry picked from commit 2f8e0edf3b)
From https://gitlab.isc.org/isc-projects/bind9/-/issues/5087, the relevant MRs have been merged in the January 2025 release. Hence this MR removes `allow_failure: true` in CI.
Backport of MR !10026
Merge branch 'backport-andoni/remove-allow-failure-in-cross-version-config-tests-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10027
Remove `allow_failure: true` from .gitlab-ci.yaml, as the relevant
MRs have been merged in the January 2025 release.
(cherry picked from commit 5d07df807a)
Reduce the amount of artifacts stored by running make clean at the end
of unit and system test run. If any of the previous commands fail, the
runner will stop executing the commands in `script` immediately, so the
cleanup only happens if none of the previous commands failed.
The build artifacts from unit and system tests are re-used anywhere and
should be safe to throw away immediately.
Backport of MR !10015
Merge branch 'backport-nicki/reduce-ci-artifacts-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10018
Reduce the amount of artifacts stored by running make clean at the end
of unit and system test run. If any of the previous commands fail, the
runner will stop executing the commands in `script` immediately, so the
cleanup only happens if none of the previous commands failed.
The build artifacts from unit and system tests are re-used anywhere and
should be safe to throw away immediately. Same for respdiff.
(cherry picked from commit e59bc5b366)
The iana.org link is gone.
Backport of MR !10010
Merge branch 'backport-mnowak/linkcheck-fix-draft-icann-dnssec-keymgmt-01.txt-link-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10011
The prior regex didn't match the actual names we use for release
branches in the private repo. This caused the merged-metadata job to not
be created upon merging to a release branch, resulting in the private MR
not being properly milestoned.
Use the correct regex along with protecting the v9.*-release branches in
the gitlab UI so that they have access to the token used to perform the
required API operations.
Backport of MR !10003
Merge branch 'backport-nicki/ci-fix-post-merge-in-private-repo-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10013
The prior regex didn't match the actual names we use for release
branches in the private repo. This caused the merged-metadata job to not
be created upon merging to a release branch, resulting in the private MR
not being properly milestoned.
Use the correct regex along with protecting the v9.*-release branches in
the gitlab UI so that they have access to the token used to perform the
required API operations.
(cherry picked from commit 2e5a3bde7e)
Add DoH and DoT stress test jobs. The DoH scenario on FreeBSD is omitted
because all Flamethrower's DoH queries timeout on this platform.
Since the response rate of DoT queries is lower than that of DoH and
TCP, the expected TCP response rate is 80%.
Due to the large number of similar stress test configurations, the
"util/generate-stress-test-configs.py" script now generates them as part
of a downstream pipeline. The script is expected to be run exclusively
within the CI environment, which sources all environmental variables and
files.
This refactoring brought the following changes:
- To start a stress test immediately and not wait for artifacts of the
autoreconf job, run the "autoreconf -fi" command as part of every job.
- Drop the BIND_STRESS_TEST_* variables as they were rarely used and
conflicted with mode and platform selection in the configuration
generator.
- Most pipelines now include a few short, randomly selected stress test
jobs. To schedule all stress tests, set the ALL_BIND_STRESS_TESTS
environmental variable, push a tag to CI, or run a scheduled pipeline.
- Set the BIND_STRESS_TESTS_RUN_TIME environmental variable to pick the
stress test runtime of your choosing, set the BIND_STRESS_TESTS_RATE
environmental variable to set different than the default query rate.
- Job timeout is set to 30 minutes plus stress test runtime in minutes.
(cherry picked from commit 9756292a5f)
Some detected links are not to be verified (127.*, dnssec-or-not.com)
and some I can't fix (flaticon, godaddy, icann), but they are not
crucial.
(cherry picked from commit 8302469507)
The changelog job is supposed to test that the text from GitLab MR
title&description is valid rst syntax and can be built with sphinx. In
49128fc1, the way gitchangelog generates entries was changed - it no
longer writes to the changelog file, but generates output on stdout
instead. Ensure the generated notes is actually written to (some)
rendered file which is part of the docs so that the subsequent sphinx
build attempts to render the note.
Backport of MR !9804
Merge branch 'backport-nicki/ci-fix-changelog-job-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9997
The changelog job is supposed to test that the text from GitLab MR
title&description is valid rst syntax and can be built with sphinx. In
49128fc1, the way gitchangelog generates entries was changed - it no
longer writes to the changelog file, but generates output on stdout
instead. Ensure the generated notes is actually written to (some)
rendered file which is part of the docs so that the subsequent sphinx
build attempts to render the note.
(cherry picked from commit 380a30ba8d)
Add support for EDE codes 1 & 2 which might occurs during DNSSEC validation in case of unsupported RRSIG algorithm or DNSKEY digest.
See #2715
Backport of MR !9948
Merge branch 'backport-2715-ede-unsupported-digest-alg-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9996
A DNSSEC validation can fail in the case where multiple DNSKEY are
available for a zone and none of them are supported, but for different
reasons: one has a DS record in the parent zone using an unsupported
digest while the other one uses an unsupported encryption algorithm.
Add a specific test case covering this flow and making sure that two
extended DNS error are provided: code 1 and 2, each of them highlighting
unsupported algorithm and digest.
(cherry picked from commit 244923b9dc)
Add support for EDE codes 1 (Unsupported DNSKEY Algorithm) and 2
(Unsupported DS Digest Type) which might occurs during DNSSEC
validation in case of unsupported DNSKEY algorithm or DS digest type.
Because DNSSEC internally kicks off various fetches, we need to copy
all encountered extended errors from fetch responses to the fetch
context. Upon an event, the errors from the fetch context are copied
to the client response.
(cherry picked from commit 46a58acdf5)
The minimal dnspython version to run this test is 2.5.0.
Backport of MR !8662
Merge branch 'backport-mnowak/pytest_rewrite_cipher-suites-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9992
Marks starting with "with" or "without" make more sense linguistically
than those starting with "have" or "have_not".
(cherry picked from commit df7e9f4ac3)
Ensure the zone transfers have completed (successfully or not) before
running the test cases, because they assume zone transfers have been
done.
(cherry picked from commit 23fb615963)
The servers are setup and torn down once per each test module. All the
logs and server state persists between individual tests within the same
module. The servers fixture representing these servers should be
module-wide as well.
(cherry picked from commit a72ff9fd57)
The minimal required dnspython version is 2.5.0 because of the need for
the "verify" argument in dns.query.tls().
(cherry picked from commit 100b759863)
The base image tends to have a rather old dnspython version and when
used with pylint and mypy it produces errors about newer dnspython
features the old version does not know about.
$ mypy "bin/tests/system/isctest/"
bin/tests/system/isctest/query.py:55: error: Unexpected keyword argument "verify" for "tls" [call-arg]
/usr/lib/python3/dist-packages/dns/query.py:958: note: "tls" defined here
$ pylint --rcfile $CI_PROJECT_DIR/.pylintrc --disable=wrong-import-position $(git ls-files 'bin/tests/system/*.py' | grep -vE 'ans\.py')
************* Module isctest.query
bin/tests/system/isctest/query.py:55:11: E1123: Unexpected keyword argument 'verify' in function call (unexpected-keyword-arg)
(cherry picked from commit b2964cc922)
When explicitly set to True, the "verify" argument lets dnspython verify
certificates used for the connection. As most certificates in the system
test will inevitably be self-signed, the "verify" argument defaults to
False.
The "verify" argument is present in dnspython since the version 2.5.0.
(cherry picked from commit df8c419058)
The "without_fips" mark disables test function when BIND 9 was built
with the FIPS mode enabled as not everything works in FIPS-enabled
builds.
(cherry picked from commit feecbd8e77)
Adjust the limit of maximum disagreements in respdiff results based on
recent pipeline results.
The respdiff and respdiff:asan seem to have almost identical results,
typically around 0.07 % of differences with ocassional spikes up to
around 0.11 %. Similar results are for respdiff:tsan, perhaps with more
common spikes with values up to around 0.12 %. Set the limit to 0.15 %
to allow for some tolerance due to network conditions, time of day etc.
The respdiff:third-party has a slightly higher disagreements average,
with typical values being around 0.12 %. Set the limit to 0.2 %.
Exceeding either of those values should be quite clear indication that
some resolution behaviour has changed, since the values appear to be
very stable within the newly configured limits.
Backport of MR !9950
Merge branch 'backport-nicki/ci-respdiff-limits-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9989
Adjust the limit of maximum disagreements in respdiff results based on
recent pipeline results.
The respdiff and respdiff:asan seem to have almost identical results,
typically around 0.07 % of differences with ocassional spikes up to
around 0.11 %. Similar results are for respdiff:tsan, perhaps with more
common spikes with values up to around 0.12 %. Set the limit to 0.15 %
to allow for some tolerance due to network conditions, time of day etc.
The respdiff:third-party has a slightly higher disagreements average,
with typical values being around 0.12 %. Set the limit to 0.2 %.
Exceeding either of those values should be quite clear indication that
some resolution behaviour has changed, since the values appear to be
very stable within the newly configured limits.
(cherry picked from commit 0584d3f65f)
Closes#5123
Backport of MR !9966
Merge branch 'backport-5123-document-refreshing-a-secondary-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9986
In the case when `dnssec-signzone` is called on an already signed zone, and the private key file is unavailable, a signature that needs to be refreshed may be dropped without being able to generate a replacement. This has been fixed.
Closes#5126
Backport of MR !9951
Merge branch 'backport-5126-dnssec-signzone-retain-rrsig-if-key-is-offline-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9982
