In isctest.kasp, introduce a new class 'KeyProperties' that can be used
to check if a Key matches expected properties. Properties are for the
time being divided in three parts: 'properties' that contain some
attributes of the expected properties (such as are we dealing with a
legacy key, is the private key available, and other things that do not
fit the metadata exactly), 'metadata' that contains expected metadata
(such as 'Algorithm', 'Lifetime', 'Length'), and 'timing', which is
metadata of the class KeyTimingMetadata.
The 'default()' method fills in the expected properties for the default
DNSSEC policy.
The 'set_expected_times()' sets the expected timing metadata, derived
from when the key was created. This method can take an offset to push
the expected timing metadata a duration in the future or back into the
past. If 'pregenerated=True', derive the expected timing metadata from
the 'Publish' metadata derived from the keyfile, rather than from the
'Created' metadata.
The calculations in the 'Ipub', 'IpubC' and 'Iret' methods are derived
from RFC 7583 DNSSEC Key Rollover Timing Considerations.
(cherry picked from commit 0b9fbca18e)
This is the first step of converting the kasp system test to pytest.
Well, perhaps not the first, because earlier the ksr system test was
already converted to pytest and then the `isctest/kasp.py` library
was already introduced. Lots of this code can be reused for the kasp
pytest code.
First of all, 'check_file_contents_equal' is moved out of the ksr test
and into the 'check' library. This feels the most appropriate place
for this function to be reused in other tests. Then, 'keystr_to_keylist'
is moved to the 'kasp' library.
Introduce two new methods that are unused in this point of time, but
we are going to need them for the kasp system test. 'zone_contains'
will be used to check if a signature exists in the zonefile. This way
we can tell whether the signature has been reused or refreshed.
'file_contents_contain' will be used to check if the comment and public
DNSKEY record in the keyfile is correct.
(cherry picked from commit ee8e9f1ded)
In PRIVATEOID keys, the key data begins with a length byte followed
by an ASN.1 object identifier that indicates the cryptographic algorithm
to use. Previously, the length byte was not accounted for when
checking the contents of keys and signatures, which could have led
to interoperability problems with any zones signed using PRIVATEOID.
This has been fixed.
Closes#5270
Backport of MR !10372
Merge branch 'backport-5270-fix-check-private-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10376
We were failing to account for the length byte before the OID.
See RFC 4034.
Algorithm number 254 is reserved for private use and will never be
assigned to a specific algorithm. The public key area in the DNSKEY
RR and the signature area in the RRSIG RR begin with an unsigned
length byte followed by a BER encoded Object Identifier (ISO OID) of
that length. The OID indicates the private algorithm in use, and the
remainder of the area is whatever is required by that algorithm.
Entities should only use OIDs they control to designate their private
algorithms.
(cherry picked from commit ca7355b7d0)
Reformat the section to be more consistent with the rest of the rndc
documentation and avoid using :program: directive which would needlessly
break rst links.
(cherry picked from commit 22bd41e308)
Caching prevents server upgrades being detected in a timely manner
and it can also prevent DNSSEC responses being requested.
(cherry picked from commit 90b2f94d9b)
When multiple nested DNS validations were destroyed out of order,
the EDE context could be freed before all EDE codes were copied,
which could cause an assertion failure. This has been fixed.
Closes#5213
Backport of MR !10365
Merge branch 'backport-5213-use-dns_ede_copy-in-dns_validator-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10366
If the nested DNS validator ends up in the same fetch because of the
loops, the code could be copying the EDE codes from the same source EDE
context as the destination EDE context. Skip copying the EDE codes if
the source and the destination is the same.
(cherry picked from commit 2988ebae21)
Pass NULL as edectx for the fetch_and_forget() fetches as nobody
is reading the EDE contexts and it can mess the main client buffer.
(cherry picked from commit fe48290140)
Instead of passing the edectx from the fetchctx into all subvalidators,
make the ede context ownership explict for dns_resolver_createfetch()
callers, and copy the ede result codes from the children validators to
the parent when finishing the validation process.
(cherry picked from commit d7593196a1)
TSAN reports a lock-order-inversion (potential deadlock) issue in
`add_trace_entry()`.
While it is true that in one case a lock in the `isc_mem_t` structure is
locked first, and then a lock in the `FILE` structure is locked second,
and in the the second case it is the other way around, this isn't an
issue, because those are `FILE` structures for totally different files,
used in different parts of the code.
Closes#5266
Backport of MR !10355
Merge branch 'backport-5266-freebsd-suppress-tsan-lock-order-inversion-false-positive-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10362
TSAN reports a lock-order-inversion (potential deadlock) issue in
add_trace_entry():
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M0001 (0x000000000001) => M0002 (0x000000000002) => M0001
Mutex M0002 acquired here while holding mutex M0001 in main thread:
#0 _pthread_mutex_lock /usr/src/contrib/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1342:3
#1 add_trace_entry lib/isc/mem.c:210:2
#2 isc__mem_get lib/isc/mem.c:606:2
#3 isc_buffer_allocate lib/isc/./include/isc/buffer.h:1080:23
#4 pushandgrow lib/isc/lex.c:321:3
#5 isc_lex_gettoken lib/isc/lex.c:445:22
#6 cfg_gettoken lib/isccfg/parser.c:3490:11
#7 cfg_parse_mapbody lib/isccfg/parser.c:2230:3
#8 cfg_parse_obj lib/isccfg/parser.c:247:11
#9 parse2 lib/isccfg/parser.c:628:11
#10 cfg_parse_file lib/isccfg/parser.c:668:11
#11 load_configuration bin/named/server.c:8069:13
#12 run_server bin/named/server.c:9518:2
#13 isc__async_cb lib/isc/async.c:110:3
#14 uv__async_io /tmp/libuv-1.50.0/src/unix/async.c:208:5
#15 uv__io_poll /tmp/libuv-1.50.0/src/unix/kqueue.c:369:9
#16 uv_run /tmp/libuv-1.50.0/src/unix/core.c:460:5
#17 loop_thread lib/isc/loop.c:327:6
#18 thread_body lib/isc/thread.c:89:8
#19 isc_thread_main lib/isc/thread.c:124:2
#20 isc_loopmgr_run lib/isc/loop.c:513:2
#21 main bin/named/main.c:1469:2
Mutex M0001 previously acquired by the same thread here:
#0 _pthread_mutex_lock /usr/src/contrib/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1342:3
#1 _flockfile /usr/src/lib/libc/stdio/_flock_stub.c:72:3
#2 cfg_gettoken lib/isccfg/parser.c:3490:11
#3 cfg_parse_mapbody lib/isccfg/parser.c:2230:3
#4 cfg_parse_obj lib/isccfg/parser.c:247:11
#5 parse2 lib/isccfg/parser.c:628:11
#6 cfg_parse_file lib/isccfg/parser.c:668:11
#7 load_configuration bin/named/server.c:8069:13
#8 run_server bin/named/server.c:9518:2
#9 isc__async_cb lib/isc/async.c:110:3
#10 uv__async_io /tmp/libuv-1.50.0/src/unix/async.c:208:5
#11 uv__io_poll /tmp/libuv-1.50.0/src/unix/kqueue.c:369:9
#12 uv_run /tmp/libuv-1.50.0/src/unix/core.c:460:5
#13 loop_thread lib/isc/loop.c:327:6
#14 thread_body lib/isc/thread.c:89:8
#15 isc_thread_main lib/isc/thread.c:124:2
#16 isc_loopmgr_run lib/isc/loop.c:513:2
#17 main bin/named/main.c:1469:2
Mutex M0001 acquired here while holding mutex M0002 in main thread:
#0 _pthread_mutex_lock /usr/src/contrib/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1342:3
#1 _flockfile /usr/src/lib/libc/stdio/_flock_stub.c:72:3
#2 print_active lib/isc/mem.c:629:3
#3 isc_mem_stats lib/isc/mem.c:694:2
#4 main bin/named/main.c:1498:4
Mutex M0002 previously acquired by the same thread here:
#0 _pthread_mutex_lock /usr/src/contrib/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1342:3
#1 isc_mem_stats lib/isc/mem.c:668:2
#2 main bin/named/main.c:1498:4
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) lib/isc/mem.c:210:2 in add_trace_entry
In the first stack frame ('M0001'->'M0002' lock order) cfg_gettoken()
uses flockfile() to lock 'M0001' for the 'FILE' object associated with
the configuration file (e.g. the configuration file itself and
whatever it includes, like a zone database), then it locks a memory
context mutex M0002.
In the other stack frmae ('M0002'->'M0001' lock order) isc_mem_stats()
locks a memory context mutex M0002, then it uses fprintf(), which
internally locks a 'M0001' mutex with flockfile() to write into the
'named.memstats' memory statistics file.
While it is true that in one case a lock in the 'isc_mem_t' structure is
locked first, and then a lock in the 'FILE' structure is locked second,
and in the the second case it is the other way around, this isn't an
issue, because those are 'FILE' structures for totally different files,
used in different parts of the code.
It was also manually confirmed that 'named.memstats' doesn't get
processed by cfg_gettoken(), and is used only in the second stack
frame's code flow when named is exiting.
(cherry picked from commit 33dff63cbb)
For pipelines in the private repository, add an optional manual job,
which allows the current branch to be pushed into the specified
customer's git repository. This can be useful to provide patch previews
for early testing.
For tags created in a private repository, add a manual job which pushes
the created tag to all entitled customers.
Backport of MR !10323
Merge branch 'backport-nicki/ci-customer-git-automation-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10356
For pipelines in the private repository, add an optional manual job,
which allows the current branch to be pushed into the specified
customer's git repository. This can be useful to provide patch previews
for early testing.
For tags created in a private repository, add a manual job which pushes
the created tag to all entitled customers.
(cherry picked from commit 378b412e94)
TSAN reports a data race in FreeBSD's memset(), called by its
__crt_calloc() memory allocation function. There is a very similar
bug report [1] in FreeBSD bug tracker, and an existing code-review [2]
that tries to address an issue, the description of which is very
similar to what we are seeing.
Suppress this report by adding its signature to '.tsan-suppress'.
[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=282794
[2] https://reviews.freebsd.org/D28536?id=86694Closes#5267
Backport of MR !10353
Merge branch 'backport-5267-freebsd-suppress-tsan-data-race-false-positive-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10354
TSAN reports a data race in FreeBSD's memset(), called by its
__crt_calloc() memory allocation function. There is a very similar
bug report [1] in FreeBSD bug tracker, and an existing code-review [2]
that tries to address an issue, the description of which is very
similar to what we are seeing.
Suppress this report by adding its signature to '.tsan-suppress'.
[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=282794
[2] https://reviews.freebsd.org/D28536?id=86694
(cherry picked from commit 40a91f51d7)
Previously the code would not check if the string intended to be used
for SNI is a hostname.
See also: !9923Closes#5225
Backport of MR !10254
Merge branch 'backport-5225-dig-sni-fix-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10347
Add a function that checks if a 'hostname' is not a valid IPv4 or IPv6
address. Returns 'true' if the hostname is likely a domain name, and
'false' if it represents an IP address.
(cherry picked from commit 1f199ee606)
Previously changed mem_test (!10320) introduces a test which checks for
the value of `__FILE__`, which is different if the build is done
out-of-tree or not, even though this is not relevant for the test (only
the base filename is). This result in a broken test for out-of-tree
builds. Fix this by changing the way the "grep" is done in the test,
ignoring the optional path prefix in the filename.
Backport of MR !10343
Merge branch 'backport-colin-fix-outoftree-memtest-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10344
Previously changed mem_test (!10320) introduces a test which checks for
the value of `__FILE__`, which is different if the build is done
out-of-tree or not, even though this is not relevant for the test (only
the base filename is). This result in a broken test for out-of-tree
builds. Fix this by changing the way the "grep" is done in the test,
ignoring the optional path prefix in the filename.
(cherry picked from commit b7ae514b3b)
When allocating memory under -m trace|record, the __FILE__ pointer is
stored, so it can be printed out later in order to figure out in which
file an allocation leaked. (among others, like the line number).
However named crashes when called with -m record and using a plugin
leaking memory. The reason is that plugins are unloaded earlier than
when the leaked allocations are dumped (obviously, as it's done as late
as possible). In such circumstances, `__FILE__` is dangling because the
dynamically loaded library (the plugin) is not in memory anymore.
Fix the crash by systematically copying the `__FILE__` string
instead of copying the pointer. Of course, this make each allocation to
consume a bit more memory (and longer, as it needs to calculate the
length of `__FILE__`) but this occurs only under -m trace|record debugging
flags.
Backport of MR !10320
Merge branch 'backport-colin-memdump-plugins-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10336
When allocating memory under -m trace|record, the __FILE__ pointer is
stored, so it can be printed out later in order to figure out in which
file an allocation leaked. (among others, like the line number).
However named crashes when called with -m record and using a plugin
leaking memory. The reason is that plugins are unloaded earlier than
when the leaked allocations are dumped (obviously, as it's done as late
as possible). In such circumstances, __FILE__ is dangling because the
dynamically loaded library (the plugin) is not in memory anymore.
Fix the crash by systematically copying the __FILE__ string
instead of copying the pointer. Of course, this make each allocation to
consume a bit more memory (and longer, as it needs to calculate the
length of __FILE__) but this occurs only under -m trace|record debugging
flags.
In term of unit test, because grepping in C is not fun, and because the
whole "syntax" of the dump output is tested in other tests, this simply
search for a substring in the whole buffer to make sure the expected
allocations are found.
(cherry picked from commit 4eb2cd364a)
The new ``reset-stats`` command for ``rndc`` allows some statistics
counters to be reset during runtime. At the moment only two "high-water"
counters are supported, so the ability to reset them after the
initial peaks during the server's "warm-up" phase may be useful for
some operators.
Closes#5251
Backport of MR !10318
Merge branch 'backport-5251-feature-rndc-reset-high-water-statistics-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10335
This new rndc option allows to reset some statistics counters during
runtime. At this moment only the high-water type counters are supported
as such an ability to reset them after the initial peaks during the
server's "warm-up" phase can be useful for some operators.
(cherry picked from commit bab20802b3)
Unit test for isc_netaddr_masktoprefixlen were missing IPv6 mask cases.
Add those and few other IPv4 cases. Also, the test is refactored in
order to make it easy to add new cases.
Backport of MR !10328
Merge branch 'backport-colin-netaddrmasktest-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10332
Unit test for isc_netaddr_masktoprefixlen were missing IPv6 mask cases.
Add those and few other IPv4 cases. Also, the test is refactored in
order to make it easy to add new cases.
(cherry picked from commit ad2c3fa4f7)
The string literal initialalising compressed was too big for the
array as it has an unwanted NUL terminator. This is allowed for
in C for historical reasons but produces a warning with some
compilers. Adjust the declaration to include the NUL and adjust
the users to pass in an adjusted size which excludes the NUL rather
than sizeof(compressed).
Closes#5258
Backport of MR !10326
Merge branch 'backport-5258-avoid-warning-initialising-compresss-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10331
The string literal initialalising compressed was too big for the
array as it has an unwanted NUL terminator. This is allowed for
in C for historical reasons but produces a warning with some
compilers. Adjust the declaration to include the NUL and adjust
the users to pass in an adjusted size which excludes the NUL rather
than sizeof(compressed).
(cherry picked from commit 6a6b6be824)
Previously, in some cases, the resolver could return rdatasets of type CNAME or DNAME without the result code being set to `DNS_R_CNAME` or `DNS_R_DNAME`. This could trigger an assertion failure in the ADB. The resolver error has been fixed.
Closes#5201
Backport of MR !10172
Backport of MR !10178
Merge branch 'backport-5201-adb-cname-error-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10329
the fix in commit 1edbbc32b4 was incomplete; the wrong
event result could also be set in cache_name() and validated().
(cherry picked from commit 9ebeb60174)
when the caching of a negative record failed because of the
presence of a positive one, ncache_adderesult() could override
this to ISC_R_SUCCESS. this could cause CNAME and DNAME responses
to be handled incorrectly. ncache_adderesult() now sets the result
code correctly in such cases.
(cherry picked from commit 1edbbc32b4)
Otherwise curl 8.13 rejects the line with:
I:Check HTTP/1.1 keep-alive with truncated stream (21)
curl: option --header: error encountered when reading a file
curl: try 'curl --help' or 'curl --manual' for more information
Also, see https://github.com/curl/curl/pull/16572.
Closes#5249
Backport of MR !10319
Merge branch 'backport-5249-statschannel-limit-http-header-size-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10321
Otherwise curl 8.13 rejects the line with:
I:Check HTTP/1.1 keep-alive with truncated stream (21)
curl: option --header: error encountered when reading a file
curl: try 'curl --help' or 'curl --manual' for more information
Also, see https://github.com/curl/curl/pull/16572.
(cherry picked from commit bcb931f4a8)
Previously, when named-checkzone loaded the zone to the QP database, the delayed
memory reclamation could cause an assertion check on exit. This has
been fixed.
Closes#5220
Merge branch '5220-add-rcu-barrier-to-named-checkconf-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!10307
