2
0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-30 22:15:20 +00:00
Commit Graph

39668 Commits

Author SHA1 Message Date
Tom Krizek
21980b43b8 Compile system test binaries during make
Using check_PROGRAMS would postpone compiling the binaries needed by
system tests until `make check` would be called. Since it's preferable
to invoke pytest directly to run the system test suite, compile these
binaries without installing them during `make all` instead by using
noinst_PROGRAMS.

This removes the need to use TESTS= make -e check hack invoked from
pytest to work around this issue.
2023-08-21 20:04:48 +02:00
Ondřej Surý
664d7c642d Merge branch '4041-add-userspace-tracing' into 'main'
Add support for userspace tracing via USDT

Closes #4041

See merge request isc-projects/bind9!7872
2023-08-21 18:04:05 +00:00
Ondřej Surý
52cabc662f Add CHANGES and release note for [GL #4041] 2023-08-21 18:39:53 +02:00
Ondřej Surý
9b858769dc Add zone name to the LIBDNS_XFRIN probes
We already print the formatted zone name via the xfrin_log() function,
generate the text once and store it in xfr->info static buffer.  Then we
can reuse the string to pass it to the LIBDNS_XFRIN probes.
2023-08-21 18:39:53 +02:00
Ondřej Surý
d332f07f38 Add a probe when the response rate limiting drops or slips query
Add a trace point that would report when a query gets dropped or slipped
by rate limits. It reports the client IP, the zone, and the RRL result
code.

Co-authored-by: Paul Frieden <pfrieden@yahooinc.com>
2023-08-21 18:39:53 +02:00
Ondřej Surý
96ccba5d51 Add tracing probes to the dns_xfrin unit
Add tracing probes to incoming transfers, so we can accurately measure
the individual events when sending, receiving and parsing the incoming
transfers.
2023-08-21 18:39:53 +02:00
Ondřej Surý
2484a3702a Add tracing probes to the isc_job unit
Add tracing probes to isc_job unit:

 * libisc:job_cb_before - before the job callback is called
 * libisc:job_cb_after - after the job callback is called
2023-08-21 18:39:53 +02:00
Ondřej Surý
dcd60215ac Add tracing probes to the custom isc_rwlock implementation
Add tracing probes to ISC own isc_rwlock implementation to allow
fine-grained tracing.  The pthread rwlock already has probes inside
glibc, and it's difficult to add probes to headers included from the
other libraries.
2023-08-21 18:39:53 +02:00
Ondřej Surý
784d055809 Add support for User Statically Defined Tracing (USDT) probes
This adds support for User Statically Defined Tracing (USDT).  On
Linux, this uses the header from SystemTap and dtrace utility, but the
support is universal as long as dtrace is available.

Also add the required infrastructure to add probes to libisc, libdns and
libns libraries, where most of the probes will be.
2023-08-21 18:39:53 +02:00
Michal Nowak
1e16d41446 Merge branch '3893-make-debian-12-bookworm-base-image' into 'main'
Make Debian 12 "bookworm" the base image

See merge request isc-projects/bind9!8075
2023-08-21 15:19:49 +00:00
Michal Nowak
077d82403e Move clang:bullseye PKCS11 testing to gcc:bullseye
Move clang:bullseye:amd64 PKCS11 testing to the gcc:bullseye:amd64 job
to evenly represent compilers in PKCS11 testing.
2023-08-21 16:35:52 +02:00
Michal Nowak
55f5aa08f0 Drop unnecessary gcovr workarounds
Many problems of the Debian 11 gcovr version were fixed in the Debian 12
one. Replace workarounds we accumulated over the years with two new,
simple ones.
2023-08-21 16:35:52 +02:00
Michal Nowak
cfb06dabfe Drop PKCS11 testing from "sid", restore it on "clang:bookworm"
The gcc:sid:amd64 job was used to test PKCS11 via the OpenSSL engine API
but is now designated for future PKCS11 testing via the provider API.

The ability to use PKCS11 via engine API in clang:bookworm:amd64 has
been restored using only deprecated APIs in OpenSSL 3.
2023-08-21 16:35:52 +02:00
Michal Nowak
7929168295 Disable mandoc stylistic warning
We are not concerned by the "input text line longer than 80 bytes"
stylistic warning.
2023-08-21 16:35:52 +02:00
Michal Nowak
a330ab221e Drop unneeded -Wno-compound-token-split-by-macro option
Debian 12 has Perl 5.36 and the option is no longer needed.
2023-08-21 16:35:52 +02:00
Michal Nowak
5e1120d9a3 Reintroduce Debian 11 "bullseye" Clang and GCC jobs 2023-08-21 16:35:52 +02:00
Michal Nowak
2d18c57c54 Make Debian 12 "bookworm" the base image
Just replace "bullseye" with "bookworm" and reintroduce Debian 11
"bullseye" later.
2023-08-21 16:35:52 +02:00
Ondřej Surý
a5884c2c1d Merge branch '4265-remove-caching-of-netmgr-sockets-and-uvreqs' into 'main'
Limit the number of inactive handles and uvreqs kept for reuse

Closes #4265

See merge request isc-projects/bind9!8206
2023-08-21 14:35:37 +00:00
Ondřej Surý
db7c501bd3 Add CHANGES note for [GL #4265] 2023-08-21 16:34:30 +02:00
Ondřej Surý
0c9cf8fabb Limit the memory pool for the uvreqs
Set the number of maximum free items for the uvreq memory pool to 64.
2023-08-21 16:34:30 +02:00
Ondřej Surý
f36e118b9a Limit the number of inactive handles kept for reuse
Instead of growing and never shrinking the list of the inactive
handles (to be reused mostly on the UDP connections), limit the number
of maximum number of inactive handles kept to 64.  Instead of caching
the inactive handles for all listening sockets, enable the caching on on
UDP listening sockets.  For TCP, the handles were cached for each
accepted socket thus reusing the handles only for long-standing TCP
connections, but not reusing the handles across different TCP streams.
2023-08-21 16:34:30 +02:00
Tom Krizek
1a958b709a Merge branch '4252-pytest-symlink-to-test-artifacts' into 'main'
Create symlinks to test artifacts for pytest runner

Closes #4252

See merge request isc-projects/bind9!8194
2023-08-21 13:55:07 +00:00
Tom Krizek
355dc73391 .gitignore temporary directories and symlinks in system test dir 2023-08-21 15:54:40 +02:00
Tom Krizek
d66ff81543 Add clean-local target to clean pytest runner artifacts
The command finds all directories in bin/tests/system which contain an
underscore. Underscore indicates either a temporary directory (_tmp_), a
symlink to test artifacts (TESTNAME_MODULENAME), or a python-related
cache. Using underscore for a system test name is invalid and a hyphen
must be used instead.
2023-08-21 15:54:40 +02:00
Tom Krizek
83ddca7690 Silence pylint's refactoring suggestions for system_test_dir()
While it'd be fairly easy to split the function up into smaller ones,
the readability wouldn't be improved in this case. Silence the
suggestions instead.
2023-08-21 15:54:40 +02:00
Tom Krizek
f91d0b13e8 Improve tempdir logging for pytest runner
At the end of the test, display the symlink path to the artifact
directory in case it's preserved. Log the full tempdir name in debug
log.
2023-08-21 15:54:40 +02:00
Tom Krizek
e1ca5c8071 Create symlinks to test artifacts for pytest runner
While temporary directories are useful for test execution to keep
everything clean, they are difficult to work with manually. Create a
symlink for each test artifact directory with a stable and predictable
path. The symlink always either points to the latest artifacts, or is
missing in case the last run succeeded.

Ensure these symlinked directories aren't detected as test suites by the
pytest runner.
2023-08-21 15:54:38 +02:00
Ondřej Surý
838dc9b053 Merge branch '4125-release-notes' into 'main'
Add release note for [GL #4125]

Closes #4125

See merge request isc-projects/bind9!8216
2023-08-21 13:19:00 +00:00
Ondřej Surý
58a8de5246 Add release note for [GL #4125] 2023-08-21 15:16:41 +02:00
Tom Krizek
f52dd73cac Merge branch '4246-remove-legacy-test-runner-from-ci' into 'main'
Use pytest runner for out-of-tree and OpenBSD system tests in CI

Closes #4246

See merge request isc-projects/bind9!8193
2023-08-21 13:13:33 +00:00
Tom Krizek
b1a9d1fe25 Improve pytest message when sanitizer report is found 2023-08-21 15:12:53 +02:00
Tom Krizek
bc66bf146f ci: remove legacy system test runner template 2023-08-21 15:12:53 +02:00
Tom Krizek
d49d9ec80e ci: run out-of-tree system tests with pytest runner
Out-of-tree builds are built in a directory that is different from
source directory. The build directory doesn't contain the non-compiled
test files from bin/tests/system which are the test cases required by
the pytest runner.

In order to run the system tests for out-of-tree build, copy over the
contents (tests) of bin/tests/system/ from the source directory into the
build directory. Then, it is possible to invoke the pytest runner inside
the build directory.
2023-08-21 15:12:53 +02:00
Tom Krizek
8846877dfe ci: use pytest system test runner on OpenBSD
A supported pytest version finally made it to OpenBSD repositories,
allowing us to run system tests with the pytest runner.
2023-08-21 15:12:51 +02:00
Ondřej Surý
f9657267af Merge branch '4125-statschannel-content-length-overflow' into 'main'
Parse statschannel Content-Length: more carefully

Closes #4125

See merge request isc-projects/bind9!8018
2023-08-21 13:01:28 +00:00
Tony Finch
1ba538f32f Add CHANGES entry for [GL #4125] 2023-08-21 14:14:18 +02:00
Tony Finch
26e10e8fb5 Parse statschannel Content-Length: more carefully
A negative or excessively large Content-Length could cause a crash
by making `INSIST(httpd->consume != 0)` fail.
2023-08-21 14:14:18 +02:00
Michał Kępień
159c880240 Merge branch 'michal/post-release-tweaks' into 'main'
Post-release tweaks (cross-version testing cleanup, release checklist tweaks)

See merge request isc-projects/bind9!8210
2023-08-17 15:31:51 +00:00
Michał Kępień
45bf6123ef Update release checklist
Update the release checklist with links to recently implemented tools
for automating certain tasks.
2023-08-17 17:30:37 +02:00
Michał Kępień
96e4139fb8 Revert "Exclude dupsigs and keymgr2kasp from cross-version-config-tests"
This reverts commit 4eac32fe23.

With the v9.19.16 release tag merged, the "cross-version-config-tests"
GitLab CI job will no longer fail due to the two relevant system tests
being absent from the development branch.  This makes the pytest
filtering expression added to work around that issue unnecessary, so
remove it.
2023-08-17 17:30:37 +02:00
Michal Nowak
4970d0ca38 Merge branch '4139-test_send_timeout-check-can-be-unstable' into 'main'
Make the test_send_timeout check more stable

Closes #4139

See merge request isc-projects/bind9!8168
2023-08-17 13:11:04 +00:00
Michal Nowak
681b23c398 Mark test_send_timeout as flaky
In some cases, BIND is not fast enough to fill the send buffer and
manages to answer all queries, contrary to what the test expects.
Repeat the check up to 3 times to limit this test instability.
2023-08-17 14:13:59 +02:00
Tom Krizek
5b703de733 Add custom flaky decorator to handle unstable tests
If the flaky plugin for pytest is available, use its decorator to
support re-running unstable tests. In case the package is missing,
execute the test as usual without attempts to re-run it in case of
failure.

This is mostly intended to increase the test stability in CI. Using a
custom decorator enables us to keep the flaky package as an optional
dependency.
2023-08-17 14:13:59 +02:00
Michał Kępień
e27a33e621 Merge tag 'v9.19.16' 2023-08-17 14:10:53 +02:00
Michal Nowak
b88f914c79 Merge branch 'mnowak/clean-leftover-files' into 'main'
Clean leftover files in autosign and masterformat

See merge request isc-projects/bind9!8167
2023-08-16 12:45:24 +00:00
Michal Nowak
8a0da13479 Clean leftover files in autosign and masterformat
The following files were reported in CI by the legacy system test runner
and prevented job to pass. They should be removed.

    $ if git rev-parse > /dev/null 2>&1; then ( ! grep "^I:.*:file.*not removed$" *.log ); fi
    autosign.log:I:autosign:file autosign/ns3/kskonly.example.db.jbk not removed
    autosign.log:I:autosign:file autosign/ns3/optout.example.db.jbk not removed
    autosign.log:I:autosign:file autosign/ns3/reconf.example.db.jbk not removed
    masterformat.log:I:masterformat:file masterformat/ns1/signed.db.raw.jbk not removed
    masterformat.log:I:masterformat:file masterformat/ns1/signed.db.raw.signed not removed
    masterformat.log:I:masterformat:file masterformat/ns1/signed.db.raw.signed.jnl not removed

Don't print an error when the ns*/inactive directory is not
present:

    rmdir: ns*/inactive: No such file or directory

Remove nsupdate.out.test file instead of nsupdate.out, as the latter
does not exist.
2023-08-16 12:01:11 +02:00
Ondřej Surý
2c51e936e4 Merge branch '4228-fix-heap-use-after-free-in-dns_dispatch_createtcp' into 'main'
Attach to the dns_dispatchmgr in the dns_view object

Closes #4228

See merge request isc-projects/bind9!8203
2023-08-16 07:22:23 +00:00
Ondřej Surý
ce1b1b4058 Add CHANGES note for [GL #4228] 2023-08-15 10:26:02 -07:00
Ondřej Surý
d76ab69772 Attach to the dns_dispatchmgr in the dns_view object
The dns_dispatchmgr object was only set in the dns_view object making it
prone to use-after-free in the dns_xfrin unit when shutting down named.

Remove dns_view_setdispatchmgr() and optionally pass the dispatchmgr
directly to dns_view_create() when it is attached and not just assigned,
so the dns_dispatchmgr doesn't cease to exist too early.

The dns_view_getdnsdispatchmgr() is now protected by the RCU lock, the
dispatchmgr reference is incremented, so the caller needs to detach from
it, and the function can return NULL in case the dns_view has been
already shut down.
2023-08-15 10:25:37 -07:00
Evan Hunt
cfad194e1d Merge branch 'each-qp-nta' into 'main'
Use a QP trie for the NTA table, forwarders and key tables

See merge request isc-projects/bind9!7811
2023-08-15 17:16:21 +00:00