While using mutrace, the phtread-rwlock based isc_rwlock implementation
would be all tracked in the rwlock.c unit losing all useful information
as all rwlocks would be traced in a single place. Rewrite the
pthread_rwlock based implementation to be header-only macros, so we can
use mutrace to properly track the rwlock contention without heavily
patching mutrace to understand the libisc synchronization primitives.
The dns_rbtdb unit already tracks the state of the node and tree rwlocks
during the top level function and passes the states of the locks to the
called functions.
Add the tree locking family of macros modeled after node locking macros,
and expand both to track the state of the lock in an external variable.
Additionally, in developer mode, add precondition to the macros, so the
lock is in required state - this should cause an assertion failure on
double locking instead of the thread getting stuck.
On Linux, the libcap is now mandatory. It makes things simpler for us.
System without {set,get}res{uid,gid} now have compatibility shim using
setreuid/setregid or seteuid/setegid to setup effective UID/GID, so the
same code can be called all the time (including on Linux).
The NetBSD system allocator is in fact based on the jemalloc, but it
doesn't export the extended interface, so we can't use that. Remove
the jemalloc enforcement for the NetBSD.
There's a known memory leak in the engine_pkcs11 at the time of writing
this and it interferes with the named ability to check for memory leaks
in the OpenSSL memory context by default.
Add an autoconf option to explicitly enable the memory leak detection,
and use it in the CI except for pkcs11 enabled builds. When this gets
fixed in the engine_pkc11, the option can be enabled by default.
Instead of using generic HAVE_BUILTIN_OVERFLOW, we need to check whether
the overflow functions actually work as there was a bug in GCC that it
would not detect mul overflow when compiled with `-m32` option without
optimizations and the bug was fixed only for GCC 6.5+ and 7.3+/8+.
For further details see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82274
By bumping the minimum libuv version to 1.34.0, it allows us to remove
all libuv shims we ever had and makes the code much cleaner. The
up-to-date libuv is available in all distributions supported by BIND
9.19+ either natively or as a backport.
previously, when ISC_BUFFER_USEINLINE was defined, macros were
used to implement isc_buffer primitives (isc_buffer_init(),
isc_buffer_region(), etc). these macros were missing the DbC
assertions for those primitives, which made it possible for
coding errors to go undetected.
adding the assertions to the macros caused compiler warnings on
some platforms. therefore, this commit converts the ISC__BUFFER
macros to static inline functions instead, with assertions included,
and eliminates the non-inline implementation from buffer.c.
the --enable-buffer-useinline configure option has been removed.
When jemalloc is the system allocator (on FreeBSD and NetBSD), trying
to build --without-jemalloc caused an obscure compiler error. Instead,
complain at configure time that --without-jemalloc cannot work. (It
needs to remain an error because it is vexing when configure quietly
ignores an explicit direction.)
When Address Sanitizer is enabled in gcc-11+, number of false positives
might appear like this:
netmgr/udp.c: In function 'isc__nm_udp_send':
netmgr/udp.c:729:13: warning: 'uv_udp_send' reading 16 bytes from a region of size 8 [-Wstringop-overread]
729 | r = uv_udp_send(&uvreq->uv_req.udp_send, &sock->uv_handle.udp,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
730 | &uvreq->uvbuf, 1, sa, udp_send_cb);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
netmgr/udp.c:729:13: note: referencing argument 3 of type 'const uv_buf_t[0]'
In file included from ./include/isc/uv.h:17,
from ./include/isc/barrier.h:31,
from netmgr/udp.c:17:
/usr/include/uv.h:711:15: note: in a call to function 'uv_udp_send'
711 | UV_EXTERN int uv_udp_send(uv_udp_send_t* req,
| ^~~~~~~~~~~
Disable the warning globally in the autoconf, instead of just locally in
a single CI job, as it might affect people outside our GitLab CI.
sd_notify() may be called by a service to notify the service manager
about state changes. It can be used to send arbitrary information,
encoded in an environment-block-like string. Most importantly, it can be
used for start-up completion notification.
Add libsystemd check to autoconf script and when the library is detected
add calls to sd_notify() around the server->reload_status changes.
Co-authored-by: Petr Špaček <pspacek@isc.org>
Move the libtest code into a 'libtest' subdirectory and make it
one of the SUBDIRS in the tests Makefile. having it at the top level
required having "." as one of the subdirs, and that caused the
unit tests to be executed twice.
The unit tests are now using a common base, which means that
lib/dns/tests/ code now has to include lib/isc/include/isc/test.h and
link with lib/isc/test.c and lib/ns/tests has to include both libisc and
libdns parts.
Instead of cross-linking code between the directories, move the
/lib/<foo>/test.c to /tests/<foo>.c and /lib/<foo>/include/<foo>test.h
to /tests/include/tests/<foo>.h and create a single libtest.la
convenience library in /tests/.
At the same time, move the /lib/<foo>/tests/ to /tests/<foo>/ (but keep
it symlinked to the old location) and adjust paths accordingly. In few
places, we are now using absolute paths instead of relative paths,
because the directory level has changed. By moving the directories
under the /tests/ directory, the test-related code is kept in a single
place and we can avoid referencing files between libns->libdns->libisc
which is unhealthy because they live in a separate Makefile-space.
In the future, the /bin/tests/ should be merged to /tests/ and symlink
kept, and the /fuzz/ directory moved to /tests/fuzz/.
When autoconf was checking for libuv features, the LIBUV_CFLAGS was not
added to CFLAGS and LIBUV_LIBS to LIBS which resulted in false
negatives.
Use AX_SAVE_FLAGS and AX_RESTORE_FLAGS to temporarily add LIBUV_CFLAGS
and LIBUV_LIBS to their respective variables.
From the ld man page:
When creating a dynamically linked executable, using the -E option or
the --export-dynamic option causes the linker to add all symbols to
the dynamic symbol table. The dynamic symbol table is the set of
symbols which are visible from dynamic objects at run time.
This should allow the backtrace(3) to fully resolve the symbols when
creating backtrace on an assertion failure.
The connect()ed UDP socket provides feedback on a variety of ICMP
errors (eg port unreachable) which bind can then use to decide what to
do with errors (report them to the client, try again with a different
nameserver etc). However, Linux's implementation does not report what
it considers "transient" conditions, which is defined as Destination
host Unreachable, Destination network unreachable, Source Route Failed
and Message Too Big.
Explicitly enable IP_RECVERR / IPV6_RECVERR (via libuv uv_udp_bind()
flag) to learn about ICMP destination network/host unreachable.
configure.ac currently requires Python 3.4 for running Python-based
system tests. Meanwhile, there are some features in Python 3.6+ that we
would like to use for making our Python code cleaner (e.g. f-strings).
Update the minimum Python version required for running Python-based
system tests to 3.6, noting that:
- Python 3.4 has reached end-of-life on March 18th, 2019.
- Python 3.5 has reached end-of-life on September 5th, 2020.
The implementation is done on top of the reference counting
functionality found in OpenSSL/LibreSSL, which allows for avoiding
wrapping the object.
Adding this function allows using reference counting for TLS contexts
in BIND 9's codebase.
C11 has builtin support for _Noreturn function specifier with
convenience noreturn macro defined in <stdnoreturn.h> header.
Replace ISC_NORETURN macro by C11 noreturn with fallback to
__attribute__((noreturn)) if the C11 support is not complete.
From an attacker's point of view, a VLA declaration is essentially a
primitive for performing arbitrary arithmetic on the stack pointer. If
the attacker can control the size of a VLA they have a very powerful
tool for causing memory corruption.
To mitigate this kind of attack, and the more general class of stack
clash vulnerabilities, C compilers insert extra code when allocating a
VLA to probe the growing stack one page at a time. If these probes hit
the stack guard page, the program will crash.
From the point of view of a C programmer, there are a few things to
consider about VLAs:
* If it is important to handle allocation failures in a controlled
manner, don't use VLAs. You can use VLAs if it is OK for
unreasonable inputs to cause an uncontrolled crash.
* If the VLA is known to be smaller than some known fixed size,
use a fixed size array and a run-time check to ensure it is large
enough. This will be more efficient than the compiler's stack
probes that need to cope with arbitrary-size VLAs.
* If the VLA might be large, allocate it on the heap. The heap
allocator can allocate multiple pages in one shot, whereas the
stack clash probes work one page at a time.
Most of the existing uses of VLAs in BIND are in test code where they
are benign, but there was one instance in `named`, in the GSS-TSIG
verification code, which has now been removed.
This commit adjusts the style guide and the C compiler flags to allow
VLAs in test code but not elsewhere.
The AX_PROG_CC_FOR_BUILD implementation to find a native CC compiler is
slightly better because it uses AC_PROG_CC and AC_PROG_CPP to find the
native compiler instead of just defaulting to `gcc` as AX_CC_FOR_BUILD
does.
AX_PROG_CC_FOR_BUILD also sets BUILD_EXEEXT that we already use in the
Makefile.am for `lib/dns/gen` while AX_CC_FOR_BUILD uses
EXEEXT_FOR_BUILD.
BIND unconditionally uses shims for BN_GENCB_new(), BN_GENCB_free(),
and BN_GENCB_get_arg() for all LibreSSL versions and, correctly, for
OpenSSL <1.1.0 versions.
This breaks LibreSSL compilation starting with LibreSSL 3.5.0.
Use autoconf check instead to check whether the family of the functions
are available.
After the build system refactoring, we no longer call AM_PROG_CC_C_O
because it is obsolescent macro. According to the automake manual the
`AC_PROG_CC` has been rewritten in automake 1.14 to not required the
call, thus we need to require at least automake version 1.14.
In autoconf, the AC_INIT() accepts bugreport address for reporting
issues (f.e. when the test suite fails). Instead of providing generic
emails address, change this to the address where to report with the
default Bug template applied.
There are reported occurences where the statitic counters underflows and
starts reporting non-sense.
Add a check for the underflow, when ``named`` is compiled in the
developer mode.
The isc_thread_setaffinity call was removed in !5265 and we are not
going to restore it because it was proven that the performance is better
without it. Additionally, remove the already disabled cpu system test.
The isc_thread_setconcurrency function is unused and also calling
pthread_setconcurrency() on Linux has no meaning, formerly it was
added because of Solaris in 2001 and it was removed when taskmgr was
refactored to run on top of netmgr in !4918.
For reproducible builds, we use last modification time of the CHANGES
file. This works pretty well, unless the builds are made in different
timezones.
Use UTC option to date command to make the builds reproducible.
Extend the error message displayed when support for DNS over HTTPS is
requested but libnghttp2 is unavailable at build time, in order to help
the user find a way out of such a situation.
gnutls-cli is tricky to script around as it immediately closes the
server connection when its standard input is closed. This prevents
simple shell-based I/O redirection from being used for capturing the DNS
response sent over a TLS connection and the workarounds for this issue
employ non-standard utilities like "timeout".
Instead of resorting to clever shell hacks, reimplement the relevant
check in Python. Exit immediately upon receiving a valid DNS response
or when gnutls-cli exits in order to decrease the test's run time.
Employ dnspython to avoid the need for storing DNS queries in binary
files and to improve test readability. Capture more diagnostic output
to facilitate troubleshooting. Use a pytest fixture instead of an
Autoconf macro to keep test requirements localized.
The detection of MUSL libc via autoconf $host turned out to be
not reliable.
Convert the autoconf check from $host detection to actually detect
the padding used in the struct msghdr.
The Linux kernel diverts from the POSIX specification for two members of
struct msghdr making them size_t sized (instead of int and socklen_t).
In glibc, the developers have decided to use that. However, the MUSL
developers used padding for the struct and kept the members defined
according to the POSIX.
This creates a problem, because libuv doesn't use recvmmsg() library
call where the padding members are correctly zeroed and instead calls
the syscall directly, the struct msghdr is passed to the kernel with
enormous values in those two members (because of the random junk in the
padding members) and the syscall thus fail with EMSGSIZE.
Disable udp recvmmsg support on systems with MUSL libc until the libuv
starts zeroing the struct msghdr before passing it to the syscall.
Previously, the netmgr/udp.c tried to detect the recvmmsg detection in
libuv with #ifdef UV_UDP_<foo> preprocessor macros. However, because
the UV_UDP_<foo> are not preprocessor macros, but enum members, the
detection didn't work. Because the detection didn't work, the code
didn't have access to the information when we received the final chunk
of the recvmmsg and tried to free the uvbuf every time. Fortunately,
the isc__nm_free_uvbuf() had a kludge that detected attempt to free in
the middle of the receive buffer, so the code worked.
However, libuv 1.37.0 changed the way the recvmmsg was enabled from
implicit to explicit, and we checked for yet another enum member
presence with preprocessor macro, so in fact libuv recvmmsg support was
never enabled with libuv >= 1.37.0.
This commit changes to the preprocessor macros to autoconf checks for
declaration, so the detection now works again. On top of that, it's now
possible to cleanup the alloc_cb and free_uvbuf functions because now,
the information whether we can or cannot free the buffer is available to
us.
This commit converts the license handling to adhere to the REUSE
specification. It specifically:
1. Adds used licnses to LICENSES/ directory
2. Add "isc" template for adding the copyright boilerplate
3. Changes all source files to include copyright and SPDX license
header, this includes all the C sources, documentation, zone files,
configuration files. There are notes in the doc/dev/copyrights file
on how to add correct headers to the new files.
4. Handle the rest that can't be modified via .reuse/dep5 file. The
binary (or otherwise unmodifiable) files could have license places
next to them in <foo>.license file, but this would lead to cluttered
repository and most of the files handled in the .reuse/dep5 file are
system test files.
