- isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection.
- isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a
timeout argument to ensure connections time out and are correctly
cleaned up on failure.
- isc_nm_read() now supports UDP; it reads a single datagram and then
stops until the next time it's called.
- isc_nm_cancelread() now runs asynchronously to prevent assertion
failure if reading is interrupted by a non-network thread (e.g.
a timeout).
- isc_nm_cancelread() can now apply to UDP sockets.
- added shim code to support UDP connection in versions of libuv
prior to 1.27, when uv_udp_connect() was added
all these functions will be used to support outgoing queries in dig,
xfrin, dispatch, etc.
While libltdl is a feature-rich library, BIND 9 code only uses its basic
capabilities, which are also provided by libuv and which BIND 9 already
uses for other purposes. As libuv's cross-platform shared library
handling interface is modeled after the POSIX dlopen() interface,
converting code using the latter to the former is simple. Replace
libltdl function calls with their libuv counterparts, refactoring the
code as necessary. Remove all use of libltdl from the BIND 9 source
tree.
Since Mac OS X 10.1, Mach-O object files are by default built with a
so-called two-level namespace which prevents symbol lookups in BIND unit
tests that attempt to override the implementations of certain library
functions from working as intended. This feature can be disabled by
passing the "-flat_namespace" flag to the linker. Fix unit tests
affected by this issue on macOS by adding "-flat_namespace" to LDFLAGS
used for building all object files on that operating system (it is not
enough to only set that flag for the unit test executables).
As currently used in the BIND source tree, the --wrap linker option is
redundant because:
- static builds are no longer supported,
- there is no need to wrap around existing functions - what is
actually required (at least for now) is to replace them altogether
in unit tests,
- only functions exposed by shared libraries linked into unit test
binaries are currently being replaced.
Given the above, providing the alternative implementations of functions
to be overridden in lib/ns/tests/nstest.c is a much simpler alternative
to using the --wrap linker option. Drop the code detecting support for
the latter from configure.ac, simplify the relevant Makefile.am, and
remove lib/ns/tests/wrap.c, updating lib/ns/tests/nstest.c accordingly
(it is harmless for unit tests which are not calling the overridden
functions).
Pairwise testing is a test case generation technique based on the
observation that most faults are caused by interactions of at most two
factors. For BIND, its configure options can be thought of as such
factors.
Process BIND configure options into a model that is subsequently
processed by the PICT tool in order to find an effective test vector.
That test vector is then used for configuring and building BIND using
various combinations of configure options.
This switch is believed to be unnecessary. The possibility to use
gperftools CPU profiler was kept, one needs to set 'CFLAGS' and
'LDFLAGS' accordingly.
This commit updates and simplifies the checks for the readline support
in nslookup and nsupdate:
* Change the autoconf checks to pkg-config only, all supported
libraries have accompanying .pc files now.
* Add editline support in addition to libedit and GNU readline
* Add isc/readline.h shim header that defines dummy readline()
function when no readline library is available
The --enable-fuzzing option now allows third choice "ossfuzz" that just
adds $LIB_FUZZING_ENGINE to FUZZ_LDFLAGS to make the fuzzer builds
compatible with OSS-Fuzz project that has some special quirks (the
main() routine is provided in the static library the project provides).
Previously, we have disallowed static linking (for good reasons).
However, there are legitimate reasons where static linking might be
useful, and one of the reasons is the OSS-Fuzz project that doesn't have
the libraries used for build, so static linking is the sane option here.
The static linking is still disallowed in the "production" builds, but
it's not possible to disable shared and enable static libraries when
used together with --enable-developer.
The fuzzing tests were temporarily disabled when the build system has been
converted to automake. This commit restores the functionality to run the
fuzzing tests as part of the `make check`. When the afl or libfuzzer
is enabled via ./configure, it uses a custom LOG_DRIVER (fuzz/<fuzzer.sh>).
Currently only libfuzzer.sh has been implemented that runs each fuzz
test for 5 seconds each.
Creation of EVP_MD_CTX and EVP_PKEY is quite expensive, so until we fix the code
to reuse the OpenSSL contexts and keys we'll use our own implementation of
siphash instead of trying to integrate with OpenSSL.
The LT_INIT() call in configure.ac is effectively a no-op because it is
preceded by a call to AC_PROG_LIBTOOL(), which is the previous name of
LT_INIT() used in older libtool versions. Replace AC_PROG_LIBTOOL()
with AC_PATH_PROG() to look for libtool in PATH without initializing it,
which is the originally intended behavior.
Without this change, --enable-static is used by default, which causes a
plain ./configure invocation to fail because static linking is now
disallowed. Drop --disable-static from the ./configure invocations used
in GitLab CI to test this scenario continuously.
Linking BIND 9 programs and libraries statically disables several
important features:
* dlopen() - relied on by dynamic loading of modules, dlz, and dyndb,
* RELRO (read-only relocations) and ASLR (address space layout
randomization) - security features which are important for any
program interacting with the network and/or user input.
Disable and disallow linking BIND 9 binaries statically, thus enforcing
dlopen() support and allowing use of RELRO and ASLR by default.
When maintainer mode is enabled (./configure --enable-maintainer-mode)
it enables rebuild of documentation source files that require extra
tools to be installed or compiled. For a convenience, those files are
already committed into the repository and their rebuild is not required
to build BIND 9 from sources.
The "srcid" file present in each BIND source tarball contains a
shortened hash of the Git commit corresponding to a given BIND release.
This allows a Git reference to be included in an archive that otherwise
lacks any Git information.
Before the move to Automake, if an "srcid" file was present in the root
source directory at the time ./configure was run, its contents were used
as the value of a compile-time constant which was then baked into BIND
binaries; otherwise, "git rev-parse" was used to determine the value of
that constant.
With Automake, a similar approach was attempted that required the
"srcid" file to be present at autoreconf time in order for it to be
used. However, note that this means that even if that file is present
in a source tarball created using "make dist", its contents are not
going to influence the value of the aforementioned compile-time constant
because autoreconf hardcodes the output of "git rev-parse" into the
configure script at autoreconf time.
To make things more clear, always use "git rev-parse" for determining
the value of the PACKAGE_SRCID compile-time constant when running
autoreconf. This causes "srcid" to be an empty string in source
tarballs built from other source tarballs, but that is not deemed to be
much of an issue as "make dist" is expected to be run from Git
repository clones. Remove stderr redirections to /dev/null to ensure
errors caused e.g. by running "make dist" from outside a Git repository
clone are not hidden. Trim the Git commit hash to 7 characters for
consistency between Unix and Windows systems.
Despite the above, ensure the "srcid" file is present in source tarballs
created using "make dist" as that file is used by the build process on
Windows.
Make various adjustments necessary to enable "make dist" to build a BIND
source tarball whose contents are complete enough to build binaries, run
unit & system tests, and generate documentation on Unix systems.
Known outstanding issues:
- "make distcheck" does not work yet.
- Tests do not work for out-of-tree source-tarball-based builds.
- Source tarballs are not complete enough for building on Windows.
All of the above will be addressed in due course.
when built with "configure --enable-singletrace", named will produce
detailed query logging at the highest debug level for any query with
query ID zero.
this enables monitoring of the progress of a single query by specifying
the QID using "dig +qid=0". the "client" logging category should be set
to a low severity level to suppress logging of other queries. (the
chance of another query using QID=0 at the same time is only 1 in 2^16.)
"--enable-singletrace" turns on "--enable-querytrace" as well, so if the
logging severity is not lowered, all other queries will be logged
verbosely as well. compiling with either of these options will impair
query performance; they should only be turned on when testing or
troubleshooting.
This adds a unit test driver for BIND with Automake. It runs the unit
test program provided as its sole command line argument and then looks
for a core dump generated by that test program. If one is found, the
driver prints the backtrace into the test log.
Some operating systems (e.g. CentOS, OpenBSD) install the main pytest
script as "py.test-3". Add that name to the list of names passed to
AC_PATH_PROGS() in order for pytest to be properly detected on a broader
range of operating systems.
The ARM and the manpages have been converted into Sphinx documentation
format.
Sphinx uses reStructuredText as its markup language, and many of its
strengths come from the power and straightforwardness of
reStructuredText and its parsing and translating suite, the Docutils.
The system tests currently uses patchwork of shell scripts which doesn't
offer proper error handling.
This commit introduced option to write new tests in pytest framework
that also allows easier manipulation of DNS traffic (using dnspython),
native XML and JSON manipulation and proper error reporting.
Add recursive "test" and "unit" rules, which execute "make check"
in specific directories - "make test" runs the system tests, and
"make unit" runs the unit tests.
There are several improvements over the default/previous behaviour of
the test log driver and log compiler:
* The system-test-driver.sh was dropped (it was used incorrectly)
* The run.sh script is now both log compiler and cli script to run
individual tests
* The custom-test-driver was added as extended version of the automake
test-driver with capability to tee the test output to stdout when
`--verbose yes` is passed to it (you can use LOG_DRIVER_FLAGS to
add the option by default)
* Makefile.am has been extended to honor V=1 for the system tests
test-driver (e.g. V=1 adds `--verbose yes` to AM_LOG_DRIVER_FLAGS)
fstrm_capture is not an essential utility, but its corresponding
Makefile token needs to substituted even if it is not found in PATH or
else the "dnstap" system test will consistently fail.
The rewrite of BIND 9 build system is a large work and cannot be reasonable
split into separate merge requests. Addition of the automake has a positive
effect on the readability and maintainability of the build system as it is more
declarative, it allows conditional and we are able to drop all of the custom
make code that BIND 9 developed over the years to overcome the deficiencies of
autoconf + custom Makefile.in files.
This squashed commit contains following changes:
- conversion (or rather fresh rewrite) of all Makefile.in files to Makefile.am
by using automake
- the libtool is now properly integrated with automake (the way we used it
was rather hackish as the only official way how to use libtool is via
automake
- the dynamic module loading was rewritten from a custom patchwork to libtool's
libltdl (which includes the patchwork to support module loading on different
systems internally)
- conversion of the unit test executor from kyua to automake parallel driver
- conversion of the system test executor from custom make/shell to automake
parallel driver
- The GSSAPI has been refactored, the custom SPNEGO on the basis that
all major KRB5/GSSAPI (mit-krb5, heimdal and Windows) implementations
support SPNEGO mechanism.
- The various defunct tests from bin/tests have been removed:
bin/tests/optional and bin/tests/pkcs11
- The text files generated from the MD files have been removed, the
MarkDown has been designed to be readable by both humans and computers
- The xsl header is now generated by a simple sed command instead of
perl helper
- The <irs/platform.h> header has been removed
- cleanups of configure.ac script to make it more simpler, addition of multiple
macros (there's still work to be done though)
- the tarball can now be prepared with `make dist`
- the system tests are partially able to run in oot build
Here's a list of unfinished work that needs to be completed in subsequent merge
requests:
- `make distcheck` doesn't yet work (because of system tests oot run is not yet
finished)
- documentation is not yet built, there's a different merge request with docbook
to sphinx-build rst conversion that needs to be rebased and adapted on top of
the automake
- msvc build is non functional yet and we need to decide whether we will just
cross-compile bind9 using mingw-w64 or fix the msvc build
- contributed dlz modules are not included neither in the autoconf nor automake
With the introduction of dnssec-policy, the aforementioned tools were
either rendered obsolete, or they will be replaced with dnssec-policy
based tools. Remove the tools and the requirement to have Python
installed. Python 3 is still being used for tests, so keep the autoconf
test, but make it much simpler.
Revert the change from ad03c22e976411cad743bc02746b803a2f119df7 as
further testing has shown that with hyper-threading disabled, named with
ISC rwlocks outperforms named with pthread rwlocks in cold cache testing
scenarios. Since building named with pthread rwlocks might still be a
better choice for some workloads, keep the compile-time option which
enables that.
The previous commit removed the code related to the internal symbol
table. On platforms where available, we can now use backtrace_symbols()
to print more verbose symbols table to the output.
As there's now general availability of backtrace() and
backtrace_symbols() functions (see below), the commit also removes the
usage of glibc internals and the custom stack tracing.
* backtrace(), backtrace_symbols(), and backtrace_symbols_fd() are
provided in glibc since version 2.1.
* backtrace(), backtrace_symbols(), and backtrace_symbols_fd() first
appeared in Mac OS X 10.5.
* The backtrace() library of functions first appeared in NetBSD 7.0 and
FreeBSD 10.0.
