2
0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-28 04:58:04 +00:00

208 Commits

Author SHA1 Message Date
Pavel Březina
6f21cf201b update docs: only one loopmgr is expected
Resolves: https://gitlab.isc.org/isc-projects/bind9/-/issues/5057
2024-12-05 10:32:11 +00:00
Nicki Křížek
f2cb2e5723 Remove invocations and mentions of clean.sh 2024-11-08 10:54:24 +01:00
Ondřej Surý
679e90a57d Add isc_log_createandusechannel() function to simplify usage
The new
isc_log_createandusechannel() function combines following calls:

    isc_log_createchannel()
    isc_log_usechannel()

calls into a single call that cannot fail and therefore can be used in
places where we know this cannot fail thus simplifying the error
handling.
2024-08-20 12:50:39 +00:00
Ondřej Surý
091d738c72 Convert all categories and modules into static lists
Remove the complicated mechanism that could be (in theory) used by
external libraries to register new categories and modules with
statically defined lists in <isc/log.h>.  This is similar to what we
have done for <isc/result.h> result codes.  All the libraries are now
internal to BIND 9, so we don't need to provide a mechanism to register
extra categories and modules.
2024-08-20 12:50:39 +00:00
Ondřej Surý
8506102216 Remove logging context (isc_log_t) from the public namespace
Now that the logging uses single global context, remove the isc_log_t
from the public namespace.
2024-08-20 12:50:39 +00:00
Ondřej Surý
b2dda86254 Replace isc_log_create/destroy with isc_logconfig_get()
Add isc_logconfig_get() function to get the current logconfig and use
the getter to replace most of the little dancing around setting up
logging in the tools. Thus:

    isc_log_create(mctx, &lctx, &logconfig);
    isc_log_setcontext(lctx);
    dns_log_setcontext(lctx);
    ...
    ...use lcfg...
    ...
    isc_log_destroy();

is now only:

    logconfig = isc_logconfig_get(lctx);
    ...use lcfg...

For thread-safety, isc_logconfig_get() should be surrounded by RCU read
lock, but since we never use isc_logconfig_get() in threaded context,
the only place where it is actually used (but not really needed) is
named_log_init().
2024-08-20 12:50:39 +00:00
Nicki Křížek
3489840948
Update documentation for 9.20.0 2024-07-08 14:53:01 +02:00
Tom Krizek
5893debf46
Remove trailing whitespace from all text files
I've used the following command to remove the trailing whitespace for
all tracked text files:

git grep -Il '' | xargs sed -i 's/[ \t]*$//'
2023-06-13 15:05:40 +02:00
Tony Finch
fc770a8bd0
Remove the now-unused ISC_STACK
We are using the liburcu concurrent data structures instead.
2023-05-12 20:49:43 +01:00
Tony Finch
05ca11e122
Remove isc_qsbr (we are using liburcu instead)
This commit breaks the qp-trie code.
2023-05-12 20:48:31 +01:00
Tony Finch
330ff06d4a Move irs_resconf into libdns and remove libirs
`libirs` used to be a reference implementation of `getaddrinfo` and
related modern resolver APIs. It was stripped down in BIND 9.18
leaving only the `irs_resconf` module, which parses
`/etc/resolv.conf`. I have kept its include path and namespace prefix,
so it remains a little fragment of libirs now embedded in libdns.
2023-02-24 09:38:59 +00:00
Tony Finch
9b7aa536ba QSBR: safe memory reclamation for lock-free data structures
This "quiescent state based reclamation" module provides support for
the qp-trie module in dns/qp. It is a replacement for liburcu, written
without reference to the urcu source code, and in fact it works in a
significantly different way.

A few specifics of BIND make this variant of QSBR somewhat simpler:

  * We can require that wait-free access to a qp-trie only happens in
    an isc_loop callback. The loop provides a natural quiescent state,
    after the callbacks are done, when no qp-trie access occurs.

  * We can dispense with any API like rcu_synchronize(). In practice,
    it takes far too long to wait for a grace period to elapse for each
    write to a data structure.

  * We use the idea of "phases" (aka epochs or eras) from EBR to
    reduce the amount of bookkeeping needed to track memory that is no
    longer needed, knowing that the qp-trie does most of that work
    already.

I considered hazard pointers for safe memory reclamation. They have
more read-side overhead (updating the hazard pointers) and it wasn't
clear to me how to nicely schedule the cleanup work. Another
alternative, epoch-based reclamation, is designed for fine-grained
lock-free updates, so it needs some rethinking to work well with the
heavily read-biased design of the qp-trie. QSBR has the fastest read
side of the basic SMR algorithms (with no barriers), and fits well
into a libuv loop. More recent hybrid SMR algorithms do not appear to
have enough benefits to justify the extra complexity.
2023-02-23 15:57:53 +00:00
Tony Finch
36e56923ce Simple lock-free stack in <isc/stack.h>
Add a singly-linked stack that supports lock-free prepend and drain (to
empty the list and clean up its elements).  Intended for use with QSBR
to collect objects that need safe memory reclamation, or any other user
that works with adding objects to the stack and then draining them in
one go like various work queues.

In <isc/atomic.h>, add an `atomic_ptr()` macro to make type
declarations a little less abominable, and clean up a duplicate
definition of `atomic_compare_exchange_strong_acq_rel()`
2023-02-22 16:13:37 +00:00
Evan Hunt
b058f99cb8 remove references to obsolete isc_task/timer functions
removed references in code comments, doc/dev documentation, etc, to
isc_task, isc_timer_reset(), and isc_timertype_inactive. also removed a
coccinelle patch related to isc_timer_reset() that was no longer needed.
2023-02-22 08:13:30 +00:00
Tony Finch
4da9c582b8 Remove libbind9
It is now empty.
2023-02-21 13:12:26 +00:00
Tony Finch
ec50c58f52 De-duplicate __FILE__, __LINE__
Mostly generated automatically with the following semantic patch,
except where coccinelle was confused by #ifdef in lib/isc/net.c

@@ expression list args; @@
- UNEXPECTED_ERROR(__FILE__, __LINE__, args)
+ UNEXPECTED_ERROR(args)
@@ expression list args; @@
- FATAL_ERROR(__FILE__, __LINE__, args)
+ FATAL_ERROR(args)
2022-10-17 11:58:26 +01:00
Ondřej Surý
b4a43bf2f2
Add developer documentation on the netmgr callbacks
Extra care must be taken when executing the callbacks to prevent the
deadlocks on the caller's side.  Add a paragraph that addresses when we
can and when we cannot call the callbacks directly.
2022-09-29 11:12:15 +02:00
Ondřej Surý
3b31f7f563
Add autoconf option to enable memory leak detection in libraries
There's a known memory leak in the engine_pkcs11 at the time of writing
this and it interferes with the named ability to check for memory leaks
in the OpenSSL memory context by default.

Add an autoconf option to explicitly enable the memory leak detection,
and use it in the CI except for pkcs11 enabled builds.  When this gets
fixed in the engine_pkc11, the option can be enabled by default.
2022-09-27 17:53:04 +02:00
Ondřej Surý
23800ecd86
Add developer note for the libuv quirks 2022-09-19 14:38:56 +02:00
Ondřej Surý
84c90e223f
New event loop handling API
This commit introduces new APIs for applications and signal handling,
intended to replace isc_app for applications built on top of libisc.

* isc_app will be replaced with isc_loopmgr, which handles the
  starting and stopping of applications. In isc_loopmgr, the main
  thread is not blocked, but is part of the working thread set.
  The loop manager will start a number of threads, each with a
  uv_loop event loop running. Setup and teardown functions can be
  assigned which will run when the loop starts and stops, and
  jobs can be scheduled to run in the meantime. When
  isc_loopmgr_shutdown() is run from any the loops, all loops
  will shut down and the application can terminate.

* signal handling will now be handled with a separate isc_signal unit.
  isc_loopmgr only handles SIGTERM and SIGINT for application
  termination, but the application may install additional signal
  handlers, such as SIGHUP as a signal to reload configuration.

* new job running primitives, isc_job and isc_async, have been added.
  Both units schedule callbacks (specifying a callback function and
  argument) on an event loop. The difference is that isc_job unit is
  unlocked and not thread-safe, so it can be used to efficiently
  run jobs in the same thread, while isc_async is thread-safe and
  uses locking, so it can be used to pass jobs from one thread to
  another.

* isc_tid will be used to track the thread ID in isc_loop worker
  threads.

* unit tests have been added for the new APIs.
2022-08-25 12:24:29 +02:00
Tom Krizek
5d2b7cab08
Enforce Python codestyle with black
Black is an opinionated tool for auto-formatting Python code so we no
longer have to worry about the codestyle.

For the codestyle decisions and discussion, refer to the upstream
documentation [1].

[1] https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html
2022-06-08 10:27:46 +02:00
Tom Krizek
5d64d05be9
Remove trailing whitespace
My editor doesn't like that!
2022-06-08 10:27:33 +02:00
Tony Finch
1d807d84f1 Shrink decompression contexts
It's wasteful to use 20 bytes and a pointer indirection to represent
two bits of information, so turn the struct into an enum. And change
the names of the enumeration constants to make the intent more clear.

This change introduces some inline functions into another header,
which confuses `gcovr` when it is trying to collect code coverage
statistics. So, in the CI job, copy more header files into a directory
where `gcovr` looks for them.
2022-06-01 13:00:40 +01:00
Tony Finch
129a522d88 There can no longer be multiple compression methods
The aim is to get rid of the obsolete term "GLOBAL14" and instead just
refer to DNS name compression.

This is mostly mechanically renaming

from	dns_(de)compress_(get|set)methods()
to	dns_(de)compress_(get|set)permitted()

and replacing the related enum by a simple flag, because compression
is either on or off.
2022-06-01 13:00:40 +01:00
Tony Finch
e37b782c1a DNS name compression does not depend on the EDNS version
There was a proposal in the late 1990s that it might, but it turned
out to be unworkable. See RFC 6891, Extension Mechanisms for
DNS (EDNS(0)), section 5, Extended Label Types.

The remnants of the code that supported this in BIND are redundant.
2022-06-01 13:00:40 +01:00
Ondřej Surý
20f0936cf2 Remove use of the inline keyword used as suggestion to compiler
Historically, the inline keyword was a strong suggestion to the compiler
that it should inline the function marked inline.  As compilers became
better at optimising, this functionality has receded, and using inline
as a suggestion to inline a function is obsolete.  The compiler will
happily ignore it and inline something else entirely if it finds that's
a better optimisation.

Therefore, remove all the occurences of the inline keyword with static
functions inside single compilation unit and leave the decision whether
to inline a function or not entirely on the compiler

NOTE: We keep the usage the inline keyword when the purpose is to change
the linkage behaviour.
2022-03-25 08:33:43 +01:00
Ondřej Surý
584f0d7a7e Simplify way we tag unreachable code with only ISC_UNREACHABLE()
Previously, the unreachable code paths would have to be tagged with:

    INSIST(0);
    ISC_UNREACHABLE();

There was also older parts of the code that used comment annotation:

    /* NOTREACHED */

Unify the handling of unreachable code paths to just use:

    UNREACHABLE();

The UNREACHABLE() macro now asserts when reached and also uses
__builtin_unreachable(); when such builtin is available in the compiler.
2022-03-25 08:33:43 +01:00
Ondřej Surý
fe7ce629f4 Add FALLTHROUGH macro for __attribute__((fallthrough))
Gcc 7+ and Clang 10+ have implemented __attribute__((fallthrough)) which
is explicit version of the /* FALLTHROUGH */ comment we are currently
using.

Add and apply FALLTHROUGH macro that uses the attribute if available,
but does nothing on older compilers.

In one case (lib/dns/zone.c), using the macro revealed that we were
using the /* FALLTHROUGH */ comment in wrong place, remove that comment.
2022-03-25 08:33:43 +01:00
Tony Finch
599c1d2a6b Avoid using C99 variable length arrays
From an attacker's point of view, a VLA declaration is essentially a
primitive for performing arbitrary arithmetic on the stack pointer. If
the attacker can control the size of a VLA they have a very powerful
tool for causing memory corruption.

To mitigate this kind of attack, and the more general class of stack
clash vulnerabilities, C compilers insert extra code when allocating a
VLA to probe the growing stack one page at a time. If these probes hit
the stack guard page, the program will crash.

From the point of view of a C programmer, there are a few things to
consider about VLAs:

  * If it is important to handle allocation failures in a controlled
    manner, don't use VLAs. You can use VLAs if it is OK for
    unreasonable inputs to cause an uncontrolled crash.

  * If the VLA is known to be smaller than some known fixed size,
    use a fixed size array and a run-time check to ensure it is large
    enough. This will be more efficient than the compiler's stack
    probes that need to cope with arbitrary-size VLAs.

  * If the VLA might be large, allocate it on the heap. The heap
    allocator can allocate multiple pages in one shot, whereas the
    stack clash probes work one page at a time.

Most of the existing uses of VLAs in BIND are in test code where they
are benign, but there was one instance in `named`, in the GSS-TSIG
verification code, which has now been removed.

This commit adjusts the style guide and the C compiler flags to allow
VLAs in test code but not elsewhere.
2022-03-18 15:11:48 +00:00
Ondřej Surý
c259cecc90 Refactor isc_timer_create() to just create timer
The isc_timer_create() function was a bit conflated.  It could have been
used to create a timer and start it at the same time.  As there was a
single place where this was done before (see the previous commit for
nta.c), this was cleaned up and the isc_timer_create() function was
changed to only create new timer.
2022-03-14 13:00:05 -07:00
Tony Finch
e532d39146 Update the status of Python
There are no longer any Python utilities in BIND: like Perl it is now
used for test scripts and generating some documentation and source
files.
2022-03-09 18:01:28 +00:00
Tony Finch
037468f6a4 Drop references to Windows in doc/dev
Windows is not a supported platform any more.
2022-03-09 18:01:28 +00:00
Tony Finch
328d11297d Small developer documentation cleanups
GitLab replaced RT, we don't support 1990s operating systems, we
like wrapped paragraphs, and we don't need that extra comma.
2022-03-09 18:01:28 +00:00
Tony Finch
ec1e8e7001 Drop documents that have been folded into doc/dev/dev.md
There is no need to keep obsolete duplicate docs around.
2022-03-09 18:01:28 +00:00
Michal Nowak
5d2dd94cf8
Drop runall.sh
runall.sh was mainly used on Windows and as it's support was removed
from the "main" branch the script is not needed anymore.

Also, remove bin/tests/system/README text on running multiple system
test suites simultaneously with runall.sh as that support was not
present in the script anyway.
2022-01-27 11:58:17 +01:00
Petr Špaček
69d0fe9e72
Replace obsolete references to ATF with Cmocka
Fixup for MR!886.
2022-01-18 17:12:57 +01:00
Petr Špaček
631ee58010
Document new system test output format after build system rewrite
Fixup to MR !985.
2022-01-18 17:12:32 +01:00
Petr Špaček
dac41968a9
Fix command to run all unit tests in doc/dev/dev.md
Fixup for build system rewrite 978c7b2e89aa37a7ddfe2f6b6ba12ce73dd04528,
MR !985.
2022-01-18 17:12:27 +01:00
Ondřej Surý
58bd26b6cf Update the copyright information in all files in the repository
This commit converts the license handling to adhere to the REUSE
specification.  It specifically:

1. Adds used licnses to LICENSES/ directory

2. Add "isc" template for adding the copyright boilerplate

3. Changes all source files to include copyright and SPDX license
   header, this includes all the C sources, documentation, zone files,
   configuration files.  There are notes in the doc/dev/copyrights file
   on how to add correct headers to the new files.

4. Handle the rest that can't be modified via .reuse/dep5 file.  The
   binary (or otherwise unmodifiable) files could have license places
   next to them in <foo>.license file, but this would lead to cluttered
   repository and most of the files handled in the .reuse/dep5 file are
   system test files.
2022-01-11 09:05:02 +01:00
Matthijs Mekking
e073b04605 Document typenumber and classnumber format
Document the typenumber and classnumber in doc/dev/rdata so that it
is clear what the expected format is.
2021-10-25 21:33:35 +00:00
Evan Hunt
a55589f881 remove all references to isc_socket and related types
Removed socket.c, socket.h, and all references to isc_socket_t,
isc_socketmgr_t, isc_sockevent_t, etc.
2021-10-15 01:01:25 -07:00
Ondřej Surý
848fef4d4d Update isc_result_t developer documentation
There's no multiple result.{c,h} location to defining result codes,
update the documentation accordingly.
2021-10-07 08:05:16 +02:00
Matthijs Mekking
0bac9c7c5c Add stats unit test
Add a simple stats unit test that tests the existing library functions
isc_stats_ncounters, isc_stats_increment, isc_stats_decrement,
isc_stats_set, and isc_stats_update_if_greater.
2021-08-24 09:07:15 +02:00
Evan Hunt
68a28cbc0a update the "memory" section of the developer doc
Information about memory allocation was outdated.
2021-07-09 15:58:02 +02:00
Matthijs Mekking
71c6d40942 Remove HISTORY file
We should drop the HISTORY file because it's confusing and the same
information is covered by the release notes for .0 releases (or at
least they should be).

Remove references to the HISTORY file, update the README to tell
people go look somewhere else.
2021-06-24 11:44:13 +02:00
Matthijs Mekking
717a39d5b9 Add steps to update copyrights
This was written down in the outdated doc/dev/release documentation.
Since the rest of that file can go, add these steps to a separate file
and update it to current standards (e.g. use git commands).
2021-06-24 11:14:27 +02:00
Ondřej Surý
df4d072c72 Remove unused or outdated utils, developer and design documentation
The util/, doc/design/, and doc/dev/ directories included couple of
tools or documents there were completely outdated because they either
refered the the VCS we no longer use (cvs) or described processes that
have been redesigned and they are documented elsewhere.
2021-06-24 11:14:27 +02:00
Ondřej Surý
440fb3d225 Completely remove BIND 9 Windows support
The Windows support has been completely removed from the source tree
and BIND 9 now no longer supports native compilation on Windows.

We might consider reviewing mingw-w64 port if contributed by external
party, but no development efforts will be put into making BIND 9 compile
and run on Windows again.
2021-06-09 14:35:14 +02:00
Evan Hunt
1fa6f8662e update dev.md with name copying information
revise the Developer Information document with updated information
about copying names.
2021-05-22 00:37:27 -07:00
Ondřej Surý
b540722bc3 Refactor taskmgr to run on top of netmgr
This commit changes the taskmgr to run the individual tasks on the
netmgr internal workers.  While an effort has been put into keeping the
taskmgr interface intact, couple of changes have been made:

 * The taskmgr has no concept of universal privileged mode - rather the
   tasks are either privileged or unprivileged (normal).  The privileged
   tasks are run as a first thing when the netmgr is unpaused.  There
   are now four different queues in in the netmgr:

   1. priority queue - netievent on the priority queue are run even when
      the taskmgr enter exclusive mode and netmgr is paused.  This is
      needed to properly start listening on the interfaces, free
      resources and resume.

   2. privileged task queue - only privileged tasks are queued here and
      this is the first queue that gets processed when network manager
      is unpaused using isc_nm_resume().  All netmgr workers need to
      clean the privileged task queue before they all proceed normal
      operation.  Both task queues are processed when the workers are
      finished.

   3. task queue - only (traditional) task are scheduled here and this
      queue along with privileged task queues are process when the
      netmgr workers are finishing.  This is needed to process the task
      shutdown events.

   4. normal queue - this is the queue with netmgr events, e.g. reading,
      sending, callbacks and pretty much everything is processed here.

 * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t)
   object.

 * The isc_nm_destroy() function now waits for indefinite time, but it
   will print out the active objects when in tracing mode
   (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been
   made a little bit more asynchronous and it might take longer time to
   shutdown all the active networking connections.

 * Previously, the isc_nm_stoplistening() was a synchronous operation.
   This has been changed and the isc_nm_stoplistening() just schedules
   the child sockets to stop listening and exits.  This was needed to
   prevent a deadlock as the the (traditional) tasks are now executed on
   the netmgr threads.

 * The socket selection logic in isc__nm_udp_send() was flawed, but
   fortunatelly, it was broken, so we never hit the problem where we
   created uvreq_t on a socket from nmhandle_t, but then a different
   socket could be picked up and then we were trying to run the send
   callback on a socket that had different threadid than currently
   running.
2021-04-20 23:22:28 +02:00