2
0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-29 05:28:00 +00:00

26 Commits

Author SHA1 Message Date
Ondřej Surý
4e79e9baae
Give every memory context a name
Instead of giving the memory context names with an explicit call to
isc_mem_setname(), add the name to isc_mem_create() call to have all the
memory contexts an unconditional name.
2025-05-29 05:46:46 +02:00
Evan Hunt
afb424c9b6 simplify dns_name_fromtext() interface
previously, dns_name_fromtext() took both a target name and an
optional target buffer parameter, which could override the name's
dedicated buffer. this interface is unnecessarily complex.

we now have two functions, dns_name_fromtext() to convert text
into a dns_name that has a dedicated buffer, and dns_name_wirefromtext()
to convert text into uncompressed DNS wire format and append it to a
target buffer.

in cases where it really is necessary to have both, we can use
dns_name_fromtext() to load the dns_name, then dns_name_towire()
to append the wire format to the target buffer.
2025-02-25 12:53:25 -08:00
Ondřej Surý
08e966df82
Remove offsets from the dns_name and dns_fixedname structures
The offsets were meant to speed-up the repeated dns_name operations, but
it was experimentally proven that there's actually no real-world
benefit.  Remove the offsets and labels fields from the dns_name and the
static offsets fields to save 128 bytes from the fixedname in favor of
calculating labels and offsets only when needed.
2025-02-25 12:17:34 +01:00
Ondřej Surý
f5c204ac3e
Move the library init and shutdown to executables
Instead of relying on unreliable order of execution of the library
constructors and destructors, move them to individual binaries.  The
advantage is that the execution time and order will remain constant and
will not depend on the dynamic load dependency solver.

This requires more work, but that was mitigated by a simple requirement,
any executable using libisc and libdns, must include <isc/lib.h> and
<dns/lib.h> respectively (in this particular order).  In turn, these two
headers must not be included from within any library as they contain
inlined functions marked with constructor/destructor attributes.
2025-02-22 16:19:00 +01:00
Ondřej Surý
0258850f20
Remove redundant parentheses from the return statement 2024-11-19 12:27:22 +01:00
Ondřej Surý
34b3e7cb40
Remove RBTDB implementation
QPDB is now a default implementation for both cache and zone.  Remove
the venerable RBTDB database implementation, so we can fast-track the
changes to the database without having to implement the design changes
to both QPDB and RBTDB and this allows us to be more aggressive when
refactoring the database design.
2024-11-12 09:07:19 +01:00
Ondřej Surý
7b756350f5
Use clang-format-19 to update formatting
This is purely result of running:

    git-clang-format-19 --binary clang-format-19 origin/main
2024-08-22 09:21:55 +02:00
Ondřej Surý
b0c9ca7a24
Revert the s/rbt/qp/ change in load_names, it is still rbt 2024-03-06 17:54:05 +01:00
Evan Hunt
bb4464181a switch database defaults from "rbt" to "qp"
replace the string "rbt" throughout BIND with "qp" so that
qpdb databases will be used by default instead of rbtdb.
rbtdb databases can still be used by specifying "database rbt;"
in a zone statement.
2024-03-06 09:57:24 +01:00
Evan Hunt
ac9bd03a0d clean up dns_rbt
- create_node() in rbt.c cannot fail
- the dns_rbt_*name() functions, which are wrappers around
  dns_rbt_[add|find|delete]node(), were never used except in tests.

this change isn't really necessary since RBT is likely to go away
eventually anyway. but keeping the API as simple as possible while it
persists is a good thing, and may reduce confusion while QPDB is being
developed from RBTDB code.
2024-02-14 01:36:44 -08:00
Ondřej Surý
8404129471
Use DNS_QPGC_MAYBE instead of DNS_QPGC_ALL for more realistic load
In the benchmarks, DNS_QPGC_ALL was trying to hard to cleanup QP
and this was slowing down QP too much.  Use DNS_QPGC_MAYBE instead
that we are going to use anyway for more realistic load - this also
shows the memory usage matching the real loads.
2024-02-08 08:33:36 +01:00
Ondřej Surý
76997983fd
Use EXIT_SUCCESS and EXIT_FAILURE
Instead of randomly using -1 or 1 as a failure status, properly utilize
the EXIT_FAILURE define that's platform specific (as it should be).
2024-02-08 08:01:58 +01:00
Ondřej Surý
9a45fd6f33
Fix hashmap part of load-names benchmark
The name_match() was errorneously converting struct item into dns_name
pointer.  Correctly retype void *node to struct item * first and then
use item.fixed.name to pass the name to dns_name_equal() function.
2023-10-09 21:04:21 +02:00
Ondřej Surý
81f48e11e4
Use read number of items instead of raw array size in load_names
The load_names benchmark expected the input CSV with domains would fill
the whole item array and it would crash when the number of lines would
be less than that.

Fix the expectations by using the real number or lines read to calculate
the array start and end position for each benchmark thread.
2023-10-09 21:04:21 +02:00
Ondřej Surý
e270266627
Refactor isc_hashmap to accept custom match function
Refactor isc_hashmap to allow custom matching functions.  This allows us
to have better tailored keys that don't require fixed uint8_t arrays,
but can be composed of more fields from the stored data structure.
2023-09-16 07:20:48 +02:00
Tony Finch
b38c71961d
Improve qp-trie leaf return values
Make the `pval_r` and `ival_r` out arguments optional.

Add `pval_r` and `ival_r` out arguments to `dns_qp_deletekey()`
and `dns_qp_deletename()`, to return the deleted leaf.
2023-08-15 14:24:39 +02:00
Ondřej Surý
b570750382
Make the load-names benchmark multithreaded
The load-names benchmark was originally only measuring single thread
performance of the data structures.  As this is not how those are used
in the real life, it was refactored to be multi-threaded with proper
protections in place (rwlock for ht, hashmap and rbt; transactions for
qp).

The qp test has been extended to see effect of the dns_qp_compact() and
rcu_barrier() on the overall speed and memory consumption.
2023-07-31 15:51:15 +02:00
Tony Finch
c890b9b124
Get the tests working with liburcu
Mostly a few qp-trie details to adjust.
2023-05-12 20:48:31 +01:00
Tony Finch
b3e35fd120 A few qp-trie cleanups
Revert refcount debug tracing (commit a8b29f0365), there are better
ways to do it.

Use the dns_qpmethods_t typedef where appropriate.

Some stylistic improvements.
2023-04-05 12:35:04 +01:00
Tony Finch
39f38754e2 Compact more in dns_qp_compact(DNS_QPGC_ALL)
Commit 0858514ae8 enriched dns_qp_compact() to give callers more
control over how thoroughly the trie should be compacted.

In the DNS_QPGC_ALL case, if the trie is small it might be compacted
to a new position in the same memory chunk. In this situation it will
still be holding references to old leaf objects which have been
removed from the trie but will not be completely detached until the
chunk containing the references is freed.

This change resets the qp-trie allocator to a fresh chunk before a
DNS_QPGC_ALL compaction, so all the old memory chunks will be
evacuated and old leaf objects can be detached sooner.
2023-04-05 12:35:04 +01:00
Tony Finch
906d434aea Fix Coverity complaints in the qp-trie tests
The main problem was `qp_test_keytoname()` not using `qpkey_bit()`
to do bounds checking.
2023-04-03 15:10:47 +00:00
Ondřej Surý
a5f5f68502
Refactor isc_time_now() to return time, and not result
The isc_time_now() and isc_time_now_hires() were used inconsistently
through the code - either with status check, or without status check,
or via TIME_NOW() macro with RUNTIME_CHECK() on failure.

Refactor the isc_time_now() and isc_time_now_hires() to always fail when
getting current time has failed, and return the isc_time_t value as
return value instead of passing the pointer to result in the argument.
2023-03-31 15:02:06 +02:00
Tony Finch
a8b29f0365 Improve qp-trie refcount debugging
Add some qp-trie tracing macros which can be enabled by a
developer. These print a message when a leaf is attached or
detached, indicating which part of the qp-trie implementation
did so. The refcount methods must now return the refcount value
so it can be printed by the trace macros.
2023-02-27 13:47:57 +00:00
Tony Finch
4b5ec07bb7 Refactor qp-trie to use QSBR
The first working multi-threaded qp-trie was stuck with an unpleasant
trade-off:

  * Use `isc_rwlock`, which has acceptable write performance, but
    terrible read scalability because the qp-trie made all accesses
    through a single lock.

  * Use `liburcu`, which has great read scalability, but terrible
    write performance, because I was relying on `rcu_synchronize()`
    which is rather slow. And `liburcu` is LGPL.

To get the best of both worlds, we need our own scalable read side,
which we now have with `isc_qsbr`. And we need to modify the write
side so that it is not blocked by readers.

Better write performance requires an async cleanup function like
`call_rcu()`, instead of the blocking `rcu_synchronize()`. (There
is no blocking cleanup in `isc_qsbr`, because I have concluded
that it would be an attractive nuisance.)

Until now, all my multithreading qp-trie designs have been based
around two versions, read-only and mutable. This is too few to
work with asynchronous cleanup. The bare minimum (as in epoch
based reclamation) is three, but it makes more sense to support an
arbitrary number. Doing multi-version support "properly" makes
fewer assumptions about how safe memory reclamation works, and it
makes snapshots and rollbacks simpler.

To avoid making the memory management even more complicated, I
have introduced a new kind of "packed reader node" to anchor the
root of a version of the trie. This is simpler because it re-uses
the existing chunk lifetime logic - see the discussion under
"packed reader nodes" in `qp_p.h`.

I have also made the chunk lifetime logic simpler. The idea of a
"generation" is gone; instead, chunks are either mutable or
immutable. And the QSBR phase number is used to indicate when a
chunk can be reclaimed.

Instead of the `shared_base` flag (which was basically a one-bit
reference count, with a two version limit) the base array now has a
refcount, which replaces the confusing ad-hoc lifetime logic with
something more familiar and systematic.
2023-02-27 13:47:55 +00:00
Tony Finch
549854f63b Some minor qp-trie improvements
Adjust the dns_qp_memusage() and dns_qp_compact() functions
to be more informative and flexible about handling fragmentation.

Avoid wasting space in runt chunks.

Switch from twigs_mutable() to cells_immutable() because that is the
sense we usually want.

Drop the redundant evacuate() function and rename evacuate_twigs() to
evacuate(). Move some chunk test functions closer to their point of
use.

Clarify compact_recursive(). Some small cleanups to comments.

Use isc_time_monotonic() for qp-trie timing stats.

Use #define constants to control debug logging.

Set up DNS name label offsets in dns_qpkey_fromname() so it is easier
to use in cases where the name is not fully hydrated.
2023-02-27 13:47:25 +00:00
Tony Finch
a9d57b91db Benchmarks for the qp-trie
The main benchmark is `qpmulti`, which exercizes the qp-trie
transactional API with differing numbers of threads and differing data
sizes, to get some idea of how its performance scales.

The `load-names` benchmark compares the times to populate and query
and the memory used by various BIND data structures: qp-trie, hash
table (chained), hash map (closed), and red-black tree.

The `qp-dump` program is a test utility rather than a benchmark. It
populates a qp-trie and prints it out, either in an ad-hoc text
format, or as input to the graphviz `dot` program.
2023-02-27 13:47:25 +00:00