mir/bind
2
0
Fork 0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-23 10:39:16 +00:00
Code Issues Releases Wiki Activity
bind/lib/dns/adb.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

3355 lines
82 KiB
C
Raw Permalink Normal View History

Start on address.c
1999-09-21 22:46:42 +00:00
/*
formally cast to int
2018-02-14 19:13:56 +11:00
* Copyright (C) Internet Systems Consortium, Inc. ("ISC")
Start on address.c
1999-09-21 22:46:42 +00:00
*
* SPDX-License-Identifier: MPL-2.0
Update the copyright information in all files in the repository This commit converts the license handling to adhere to the REUSE specification. It specifically: 1. Adds used licnses to LICENSES/ directory 2. Add "isc" template for adding the copyright boilerplate 3. Changes all source files to include copyright and SPDX license header, this includes all the C sources, documentation, zone files, configuration files. There are notes in the doc/dev/copyrights file on how to add correct headers to the new files. 4. Handle the rest that can't be modified via .reuse/dep5 file. The binary (or otherwise unmodifiable) files could have license places next to them in <foo>.license file, but this would lead to cluttered repository and most of the files handled in the .reuse/dep5 file are system test files.
2021-06-03 08:37:05 +02:00
*
Start on address.c
1999-09-21 22:46:42 +00:00
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, you can obtain one at https://mozilla.org/MPL/2.0/.
Update license headers to not include years in copyright in all applicable files
2018-02-23 09:53:12 +01:00
*
Start on address.c
1999-09-21 22:46:42 +00:00
* See the COPYRIGHT file distributed with this work for additional
2947. [func] Add new zone type "static-stub". It's like a stub zone, but the nameserver names and/or their IP addresses are statically configured. [RT #21474] (for 9.8.0)
2010-12-16 09:51:30 +00:00
* information regarding copyright ownership.
Start on address.c
1999-09-21 22:46:42 +00:00
*/
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
/*! \file */
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
Replace custom isc_u?intNN_t types with C99 u?intNN_t types
2018-03-28 14:19:37 +02:00
#include <inttypes.h>
add limits.h for INT_MAX
2000-01-21 02:13:38 +00:00
#include <limits.h>
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
#include <stdbool.h>
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
#include <isc/async.h>
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
#include <isc/atomic.h>
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
#include <isc/attributes.h>
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
#include <isc/hash.h>
Use isc_hashmap instead of isc_ht in the dns_adb API Replace the use of isc_ht API with isc_hashmap API in the dns_adb database implementation. This requires extending the dns_adbnamebucket_t and dns_adbentrybucket_t structures to include keysize and copy of the key because the isc_hashmap API needs the raw key in case of resizing the hashmap table.
2022-06-27 12:46:06 +02:00
#include <isc/hashmap.h>
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#include <isc/list.h>
Convert all categories and modules into static lists Remove the complicated mechanism that could be (in theory) used by external libraries to register new categories and modules with statically defined lists in <isc/log.h>. This is similar to what we have done for <isc/result.h> result codes. All the libraries are now internal to BIND 9, so we don't need to provide a mechanism to register extra categories and modules.
2024-08-14 13:25:50 +02:00
#include <isc/log.h>
Create per-thread task for dns_adb resolver fetches The dns_adb would serialize all fetches on a single task. Create a per-thread task, so the fetches will stay local to the thread that initiated the fetch.
2022-10-13 08:11:30 +02:00
#include <isc/loop.h>
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#include <isc/mutex.h>
dump the ADB in a more user-friendly format (unless a debug flag is set)
2000-12-18 20:03:33 +00:00
#include <isc/netaddr.h>
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
#include <isc/os.h>
snapshot
1999-10-20 23:26:06 +00:00
#include <isc/random.h>
Make isc_result a static enum Remove the dynamic registration of result codes. Convert isc_result_t from unsigned + #defines into 32-bit enum type in grand unified <isc/result.h> header. Keep the existing values of the result codes even at the expense of the description and identifier tables being unnecessary large. Additionally, add couple of: switch (result) { [...] default: break; } statements where compiler now complains about missing enum values in the switch statement.
2021-10-04 17:14:53 +02:00
#include <isc/result.h>
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
#include <isc/rwlock.h>
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
#include <isc/sieve.h>
2537. [func] Added more statistics counters including those on socket I/O events and query RTT histograms. [RT #18802]
2009-01-27 22:30:00 +00:00
#include <isc/stats.h>
Clean up #include <isc/string.h> It isn't just about HP/UX any more.
2022-04-11 15:53:34 +01:00
#include <isc/string.h>
Create per-thread task for dns_adb resolver fetches The dns_adb would serialize all fetches on a single task. Create a per-thread task, so the fetches will stay local to the thread that initiated the fetch.
2022-10-13 08:11:30 +02:00
#include <isc/tid.h>
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
#include <isc/urcu.h>
move util.h to <isc/util.h>
1999-12-16 22:24:22 +00:00
#include <isc/util.h>
snapshot address.[ch]
1999-09-22 00:26:20 +00:00
s/address/adb/
1999-10-22 01:02:38 +00:00
#include <dns/adb.h>
database lookups
1999-10-21 01:18:06 +00:00
#include <dns/db.h>
snapshot
1999-10-05 23:50:43 +00:00
#include <dns/rdata.h>
clean up suspicious looking and incorrect uses of dns_name_fromregion
2000-10-07 00:09:28 +00:00
#include <dns/rdatastruct.h>
1889. [func] The lame cache is now done on a <qname,qclass,qtype> basis as some servers only appear to be lame for certain query types. [RT #14916]
2005-06-23 04:22:02 +00:00
#include <dns/rdatatype.h>
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
#include <dns/stats.h>
Resolver query forwarding to DoT-enabled upstream servers Implement TLS transport usage in the resolver. Use the configured TLS transport for the forwarders in the resolver.
2022-12-08 14:18:22 +00:00
#include <dns/transport.h>
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
#include <dns/types.h>
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
#define DNS_ADB_MAGIC ISC_MAGIC('D', 'a', 'd', 'b')
#define DNS_ADB_VALID(x) ISC_MAGIC_VALID(x, DNS_ADB_MAGIC)
#define DNS_ADBNAME_MAGIC ISC_MAGIC('a', 'd', 'b', 'N')
#define DNS_ADBNAME_VALID(x) ISC_MAGIC_VALID(x, DNS_ADBNAME_MAGIC)
#define DNS_ADBNAMEHOOK_MAGIC ISC_MAGIC('a', 'd', 'N', 'H')
use isc/magic.h, and (unfortunately) export dns_adbhandle_t's definition
1999-09-25 01:25:46 +00:00
#define DNS_ADBNAMEHOOK_VALID(x) ISC_MAGIC_VALID(x, DNS_ADBNAMEHOOK_MAGIC)
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
#define DNS_ADBENTRY_MAGIC ISC_MAGIC('a', 'd', 'b', 'E')
#define DNS_ADBENTRY_VALID(x) ISC_MAGIC_VALID(x, DNS_ADBENTRY_MAGIC)
#define DNS_ADBFETCH_MAGIC ISC_MAGIC('a', 'd', 'F', '4')
#define DNS_ADBFETCH_VALID(x) ISC_MAGIC_VALID(x, DNS_ADBFETCH_MAGIC)
#define DNS_ADBFETCH6_MAGIC ISC_MAGIC('a', 'd', 'F', '6')
#define DNS_ADBFETCH6_VALID(x) ISC_MAGIC_VALID(x, DNS_ADBFETCH6_MAGIC)
1851. [doc] Doxygen comment markup. [RT #11398]
2005-04-27 04:57:32 +00:00
/*!
Ignore ipv6 mapped ipv4 addresses. Set a default TTL for type 3 negative cache elements. This is local to the ADB, and cannot leak to clients, so no specs are violated. Change a logging level to be less verbose.
1999-12-01 02:05:16 +00:00
* For type 3 negative cache entries, we will remember that the address is
change macro name from ADB_NCACHE_MINIMUM to ADB_CACHE_MINIMUM since it's used as a general purpose minimum ttl, and actually set the TTL to the minimum when re get a negative cache entry.
1999-12-11 02:30:24 +00:00
* broken for this long. XXXMLG This is also used for actual addresses, too.
1402. [cleanup] A6 has been moved to experimental and is no longer fully supported. developer: jinmei reviewer: marka
2002-11-27 09:52:58 +00:00
* The intent is to keep us from constantly asking about A/AAAA records
change macro name from ADB_NCACHE_MINIMUM to ADB_CACHE_MINIMUM since it's used as a general purpose minimum ttl, and actually set the TTL to the minimum when re get a negative cache entry.
1999-12-11 02:30:24 +00:00
* if the zone has extremely low TTLs.
Ignore ipv6 mapped ipv4 addresses. Set a default TTL for type 3 negative cache elements. This is local to the ADB, and cannot leak to clients, so no specs are violated. Change a logging level to be less verbose.
1999-12-01 02:05:16 +00:00
*/
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
#define ADB_CACHE_MINIMUM 10 /*%< seconds */
#define ADB_CACHE_MAXIMUM 86400 /*%< seconds (86400 = 24 hours) */
Change the ADB_ENTRY_WINDOW to 60 seconds The previous value of 30 minutes used to cache the ADB names and entries was quite long. Change the value to 60 seconds for faster recovery after cached intermittent failure of the remote nameservers.
2024-04-03 12:07:41 +02:00
#define ADB_ENTRY_WINDOW 60 /*%< seconds */
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
#define ADB_HASH_SIZE (1 << 12)
Use isc_hashmap instead of isc_ht in the dns_adb API Replace the use of isc_ht API with isc_hashmap API in the dns_adb database implementation. This requires extending the dns_adbnamebucket_t and dns_adbentrybucket_t structures to include keysize and copy of the key because the isc_hashmap API needs the raw key in case of resizing the hashmap table.
2022-06-27 12:46:06 +02:00
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
/*%
* The period in seconds after which an ADB name entry is regarded as stale
* and forced to be cleaned up.
* TODO: This should probably be configurable at run-time.
*/
#ifndef ADB_STALE_MARGIN
#define ADB_STALE_MARGIN 1800
#endif /* ifndef ADB_STALE_MARGIN */
start on cleanup timer/task
1999-10-25 22:53:15 +00:00
silence compiler warnings
2013-03-05 23:41:22 +11:00
#define DNS_ADB_MINADBSIZE (1024U * 1024U) /*%< 1 Megabyte */
1073. [bug] The adb cache should also be space driven, max-adb-size. [RT #1915, #1938]
2001-10-25 04:57:46 +00:00
adjust what points to what. You are lost in a maze of twisty little pointers, all alike.
1999-09-22 01:01:14 +00:00
typedef ISC_LIST(dns_adbname_t) dns_adbnamelist_t;
snapshot address.[ch]
1999-09-22 00:26:20 +00:00
typedef struct dns_adbnamehook dns_adbnamehook_t;
more v4/v6 splitting out
1999-10-29 01:26:45 +00:00
typedef ISC_LIST(dns_adbnamehook_t) dns_adbnamehooklist_t;
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
typedef ISC_LIST(dns_adbentry_t) dns_adbentrylist_t;
start on fetch infrastructure
1999-10-25 21:02:54 +00:00
typedef struct dns_adbfetch dns_adbfetch_t;
a6 checkpoint
1999-10-30 01:22:16 +00:00
typedef struct dns_adbfetch6 dns_adbfetch6_t;
Start on address.c
1999-09-21 22:46:42 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
typedef struct dns_adblru {
ISC_SIEVE(dns_adbname_t) names;
ISC_SIEVE(dns_adbentry_t) entries;
uint8_t __padding[ISC_OS_CACHELINE_SIZE -
(sizeof(ISC_SIEVE(dns_adbname_t)) +
sizeof(ISC_SIEVE(dns_adbentry_t))) %
ISC_OS_CACHELINE_SIZE];
} dns_adblru_t;
1851. [doc] Doxygen comment markup. [RT #11398]
2005-04-27 04:57:32 +00:00
/*% dns adb structure */
Start on address.c
1999-09-21 22:46:42 +00:00
struct dns_adb {
update copyright notice
2008-02-07 23:46:54 +00:00
unsigned int magic;
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
uint32_t nloops;
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
isc_mutex_t lock;
isc_mem_t *mctx;
Exclude the ADB hashmaps from ADB overmem accounting The ADB overmem accounting would include the memory used by hashtables thus vastly reducing the space that can be used for ADB names and entries when the hashtables would grow. Create own memory context for the ADB names and entries hash tables.
2022-12-13 14:14:21 +01:00
isc_mem_t *hmctx;
update copyright notice
2008-02-07 23:46:54 +00:00
dns_view_t *view;
attach the resolver to the ADB the ADB depends on the resolver, but previously only accessed it via the view. as view->resolver may now be detached before the ADB finishes, a shutdown race was possible. attaching to the resolver directly prevents this.
2022-05-12 15:51:10 -07:00
dns_resolver_t *res;
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
isc_refcount_t references;
update copyright notice
2008-02-07 23:46:54 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adblru_t *lru;
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
struct cds_lfht *names_ht;
struct cds_lfht *entries_ht;
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
isc_stats_t *stats;
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
atomic_bool shuttingdown;
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
Replace custom isc_u?intNN_t types with C99 u?intNN_t types
2018-03-28 14:19:37 +02:00
uint32_t quota;
uint32_t atr_freq;
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
double atr_low;
double atr_high;
double atr_discount;
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
struct rcu_head rcu_head;
Start on address.c
1999-09-21 22:46:42 +00:00
};
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/*%
* dns_adbname structure:
*
* This is the structure representing a nameserver name; it can be looked
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
* up via the adb->names hash table. It holds references to fetches
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* for A and AAAA records while they are ongoing (fetch_a, fetch_aaaa), and
* lists of records pointing to address information when the fetches are
* complete (v4, v6).
review checkpoint
1999-11-25 01:23:32 +00:00
*/
snapshot address.[ch]
1999-09-22 00:26:20 +00:00
struct dns_adbname {
update copyright notice
2008-02-07 23:46:54 +00:00
unsigned int magic;
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
isc_refcount_t references;
update copyright notice
2008-02-07 23:46:54 +00:00
dns_adb_t *adb;
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
dns_fixedname_t fname;
dns_name_t *name;
update copyright notice
2008-02-07 23:46:54 +00:00
unsigned int partial_result;
unsigned int flags;
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
unsigned int type;
update copyright notice
2008-02-07 23:46:54 +00:00
isc_stdtime_t expire_v4;
isc_stdtime_t expire_v6;
dns_adbnamehooklist_t v4;
dns_adbnamehooklist_t v6;
dns_adbfetch_t *fetch_a;
dns_adbfetch_t *fetch_aaaa;
unsigned int fetch_err;
unsigned int fetch6_err;
dns_adbfindlist_t finds;
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
isc_mutex_t lock;
update copyright notice
2008-02-07 23:46:54 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
/* for LFHT */
struct cds_lfht_node ht_node;
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
/* for LRU-based management */
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
ISC_LINK(dns_adbname_t) link;
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
bool visited;
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
isc_loop_t *loop;
struct rcu_head rcu_head;
struct cds_list_head lru_head;
snapshot address.[ch]
1999-09-22 00:26:20 +00:00
};
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#if DNS_ADB_TRACE
#define dns_adbname_ref(ptr) dns_adbname__ref(ptr, __func__, __FILE__, __LINE__)
#define dns_adbname_unref(ptr) \
dns_adbname__unref(ptr, __func__, __FILE__, __LINE__)
#define dns_adbname_attach(ptr, ptrp) \
dns_adbname__attach(ptr, ptrp, __func__, __FILE__, __LINE__)
#define dns_adbname_detach(ptrp) \
dns_adbname__detach(ptrp, __func__, __FILE__, __LINE__)
ISC_REFCOUNT_TRACE_DECL(dns_adbname);
#else
ISC_REFCOUNT_DECL(dns_adbname);
#endif
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/*%
* dns_adbfetch structure:
* Stores the state for an ongoing A or AAAA fetch.
*/
start on fetch infrastructure
1999-10-25 21:02:54 +00:00
struct dns_adbfetch {
update copyright notice
2008-02-07 23:46:54 +00:00
unsigned int magic;
dns_fetch_t *fetch;
dns_rdataset_t rdataset;
[master] limit recursion depth and iterative queries 4006. [security] A flaw in delegation handling could be exploited to put named into an infinite loop. This has been addressed by placing limits on the number of levels of recursion named will allow (default 7), and the number of iterative queries that it will send (default 50) before terminating a recursive query (CVE-2014-8500). The recursion depth limit is configured via the "max-recursion-depth" option. [RT #35780]
2014-11-17 23:24:44 -08:00
unsigned int depth;
start on fetch infrastructure
1999-10-25 21:02:54 +00:00
};
1851. [doc] Doxygen comment markup. [RT #11398]
2005-04-27 04:57:32 +00:00
/*%
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* dns_adbnamehook structure:
*
snapshot address.[ch]
1999-09-22 00:26:20 +00:00
* This is a small widget that dangles off a dns_adbname_t. It contains a
* pointer to the address information about this host, and a link to the next
* namehook that will contain the next address this host has.
*/
struct dns_adbnamehook {
update copyright notice
2008-02-07 23:46:54 +00:00
unsigned int magic;
dns_adbentry_t *entry;
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
ISC_LINK(dns_adbnamehook_t) name_link;
ISC_LINK(dns_adbnamehook_t) entry_link;
Start on address.c
1999-09-21 22:46:42 +00:00
};
1851. [doc] Doxygen comment markup. [RT #11398]
2005-04-27 04:57:32 +00:00
/*%
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* dns_adbentry structure:
*
* This is the structure representing a nameserver address; it can be looked
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
* up via the adb->entries hash table. Also, each dns_adbnamehook and
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* and dns_adbaddrinfo object will contain a pointer to one of these.
*
* The structure holds quite a bit of information about addresses,
* including edns state (in "flags"), RTT, and of course the address of
removed unused edns_level member from ADB entry
2001-01-22 22:53:13 +00:00
* the host.
snapshot address.[ch]
1999-09-22 00:26:20 +00:00
*/
Start on address.c
1999-09-21 22:46:42 +00:00
struct dns_adbentry {
update copyright notice
2008-02-07 23:46:54 +00:00
unsigned int magic;
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
dns_adb_t *adb;
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
isc_mutex_t lock;
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
isc_refcount_t references;
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
dns_adbnamehooklist_t nhs;
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
atomic_uint flags;
atomic_uint srtt;
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
unsigned int completed;
unsigned int timeouts;
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
unsigned char plain;
unsigned char plainto;
unsigned char edns;
Simplify the EDNS buffer size logic for DNS Flag Day 2020 The DNS Flag Day 2020 aims to remove the IP fragmentation problem from the UDP DNS communication. In this commit, we implement the required changes and simplify the logic for picking the EDNS Buffer Size. 1. The defaults for `edns-udp-size`, `max-udp-size` and `nocookie-udp-size` have been changed to `1232` (the value picked by DNS Flag Day 2020). 2. The probing heuristics that would try 512->4096->1432->1232 buffer sizes has been removed and the resolver will always use just the `edns-udp-size` value. 3. Instead of just disabling the PMTUD mechanism on the UDP sockets, we now set IP_DONTFRAG (IPV6_DONTFRAG) flag. That means that the UDP packets won't get ever fragmented. If the ICMP packets are lost the UDP will just timeout and eventually be retried over TCP.
2020-09-23 14:47:26 +02:00
unsigned char ednsto;
Reduce padding in ADB address entries Saves a couple of bytes in `struct dns_adbentry`.
2022-04-06 18:37:54 +01:00
uint16_t udpsize;
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
Replace custom isc_u?intNN_t types with C99 u?intNN_t types
2018-03-28 14:19:37 +02:00
uint8_t mode;
lib/dns/adb.c: Use atomics for adb quota values and reference counting
2019-01-22 10:47:18 +01:00
atomic_uint_fast32_t quota;
atomic_uint_fast32_t active;
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
double atr;
update copyright notice
2008-02-07 23:46:54 +00:00
isc_sockaddr_t sockaddr;
4152. [func] Implement DNS COOKIE option. This replaces the experimental SIT option of BIND 9.10. The following named.conf directives are avaliable: send-cookie, cookie-secret, cookie-algorithm and nocookie-udp-size. The following dig options are available: +[no]cookie[=value] and +[no]badcookie. [RT #39928]
2015-07-06 09:44:24 +10:00
unsigned char *cookie;
Replace custom isc_u?intNN_t types with C99 u?intNN_t types
2018-03-28 14:19:37 +02:00
uint16_t cookielen;
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
Always set ADB entry expiration to now + ADB_ENTRY_WINDOW When ADB entry was created it was set to never expire. If we never called any of the functions that adjust the expiration, it could linger in the ADB forever. Set the expiration (.expires) to now + ADB_ENTRY_WINDOW when creating the new ADB entry to ensure the ADB entry will always expire.
2024-04-03 11:33:48 +02:00
isc_stdtime_t expires;
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
_Atomic(isc_stdtime_t) lastage;
update copyright notice
2008-02-07 23:46:54 +00:00
/*%<
* A nonzero 'expires' field indicates that the entry should
* persist until that time. This allows entries found
* using dns_adb_findaddrinfo() to persist for a limited time
* even though they are not necessarily associated with a
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
* entry.
update copyright notice
2008-02-07 23:46:54 +00:00
*/
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
struct cds_lfht_node ht_node;
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
ISC_LINK(dns_adbentry_t) link;
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
bool visited;
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
isc_loop_t *loop;
struct rcu_head rcu_head;
struct cds_list_head lru_head;
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
};
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#if DNS_ADB_TRACE
#define dns_adbentry_ref(ptr) \
dns_adbentry__ref(ptr, __func__, __FILE__, __LINE__)
#define dns_adbentry_unref(ptr) \
dns_adbentry__unref(ptr, __func__, __FILE__, __LINE__)
#define dns_adbentry_attach(ptr, ptrp) \
dns_adbentry__attach(ptr, ptrp, __func__, __FILE__, __LINE__)
#define dns_adbentry_detach(ptrp) \
dns_adbentry__detach(ptrp, __func__, __FILE__, __LINE__)
ISC_REFCOUNT_TRACE_DECL(dns_adbentry);
#else
ISC_REFCOUNT_DECL(dns_adbentry);
#endif
snapshot address.[ch]
1999-09-22 00:26:20 +00:00
/*
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
* Internal functions (and prototypes).
*/
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbname_t *
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
new_adbname(dns_adb_t *adb, const dns_name_t *, unsigned int type);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
destroy_adbname(dns_adbname_t *);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
static int
match_adbname(struct cds_lfht_node *ht_node, const void *key);
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
static uint32_t
hash_adbname(const dns_adbname_t *adbname);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbnamehook_t *
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
new_adbnamehook(dns_adb_t *adb);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
free_adbnamehook(dns_adb_t *adb, dns_adbnamehook_t **namehookp);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbentry_t *
Always set ADB entry expiration to now + ADB_ENTRY_WINDOW When ADB entry was created it was set to never expire. If we never called any of the functions that adjust the expiration, it could linger in the ADB forever. Set the expiration (.expires) to now + ADB_ENTRY_WINDOW when creating the new ADB entry to ensure the ADB entry will always expire.
2024-04-03 11:33:48 +02:00
new_adbentry(dns_adb_t *adb, const isc_sockaddr_t *addr, isc_stdtime_t now);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
destroy_adbentry(dns_adbentry_t *entry);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
static int
match_adbentry(struct cds_lfht_node *ht_node, const void *key);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbfind_t *
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
new_adbfind(dns_adb_t *, in_port_t);
static void
free_adbfind(dns_adbfind_t **);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbaddrinfo_t *
set the default destination port in the view, and use it when returning addresses from the adb.
2000-06-01 00:30:58 +00:00
new_adbaddrinfo(dns_adb_t *, dns_adbentry_t *, in_port_t);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbfetch_t *
more v4/v6 splitting out
1999-10-29 01:26:45 +00:00
new_adbfetch(dns_adb_t *);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
more v4/v6 splitting out
1999-10-29 01:26:45 +00:00
free_adbfetch(dns_adb_t *, dns_adbfetch_t **);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
static void
When overmem, clean enough memory when adding new ADB names/entries The purge_stale_names()/purge_stale_entries() is opportunistic even when we are under memory pressure (overmem). Split the opportunistic LRU cleaning and overmem cleaning. This makes the stale purging much simpler as we don't have to try that hard and makes the overmem cleaning always cleanup double the amount of the newly allocated ADB name/entry.
2025-06-25 12:05:14 +02:00
purge_names_overmem(dns_adb_t *adb, size_t requested);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
static dns_adbname_t *
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
get_attached_and_locked_name(dns_adb_t *, const dns_name_t *, unsigned int type,
isc_stdtime_t now);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
static void
When overmem, clean enough memory when adding new ADB names/entries The purge_stale_names()/purge_stale_entries() is opportunistic even when we are under memory pressure (overmem). Split the opportunistic LRU cleaning and overmem cleaning. This makes the stale purging much simpler as we don't have to try that hard and makes the overmem cleaning always cleanup double the amount of the newly allocated ADB name/entry.
2025-06-25 12:05:14 +02:00
purge_entries_overmem(dns_adb_t *adb, size_t requested);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbentry_t *
Lock the adbname and adbentry prior to unlocking hash locks There was a datarace that could expire a freshly created ADB names and entries between the return from get_attached_{name,entry} and locking it again. Lock the ADB name and ADB entry inside the hash table lock, so they won't get expired until the full initialization has been complete.
2022-12-13 13:48:55 +01:00
get_attached_and_locked_entry(dns_adb_t *adb, isc_stdtime_t now,
const isc_sockaddr_t *addr);
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
static void
dump_adb(dns_adb_t *, FILE *, bool debug, isc_stdtime_t);
4546. [func] Extend the use of const declarations. [RT #43379]
2016-12-30 15:45:08 +11:00
static void
dump the ADB in a more user-friendly format (unless a debug flag is set)
2000-12-18 20:03:33 +00:00
print_namehook_list(FILE *, const char *legend, dns_adb_t *adb,
update copyright notice
2008-02-07 23:46:54 +00:00
dns_adbnamehooklist_t *list, bool debug, isc_stdtime_t now);
replace all instances of 'handle' with 'find' since that's what they are now.
1999-10-29 18:30:48 +00:00
static void
print_find_list(FILE *, dns_adbname_t *);
snapshot; nearly all fetch infrastructure is in place.
1999-10-27 19:36:58 +00:00
static void
print_fetch_list(FILE *, dns_adbname_t *);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Don't expire an ADB entry without holding the entries lock The clean_namehooks() function does't hold the 'adb->entries_lock' lock, so calling maybe_expire_entry() is not thread-safe. Instead of adding a lock/unlock, leave the expiration to later, e.g. by the get_attached_and_locked_entry() function. Also fix a couple of comment typos.
2022-12-23 14:21:03 +00:00
clean_namehooks(dns_adb_t *, dns_adbnamehooklist_t *);
Lower cache minimum time to 10 seconds. Enforce the minimum. Preliminary support for the "target" attribute of the adbname, to be used for CNAME/DNAME chains later on. The resolver will now do appropriate findzonecut calls, so the ADB need not concern itself with them any more. Fix some bugs in NXDOMAIN/NXRRSET handling.
2000-01-19 01:43:58 +00:00
static void
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
clean_finds_at_name(dns_adbname_t *, dns_adbstatus_t, unsigned int);
CNAME/DNAME chaining (initial support; A6 still missing)
2000-01-21 02:50:27 +00:00
static void
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
maybe_expire_namehooks(dns_adbname_t *, isc_stdtime_t);
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
static bool
name_expired(dns_adbname_t *adbname, isc_stdtime_t now);
static bool
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
maybe_expire_name(dns_adbname_t *adbname, isc_stdtime_t now);
static void
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
expire_name(dns_adbname_t *adbname, dns_adbstatus_t astat);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
static bool
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
entry_expired(dns_adbentry_t *adbentry, isc_stdtime_t now);
static bool
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
maybe_expire_entry(dns_adbentry_t *adbentry, isc_stdtime_t now);
static void
expire_entry(dns_adbentry_t *adbentry);
fix a few bugs, and start an AAAA db lookup/query if no A6 is found
1999-11-02 00:29:45 +00:00
static isc_result_t
dbfind_name(dns_adbname_t *, isc_stdtime_t, dns_rdatatype_t);
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
static isc_result_t
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
fetch_name(dns_adbname_t *, bool, bool, unsigned int, isc_counter_t *qc,
Implement global limit for outgoing queries This global limit is not reset on query restarts and is a hard limit for any client request.
2024-11-11 11:01:50 +01:00
isc_counter_t *gqc, dns_rdatatype_t);
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
shutdown_names(dns_adb_t *);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
static void
remove isc_task completely as there is no further use of isc_task in BIND, this commit removes it, along with isc_taskmgr, isc_event, and all other related types. functions that accepted taskmgr as a parameter have been cleaned up. as a result of this change, some functions can no longer fail, so they've been changed to type void, and their callers have been updated accordingly. the tasks table has been removed from the statistics channel and the stats version has been updated. dns_dyndbctx has been changed to reference the loopmgr instead of taskmgr, and DNS_DYNDB_VERSION has been udpated as well.
2022-10-29 14:22:56 -07:00
shutdown_entries(dns_adb_t *);
static void
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
dump_entry(FILE *, dns_adb_t *, dns_adbentry_t *, bool, isc_stdtime_t);
3855. [bug] Limit smoothed round trip time aging to no more than once a second. [RT #32909]
2014-05-21 10:08:52 +10:00
static void
adjustsrtt(dns_adbaddrinfo_t *addr, unsigned int rtt, unsigned int factor,
isc_stdtime_t now);
[rt36341] 3905. [bug] Address deadlock between view.c and adb.c. [RT #36341]
2014-07-31 11:38:11 +10:00
static void
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
log_quota(dns_adbentry_t *entry, const char *fmt, ...) ISC_FORMAT_PRINTF(2, 3);
more v4/v6 splitting out
1999-10-29 01:26:45 +00:00
Change dns_adbentry_overquota() to dns_adb_overquota() The dns_adbentry_overquota() was violating the layers accessing the adbentry struct members directly. Change it to dns_adb_overquota() to match the dns_adb API.
2023-03-30 21:19:17 +02:00
static bool
adbentry_overquota(dns_adbentry_t *entry);
dead/needs_poke are flags for adbnames, merge in Bob's changes
1999-10-30 01:58:02 +00:00
/*
clean up numbering of FETCHOPT and ADDRINFO flags in the past there was overlap between the fields used as resolver fetch options and ADB addrinfo flags. this has mostly been eliminated; now we can clean up the rest of it and remove some confusing comments.
2023-07-03 12:43:17 -07:00
* Private flag(s) for adbfind objects. These are used internally and
* are not meant to be seen or used by the caller; however, we use the
* same flags field as for DNS_ADBFIND_xxx flags, so we must be careful
* that there is no overlap between these values and those. To make it
* easier, we will number these starting from the most significant bit
* instead of the least significant.
dead/needs_poke are flags for adbnames, merge in Bob's changes
1999-10-30 01:58:02 +00:00
*/
clean up numbering of FETCHOPT and ADDRINFO flags in the past there was overlap between the fields used as resolver fetch options and ADB addrinfo flags. this has mostly been eliminated; now we can clean up the rest of it and remove some confusing comments.
2023-07-03 12:43:17 -07:00
enum {
FIND_EVENT_SENT = 1 << 31,
};
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
#define FIND_EVENTSENT(h) (((h)->flags & FIND_EVENT_SENT) != 0)
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
clean up numbering of FETCHOPT and ADDRINFO flags in the past there was overlap between the fields used as resolver fetch options and ADB addrinfo flags. this has mostly been eliminated; now we can clean up the rest of it and remove some confusing comments.
2023-07-03 12:43:17 -07:00
/*
* Private flag(s) for adbname objects.
*/
enum {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
NAME_IS_ALIAS = 1 << 31,
clean up numbering of FETCHOPT and ADDRINFO flags in the past there was overlap between the fields used as resolver fetch options and ADB addrinfo flags. this has mostly been eliminated; now we can clean up the rest of it and remove some confusing comments.
2023-07-03 12:43:17 -07:00
};
remove 'target' from dns_adb the target name parameter to dns_adb_createfind() was always passed as NULL, so we can safely remove it. relatedly, the 'target' field in the dns_adbname structure was never referenced after being set. the 'expire_target' field was used, but only as a way to check whether an ADB name represents a CNAME or DNAME, and that information can be stored as a single flag.
2025-02-21 01:24:42 -08:00
#define NAME_ALIAS(n) (((n)->flags & NAME_IS_ALIAS) != 0)
fix a few bugs, and start an AAAA db lookup/query if no A6 is found
1999-11-02 00:29:45 +00:00
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
/*
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
* Currently there are no private flags for adbentry objects.
* If we ever use them again, they'll share bit space with the
* addrinfo flags, FCTX_ADDRINFO_xxx, defined in resolver.c, so
* when defining them, they should count back from the most
* significant bit instead of counting up from zero.
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
*/
clean up macros to avoid HAVE_* names. First a6 is a fetch, not name, attribute.
1999-11-04 08:48:47 +00:00
/*
* To the name, address classes are all that really exist. If it has a
1402. [cleanup] A6 has been moved to experimental and is no longer fully supported. developer: jinmei reviewer: marka
2002-11-27 09:52:58 +00:00
* V6 address it doesn't care if it came from a AAAA query.
clean up macros to avoid HAVE_* names. First a6 is a fetch, not name, attribute.
1999-11-04 08:48:47 +00:00
*/
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
#define NAME_HAS_V4(n) (!ISC_LIST_EMPTY((n)->v4))
#define NAME_HAS_V6(n) (!ISC_LIST_EMPTY((n)->v6))
cleanup: kill_name(): names are killed by a single function. link_name(), unlink_name(): names (and bucket reference counts) are managed by these. cancel fetches at names when killing the name. if fetches are pending when killing, set a "dead" flag in the name instead and let it die when the last fetch comes in.
1999-10-26 23:44:36 +00:00
clean up macros to avoid HAVE_* names. First a6 is a fetch, not name, attribute.
1999-11-04 08:48:47 +00:00
/*
1402. [cleanup] A6 has been moved to experimental and is no longer fully supported. developer: jinmei reviewer: marka
2002-11-27 09:52:58 +00:00
* Fetches are broken out into A and AAAA types. In some cases,
clean up macros to avoid HAVE_* names. First a6 is a fetch, not name, attribute.
1999-11-04 08:48:47 +00:00
* however, it makes more sense to test for a particular class of fetches,
* like V4 or V6 above.
*/
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
#define NAME_FETCH_A(n) ((n)->fetch_a != NULL)
#define NAME_FETCH_AAAA(n) ((n)->fetch_aaaa != NULL)
incidental cleanup The NAME_FETCH_A and NAME_FETCH_AAAA macros were meant to be boolean, indicating whether the pointers were set or not, while the NAME_FETCH_V4 and NAME_FETCH_V6 macros were meant to return the pointer values. The latter were only used as booleans, so they've been removed in favor of the former. Also did some style cleanup and removed an unreachable code block.
2021-10-18 17:57:58 -07:00
#define NAME_FETCH(n) (NAME_FETCH_A(n) || NAME_FETCH_AAAA(n))
fetch change: A -> pointer, AAAA -> pointer, A6 -> list
1999-10-29 21:48:51 +00:00
clean up macros to avoid HAVE_* names. First a6 is a fetch, not name, attribute.
1999-11-04 08:48:47 +00:00
/*
* Find options and tests to see if there are addresses on the list.
*/
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
#define FIND_WANTEVENT(fn) (((fn)->options & DNS_ADBFIND_WANTEVENT) != 0)
#define FIND_WANTEMPTYEVENT(fn) (((fn)->options & DNS_ADBFIND_EMPTYEVENT) != 0)
#define FIND_AVOIDFETCHES(fn) (((fn)->options & DNS_ADBFIND_AVOIDFETCHES) != 0)
#define FIND_STARTATZONE(fn) (((fn)->options & DNS_ADBFIND_STARTATZONE) != 0)
Store static-stub addresses seperately in the adb Static-stub address and addresses from other sources where being mixed together resulting in static-stub queries going to addresses not specified in the configuration or alternatively static-stub addresses being used instead of the real addresses.
2024-08-14 23:46:44 +10:00
#define FIND_STATICSTUB(fn) (((fn)->options & DNS_ADBFIND_STATICSTUB) != 0)
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
#define FIND_NOVALIDATE(fn) (((fn)->options & DNS_ADBFIND_NOVALIDATE) != 0)
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
#define FIND_HAS_ADDRS(fn) (!ISC_LIST_EMPTY((fn)->list))
Reduce the number of fetches we make when looking up addresses If there are more that 5 NS record for a zone only perform a maximum of 4 address lookups for all the name servers. This limits the amount of remote lookup performed for server addresses at each level for a given query.
2020-02-06 17:19:10 +11:00
#define FIND_NOFETCH(fn) (((fn)->options & DNS_ADBFIND_NOFETCH) != 0)
IPv6 query flood suppression
1999-12-14 00:10:06 +00:00
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
#define ADBNAME_TYPE_MASK \
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
(DNS_ADBFIND_STARTATZONE | DNS_ADBFIND_STATICSTUB | \
DNS_ADBFIND_NOVALIDATE)
Store static-stub addresses seperately in the adb Static-stub address and addresses from other sources where being mixed together resulting in static-stub queries going to addresses not specified in the configuration or alternatively static-stub addresses being used instead of the real addresses.
2024-08-14 23:46:44 +10:00
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
#define ADBNAME_TYPE(options) ((options) & ADBNAME_TYPE_MASK)
clean up macros to avoid HAVE_* names. First a6 is a fetch, not name, attribute.
1999-11-04 08:48:47 +00:00
/*
* These are currently used on simple unsigned ints, so they are
* not really associated with any particular type.
*/
Update the source code formatting using clang-format-17
2023-10-10 16:55:13 +02:00
#define WANT_INET(x) (((x) & DNS_ADBFIND_INET) != 0)
#define WANT_INET6(x) (((x) & DNS_ADBFIND_INET6) != 0)
clean up macros to avoid HAVE_* names. First a6 is a fetch, not name, attribute.
1999-11-04 08:48:47 +00:00
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
#define EXPIRE_OK(exp, now) ((exp == INT_MAX) || (exp < now))
snapshot; start noting negative cache ttls
1999-11-04 06:46:05 +00:00
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
#define ENTER_LEVEL ISC_LOG_DEBUG(50)
#define CLEAN_LEVEL ISC_LOG_DEBUG(100)
#define DEF_LEVEL ISC_LOG_DEBUG(5)
#define NCACHE_LEVEL ISC_LOG_DEBUG(20)
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
#define NCACHE_RESULT(r) \
((r) == DNS_R_NCACHENXDOMAIN || (r) == DNS_R_NCACHENXRRSET)
#define AUTH_NX(r) ((r) == DNS_R_NXDOMAIN || (r) == DNS_R_NXRRSET)
Fix a lot of negative caching issues. I believe the code will use ncache responses from fetches and database lookups correctly now, but there needs to be a little more cleanup here and there. Bugs exist.
1999-11-24 19:23:27 +00:00
Unify the expiration time handling for all ADB expiration The algorithm from the previous commit[1] is now used to calculate all the expiration values through the code (ncache results, cname/dname targets). 1. ISC_MIN(cur, ISC_MAX(now + ADB_ENTRY_WINDOW, now + rdataset->ttl))
2024-04-03 11:51:58 +02:00
/*
* Due to the ttlclamp(), the TTL is never 0 unless the trust is ultimate,
* in which case we need to set the expiration to have immediate effect.
*/
#define ADJUSTED_EXPIRE(expire, now, ttl) \
((ttl != 0) \
? ISC_MIN(expire, ISC_MAX(now + ADB_ENTRY_WINDOW, now + ttl)) \
: INT_MAX)
add find->result_v[46]
2000-08-26 02:21:45 +00:00
/*
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* Error states.
add find->result_v[46]
2000-08-26 02:21:45 +00:00
*/
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
enum {
FIND_ERR_SUCCESS = 0,
FIND_ERR_CANCELED,
FIND_ERR_FAILURE,
FIND_ERR_NXDOMAIN,
FIND_ERR_NXRRSET,
FIND_ERR_UNEXPECTED,
FIND_ERR_NOTFOUND,
};
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
further cleanup of ADB dumps
2001-01-27 02:44:35 +00:00
static const char *errnames[] = { "success", "canceled", "failure",
update copyright notice
2008-02-07 23:46:54 +00:00
"nxdomain", "nxrrset", "unexpected",
"not_found" };
further cleanup of ADB dumps
2001-01-27 02:44:35 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
static isc_result_t find_err_map[] = {
update copyright notice
2008-02-07 23:46:54 +00:00
ISC_R_SUCCESS, ISC_R_CANCELED, ISC_R_FAILURE, DNS_R_NXDOMAIN,
DNS_R_NXRRSET, ISC_R_UNEXPECTED, ISC_R_NOTFOUND /* not YET found */
add find->result_v[46]
2000-08-26 02:21:45 +00:00
};
Fix a lot of negative caching issues. I believe the code will use ncache responses from fetches and database lookups correctly now, but there needs to be a little more cleanup here and there. Bugs exist.
1999-11-24 19:23:27 +00:00
format string bugs and improved format string checking [RT #1578]
2001-08-08 22:54:55 +00:00
static void
DP(int level, const char *format, ...) ISC_FORMAT_PRINTF(2, 3);
print v6 hooks too; use logging system; general cleanup
1999-10-29 22:28:57 +00:00
static void
Megacommit of many files. Mostly, several functions that take pointers as arguments, almost always char * pointers, had those pointers qualified with "const". Those that returned pointers to previously const-qualified arguments had their return values qualified as const. Some structure members were qualified as const to retain that attribute from the variables from which they were assigned. The macro DE_CONST is used to deal with a handful of very special places where something is qualified as const but really needs to have its const qualifier removed. rdata.c now defines macros for the prototypes of the basic rdata functions, and all of the lib/dns/rdata/**/*.c files now use them. Some minor integer-compatibility issues. (IE, ~0x03 is a signed int, so assigning it to an unsigned int should use a cast. The type of an enum member is int, so there are some conversion issues there, too.) A pointers-to-function should not be cast to a pointer-to-object. Variables should not be named for C reserved identifiers. One or two set-but-not-used variables removed. Minor other ISC style cleanups.
2000-06-01 18:26:56 +00:00
DP(int level, const char *format, ...) {
update copyright notice
2008-02-07 23:46:54 +00:00
va_list args;
print v6 hooks too; use logging system; general cleanup
1999-10-29 22:28:57 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
va_start(args, format);
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
isc_log_vwrite(DNS_LOGCATEGORY_DATABASE, DNS_LOGMODULE_ADB, level,
format, args);
update copyright notice
2008-02-07 23:46:54 +00:00
va_end(args);
print v6 hooks too; use logging system; general cleanup
1999-10-29 22:28:57 +00:00
}
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
/*%
* Increment resolver-related statistics counters.
*/
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
inc_resstats(dns_adb_t *adb, isc_statscounter_t counter) {
attach the resolver to the ADB the ADB depends on the resolver, but previously only accessed it via the view. as view->resolver may now be detached before the ADB finishes, a shutdown race was possible. attaching to the resolver directly prevents this.
2022-05-12 15:51:10 -07:00
if (adb->res != NULL) {
dns_resolver_incstats(adb->res, counter);
remove resolver whenshutdown events weakly attaching and detaching when creating and destroying the resolver obviates the need to have a callback event to do the weak detach. remove the dns_resolver_whenshutdown() mechanism, as it is now unused.
2022-05-11 13:55:01 -07:00
}
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
}
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
/*%
* Set adb-related statistics counters.
*/
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Replace custom isc_u?intNN_t types with C99 u?intNN_t types
2018-03-28 14:19:37 +02:00
set_adbstat(dns_adb_t *adb, uint64_t val, isc_statscounter_t counter) {
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
if (adb->stats != NULL) {
isc_stats_set(adb->stats, val, counter);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
dec_adbstats(dns_adb_t *adb, isc_statscounter_t counter) {
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
if (adb->stats != NULL) {
isc_stats_decrement(adb->stats, counter);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
inc_adbstats(dns_adb_t *adb, isc_statscounter_t counter) {
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
if (adb->stats != NULL) {
isc_stats_increment(adb->stats, counter);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_ttl_t
clamp ttls in the adb to be 10 < ttl < 86400, that is, 10 seconds minimum and 24 hours max
2000-06-07 20:15:48 +00:00
ttlclamp(dns_ttl_t ttl) {
update copyright notice
2008-02-07 23:46:54 +00:00
if (ttl < ADB_CACHE_MINIMUM) {
ttl = ADB_CACHE_MINIMUM;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
if (ttl > ADB_CACHE_MAXIMUM) {
ttl = ADB_CACHE_MAXIMUM;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
clamp ttls in the adb to be 10 < ttl < 86400, that is, 10 seconds minimum and 24 hours max
2000-06-07 20:15:48 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
return ttl;
clamp ttls in the adb to be 10 < ttl < 86400, that is, 10 seconds minimum and 24 hours max
2000-06-07 20:15:48 +00:00
}
adb does fetches for A records now. Has a memory leak still.
1999-10-27 22:24:40 +00:00
/*
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
* Requires the name to be locked and that no entries to be locked.
simplify importing rdatasets. One function handles A and AAAA types.
1999-10-29 19:42:06 +00:00
*
* This code handles A and AAAA rdatasets only.
adb does fetches for A records now. Has a memory leak still.
1999-10-27 22:24:40 +00:00
*/
import_rdataset() can't fail the import_rdataset() function can't return any value other than ISC_R_SUCCESS, so it's been changed to void and its callers don't rely on its return value any longer.
2025-03-21 23:48:01 -07:00
static void
simplify importing rdatasets. One function handles A and AAAA types.
1999-10-29 19:42:06 +00:00
import_rdataset(dns_adbname_t *adbname, dns_rdataset_t *rdataset,
update copyright notice
2008-02-07 23:46:54 +00:00
isc_stdtime_t now) {
incidental cleanup The NAME_FETCH_A and NAME_FETCH_AAAA macros were meant to be boolean, indicating whether the pointers were set or not, while the NAME_FETCH_V4 and NAME_FETCH_V6 macros were meant to return the pointer values. The latter were only used as booleans, so they've been removed in favor of the former. Also did some style cleanup and removed an unreachable code block.
2021-10-18 17:57:58 -07:00
dns_adb_t *adb = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
dns_rdatatype_t rdtype;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(DNS_ADBNAME_VALID(adbname));
update copyright notice
2008-02-07 23:46:54 +00:00
adb = adbname->adb;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(DNS_ADB_VALID(adb));
update copyright notice
2008-02-07 23:46:54 +00:00
rdtype = rdataset->type;
Fix the expire_v4 and expire_v6 logic Correct the logic to set the expiration period of expire_{v4,v6} as follows: 1. If the trust is ultimate (local entry), immediately set the entry as expired, so the changes to the local zones have immediate effect. 3. If the expiration is already set and smaller than the new value, then leave the expiration value as it is. 2. Otherwise pick larger of `now + ADB_ENTRY_WINDOW` and `now + TTL` as the new expiration value.
2024-04-03 11:36:54 +02:00
switch (rdataset->trust) {
case dns_trust_glue:
case dns_trust_additional:
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
case dns_trust_pending_answer:
case dns_trust_pending_additional:
Fix the expire_v4 and expire_v6 logic Correct the logic to set the expiration period of expire_{v4,v6} as follows: 1. If the trust is ultimate (local entry), immediately set the entry as expired, so the changes to the local zones have immediate effect. 3. If the expiration is already set and smaller than the new value, then leave the expiration value as it is. 2. Otherwise pick larger of `now + ADB_ENTRY_WINDOW` and `now + TTL` as the new expiration value.
2024-04-03 11:36:54 +02:00
rdataset->ttl = ADB_CACHE_MINIMUM;
break;
case dns_trust_ultimate:
rdataset->ttl = 0;
break;
default:
rdataset->ttl = ttlclamp(rdataset->ttl);
}
use new dns_rdatatype classification functions modify code to use dns_rdatatype_ismulti(), dns_rdatatype_issig(), dns_rdatatype_isaddr(), and dns_rdatatype_isalias() where applicable.
2025-03-04 15:51:49 -08:00
REQUIRE(dns_rdatatype_isaddr(rdtype));
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
add DNS_RDATASET_FOREACH macro replace the pattern `for (result = dns_rdataset_first(x); result == ISC_R_SUCCES; result = dns_rdataset_next(x)` with a new `DNS_RDATASET_FOREACH` macro throughout BIND.
2025-03-21 23:32:27 -07:00
DNS_RDATASET_FOREACH(rdataset) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
/* FIXME: Move to a separate function */
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
dns_adbnamehooklist_t *hookhead = NULL;
dns_adbentry_t *entry = NULL;
dns_rdata_t rdata = DNS_RDATA_INIT;
isc_sockaddr_t sockaddr;
struct in_addr ina;
struct in6_addr in6a;
update copyright notice
2008-02-07 23:46:54 +00:00
dns_rdataset_current(rdataset, &rdata);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
switch (rdtype) {
case dns_rdatatype_a:
update copyright notice
2008-02-07 23:46:54 +00:00
INSIST(rdata.length == 4);
[master] replace memcpy() with memmove(). 3698. [cleanup] Replaced all uses of memcpy() with memmove(). [RT #35120]
2014-01-08 16:27:10 -08:00
memmove(&ina.s_addr, rdata.data, 4);
update copyright notice
2008-02-07 23:46:54 +00:00
isc_sockaddr_fromin(&sockaddr, &ina, 0);
2343. [bug] (Seemingly) duplicate IPv6 entries could be created in ADB. [RT #17837] (plan to merge to 9.[345])
2008-03-20 19:23:42 +00:00
hookhead = &adbname->v4;
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
break;
case dns_rdatatype_aaaa:
update copyright notice
2008-02-07 23:46:54 +00:00
INSIST(rdata.length == 16);
[master] replace memcpy() with memmove(). 3698. [cleanup] Replaced all uses of memcpy() with memmove(). [RT #35120]
2014-01-08 16:27:10 -08:00
memmove(in6a.s6_addr, rdata.data, 16);
update copyright notice
2008-02-07 23:46:54 +00:00
isc_sockaddr_fromin6(&sockaddr, &in6a, 0);
2343. [bug] (Seemingly) duplicate IPv6 entries could be created in ADB. [RT #17837] (plan to merge to 9.[345])
2008-03-20 19:23:42 +00:00
hookhead = &adbname->v6;
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
break;
default:
UNREACHABLE();
update copyright notice
2008-02-07 23:46:54 +00:00
}
Lock the adbname and adbentry prior to unlocking hash locks There was a datarace that could expire a freshly created ADB names and entries between the return from get_attached_{name,entry} and locking it again. Lock the ADB name and ADB entry inside the hash table lock, so they won't get expired until the full initialization has been complete.
2022-12-13 13:48:55 +01:00
entry = get_attached_and_locked_entry(adb, now, &sockaddr);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
switch to ISC_LIST_FOREACH everywhere the pattern `for (x = ISC_LIST_HEAD(...); x != NULL; ISC_LIST_NEXT(...)` has been changed to `ISC_LIST_FOREACH` throughout BIND, except in a few cases where the change would be excessively complex. in most cases this was a straightforward change. in some places, however, the list element variable was referenced after the loop ended, and the code was refactored to avoid this necessity. also, because `ISC_LIST_FOREACH` uses typeof(list.head) to declare the list elements, compilation failures can occur if the list object has a `const` qualifier. some `const` qualifiers have been removed from function parameters to avoid this problem, and where that was not possible, `UNCONST` was used.
2025-03-20 22:25:56 -07:00
bool found = false;
ISC_LIST_FOREACH(*hookhead, anh, name_link) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (anh->entry == entry) {
switch to ISC_LIST_FOREACH everywhere the pattern `for (x = ISC_LIST_HEAD(...); x != NULL; ISC_LIST_NEXT(...)` has been changed to `ISC_LIST_FOREACH` throughout BIND, except in a few cases where the change would be excessively complex. in most cases this was a straightforward change. in some places, however, the list element variable was referenced after the loop ended, and the code was refactored to avoid this necessity. also, because `ISC_LIST_FOREACH` uses typeof(list.head) to declare the list elements, compilation failures can occur if the list object has a `const` qualifier. some `const` qualifiers have been removed from function parameters to avoid this problem, and where that was not possible, `UNCONST` was used.
2025-03-20 22:25:56 -07:00
found = true;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
}
switch to ISC_LIST_FOREACH everywhere the pattern `for (x = ISC_LIST_HEAD(...); x != NULL; ISC_LIST_NEXT(...)` has been changed to `ISC_LIST_FOREACH` throughout BIND, except in a few cases where the change would be excessively complex. in most cases this was a straightforward change. in some places, however, the list element variable was referenced after the loop ended, and the code was refactored to avoid this necessity. also, because `ISC_LIST_FOREACH` uses typeof(list.head) to declare the list elements, compilation failures can occur if the list object has a `const` qualifier. some `const` qualifiers have been removed from function parameters to avoid this problem, and where that was not possible, `UNCONST` was used.
2025-03-20 22:25:56 -07:00
if (!found) {
dns_adbnamehook_t *nh = new_adbnamehook(adb);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbentry_attach(entry, &nh->entry);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
ISC_LIST_APPEND(*hookhead, nh, name_link);
ISC_LIST_APPEND(entry->nhs, nh, entry_link);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
UNLOCK(&entry->lock);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbentry_detach(&entry);
update copyright notice
2008-02-07 23:46:54 +00:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
switch (rdtype) {
case dns_rdatatype_a:
Unify the expiration time handling for all ADB expiration The algorithm from the previous commit[1] is now used to calculate all the expiration values through the code (ncache results, cname/dname targets). 1. ISC_MIN(cur, ISC_MAX(now + ADB_ENTRY_WINDOW, now + rdataset->ttl))
2024-04-03 11:51:58 +02:00
adbname->expire_v4 = ADJUSTED_EXPIRE(adbname->expire_v4, now,
rdataset->ttl);
Fix the expire_v4 and expire_v6 logic Correct the logic to set the expiration period of expire_{v4,v6} as follows: 1. If the trust is ultimate (local entry), immediately set the entry as expired, so the changes to the local zones have immediate effect. 3. If the expiration is already set and smaller than the new value, then leave the expiration value as it is. 2. Otherwise pick larger of `now + ADB_ENTRY_WINDOW` and `now + TTL` as the new expiration value.
2024-04-03 11:36:54 +02:00
DP(NCACHE_LEVEL, "expire_v4 set to %u import_rdataset",
adbname->expire_v4);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
break;
case dns_rdatatype_aaaa:
Unify the expiration time handling for all ADB expiration The algorithm from the previous commit[1] is now used to calculate all the expiration values through the code (ncache results, cname/dname targets). 1. ISC_MIN(cur, ISC_MAX(now + ADB_ENTRY_WINDOW, now + rdataset->ttl))
2024-04-03 11:51:58 +02:00
adbname->expire_v6 = ADJUSTED_EXPIRE(adbname->expire_v6, now,
rdataset->ttl);
Fix the expire_v4 and expire_v6 logic Correct the logic to set the expiration period of expire_{v4,v6} as follows: 1. If the trust is ultimate (local entry), immediately set the entry as expired, so the changes to the local zones have immediate effect. 3. If the expiration is already set and smaller than the new value, then leave the expiration value as it is. 2. Otherwise pick larger of `now + ADB_ENTRY_WINDOW` and `now + TTL` as the new expiration value.
2024-04-03 11:36:54 +02:00
DP(NCACHE_LEVEL, "expire_v6 set to %u import_rdataset",
adbname->expire_v6);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
break;
default:
UNREACHABLE();
update copyright notice
2008-02-07 23:46:54 +00:00
}
adb does fetches for A records now. Has a memory leak still.
1999-10-27 22:24:40 +00:00
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
static void
expire_name_async(void *arg) {
dns_adbname_t *adbname = arg;
dns_adb_t *adb = adbname->adb;
RUNTIME_CHECK(adbname->loop == isc_loop());
/* ... and LRU list */
ISC_SIEVE_UNLINK(adb->lru[isc_tid()].names, adbname, link);
dns_adbname_detach(&adbname);
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
}
cleanup: kill_name(): names are killed by a single function. link_name(), unlink_name(): names (and bucket reference counts) are managed by these. cancel fetches at names when killing the name. if fetches are pending when killing, set a "dead" flag in the name instead and let it die when the last fetch comes in.
1999-10-26 23:44:36 +00:00
/*
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
* Requires the name to be locked and write lock on adb->names_lock.
cleanup: kill_name(): names are killed by a single function. link_name(), unlink_name(): names (and bucket reference counts) are managed by these. cancel fetches at names when killing the name. if fetches are pending when killing, set a "dead" flag in the name instead and let it die when the last fetch comes in.
1999-10-26 23:44:36 +00:00
*/
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
static void
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
expire_name(dns_adbname_t *adbname, dns_adbstatus_t astat) {
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
REQUIRE(DNS_ADBNAME_VALID(adbname));
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adb_t *adb = adbname->adb;
update copyright notice
2008-02-07 23:46:54 +00:00
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
REQUIRE(DNS_ADB_VALID(adb));
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
DP(DEF_LEVEL, "killing name %p", adbname);
update copyright notice
2008-02-07 23:46:54 +00:00
/*
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* Clean up the name's various contents. These functions
* are destructive in that they will always empty the lists
* of finds and namehooks.
update copyright notice
2008-02-07 23:46:54 +00:00
*/
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
clean_finds_at_name(adbname, astat, DNS_ADBFIND_ADDRESSMASK);
Don't expire an ADB entry without holding the entries lock The clean_namehooks() function does't hold the 'adb->entries_lock' lock, so calling maybe_expire_entry() is not thread-safe. Instead of adding a lock/unlock, leave the expiration to later, e.g. by the get_attached_and_locked_entry() function. Also fix a couple of comment typos.
2022-12-23 14:21:03 +00:00
clean_namehooks(adb, &adbname->v4);
clean_namehooks(adb, &adbname->v6);
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
if (NAME_FETCH_A(adbname)) {
dns_resolver_cancelfetch(adbname->fetch_a->fetch);
}
if (NAME_FETCH_AAAA(adbname)) {
dns_resolver_cancelfetch(adbname->fetch_aaaa->fetch);
update copyright notice
2008-02-07 23:46:54 +00:00
}
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
/* Remove the adbname from the hashtable... */
if (cds_lfht_del(adb->names_ht, &adbname->ht_node) == 0) {
isc_async_run(adbname->loop, expire_name_async, adbname);
}
cleanup: kill_name(): names are killed by a single function. link_name(), unlink_name(): names (and bucket reference counts) are managed by these. cancel fetches at names when killing the name. if fetches are pending when killing, set a "dead" flag in the name instead and let it die when the last fetch comes in.
1999-10-26 23:44:36 +00:00
}
more v4/v6 splitting out
1999-10-29 01:26:45 +00:00
/*
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
* Requires the name to be locked and no entries to be locked.
more v4/v6 splitting out
1999-10-29 01:26:45 +00:00
*/
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
static void
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
maybe_expire_namehooks(dns_adbname_t *adbname, isc_stdtime_t now) {
REQUIRE(DNS_ADBNAME_VALID(adbname));
REQUIRE(DNS_ADB_VALID(adbname->adb));
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adb_t *adb = adbname->adb;
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Check to see if we need to remove the v4 addresses
*/
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (!NAME_FETCH_A(adbname) && EXPIRE_OK(adbname->expire_v4, now)) {
if (NAME_HAS_V4(adbname)) {
DP(DEF_LEVEL, "expiring v4 for name %p", adbname);
Don't expire an ADB entry without holding the entries lock The clean_namehooks() function does't hold the 'adb->entries_lock' lock, so calling maybe_expire_entry() is not thread-safe. Instead of adding a lock/unlock, leave the expiration to later, e.g. by the get_attached_and_locked_entry() function. Also fix a couple of comment typos.
2022-12-23 14:21:03 +00:00
clean_namehooks(adb, &adbname->v4);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
adbname->partial_result &= ~DNS_ADBFIND_INET;
update copyright notice
2008-02-07 23:46:54 +00:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
adbname->expire_v4 = INT_MAX;
adbname->fetch_err = FIND_ERR_UNEXPECTED;
update copyright notice
2008-02-07 23:46:54 +00:00
}
/*
* Check to see if we need to remove the v6 addresses
*/
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (!NAME_FETCH_AAAA(adbname) && EXPIRE_OK(adbname->expire_v6, now)) {
if (NAME_HAS_V6(adbname)) {
DP(DEF_LEVEL, "expiring v6 for name %p", adbname);
Don't expire an ADB entry without holding the entries lock The clean_namehooks() function does't hold the 'adb->entries_lock' lock, so calling maybe_expire_entry() is not thread-safe. Instead of adding a lock/unlock, leave the expiration to later, e.g. by the get_attached_and_locked_entry() function. Also fix a couple of comment typos.
2022-12-23 14:21:03 +00:00
clean_namehooks(adb, &adbname->v6);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
adbname->partial_result &= ~DNS_ADBFIND_INET6;
update copyright notice
2008-02-07 23:46:54 +00:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
adbname->expire_v6 = INT_MAX;
adbname->fetch6_err = FIND_ERR_UNEXPECTED;
update copyright notice
2008-02-07 23:46:54 +00:00
}
more v4/v6 splitting out
1999-10-29 01:26:45 +00:00
}
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
static void
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
shutdown_names(dns_adb_t *adb) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbname_t *adbname = NULL;
struct cds_lfht_iter iter;
cds_lfht_for_each_entry(adb->names_ht, &iter, adbname, ht_node) {
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
dns_adbname_ref(adbname);
LOCK(&adbname->lock);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
/*
* Run through the list. For each name, clean up finds
* found there, and cancel any fetches running. When
* all the fetches are canceled, the name will destroy
* itself.
*/
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
expire_name(adbname, DNS_ADB_SHUTTINGDOWN);
UNLOCK(&adbname->lock);
dns_adbname_detach(&adbname);
update copyright notice
2008-02-07 23:46:54 +00:00
}
add dns_adb_findaddrinfo() and dns_adb_freeaddrinfo()
2000-01-12 03:00:33 +00:00
}
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
static void
shutdown_entries(dns_adb_t *adb) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbentry_t *adbentry = NULL;
struct cds_lfht_iter iter;
cds_lfht_for_each_entry(adb->entries_ht, &iter, adbentry, ht_node) {
dns_adbentry_ref(adbentry);
LOCK(&adbentry->lock);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
expire_entry(adbentry);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
UNLOCK(&adbentry->lock);
dns_adbentry_detach(&adbentry);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
}
}
cleanup: kill_name(): names are killed by a single function. link_name(), unlink_name(): names (and bucket reference counts) are managed by these. cancel fetches at names when killing the name. if fetches are pending when killing, set a "dead" flag in the name instead and let it die when the last fetch comes in.
1999-10-26 23:44:36 +00:00
/*
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
* The name containing the 'namehooks' list must be locked.
cleanup: kill_name(): names are killed by a single function. link_name(), unlink_name(): names (and bucket reference counts) are managed by these. cancel fetches at names when killing the name. if fetches are pending when killing, set a "dead" flag in the name instead and let it die when the last fetch comes in.
1999-10-26 23:44:36 +00:00
*/
static void
Don't expire an ADB entry without holding the entries lock The clean_namehooks() function does't hold the 'adb->entries_lock' lock, so calling maybe_expire_entry() is not thread-safe. Instead of adding a lock/unlock, leave the expiration to later, e.g. by the get_attached_and_locked_entry() function. Also fix a couple of comment typos.
2022-12-23 14:21:03 +00:00
clean_namehooks(dns_adb_t *adb, dns_adbnamehooklist_t *namehooks) {
make all ISC_LIST_FOREACH calls safe previously, ISC_LIST_FOREACH and ISC_LIST_FOREACH_SAFE were two separate macros, with the _SAFE version allowing entries to be unlinked during the loop. ISC_LIST_FOREACH is now also safe, and the separate _SAFE macro has been removed. similarly, the ISC_LIST_FOREACH_REV macro is now safe, and ISC_LIST_FOREACH_REV_SAFE has also been removed.
2025-05-23 13:02:22 -07:00
ISC_LIST_FOREACH(*namehooks, namehook, name_link) {
update copyright notice
2008-02-07 23:46:54 +00:00
INSIST(DNS_ADBNAMEHOOK_VALID(namehook));
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
INSIST(DNS_ADBENTRY_VALID(namehook->entry));
update copyright notice
2008-02-07 23:46:54 +00:00
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
dns_adbentry_t *adbentry = namehook->entry;
namehook->entry = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Free the namehook
*/
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
ISC_LIST_UNLINK(*namehooks, namehook, name_link);
LOCK(&adbentry->lock);
ISC_LIST_UNLINK(adbentry->nhs, namehook, entry_link);
UNLOCK(&adbentry->lock);
dns_adbentry_detach(&adbentry);
update copyright notice
2008-02-07 23:46:54 +00:00
free_adbnamehook(adb, &namehook);
}
snapshot ; external and internal reference counts are separated out, and everything but fetches work.
1999-10-19 01:37:22 +00:00
}
snapshot
1999-10-20 23:26:06 +00:00
/*
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
* The name must be locked.
snapshot
1999-10-20 23:26:06 +00:00
*/
snapshot ; external and internal reference counts are separated out, and everything but fetches work.
1999-10-19 01:37:22 +00:00
static void
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
clean_finds_at_name(dns_adbname_t *name, dns_adbstatus_t astat,
[master] refactor max-recursion-queries - the counters weren't set correctly when fetches timed out. instead we now pass down a counter object.
2014-11-19 18:21:02 -08:00
unsigned int addrs) {
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
dns_adbfind_t *find = NULL, *next = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
DP(ENTER_LEVEL,
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
"ENTER clean_finds_at_name, name %p, astat %08x, addrs %08x", name,
astat, addrs);
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
for (find = ISC_LIST_HEAD(name->finds); find != NULL; find = next) {
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
bool process = false;
unsigned int wanted, notify;
update copyright notice
2008-02-07 23:46:54 +00:00
LOCK(&find->lock);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
next = ISC_LIST_NEXT(find, plink);
update copyright notice
2008-02-07 23:46:54 +00:00
wanted = find->flags & DNS_ADBFIND_ADDRESSMASK;
notify = wanted & addrs;
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
switch (astat) {
case DNS_ADB_MOREADDRESSES:
DP(ISC_LOG_DEBUG(3), "more addresses");
update copyright notice
2008-02-07 23:46:54 +00:00
if ((notify) != 0) {
find->flags &= ~addrs;
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
process = true;
update copyright notice
2008-02-07 23:46:54 +00:00
}
break;
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
case DNS_ADB_NOMOREADDRESSES:
DP(ISC_LOG_DEBUG(3), "no more addresses");
update copyright notice
2008-02-07 23:46:54 +00:00
find->flags &= ~addrs;
wanted = find->flags & DNS_ADBFIND_ADDRESSMASK;
if (wanted == 0) {
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
process = true;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
break;
default:
find->flags &= ~addrs;
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
process = true;
update copyright notice
2008-02-07 23:46:54 +00:00
}
if (process) {
DP(DEF_LEVEL, "cfan: processing find %p", find);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Unlink the find from the name, letting the caller
* call dns_adb_destroyfind() on it to clean it up
* later.
*/
ISC_LIST_UNLINK(name->finds, find, plink);
find->adbname = NULL;
INSIST(!FIND_EVENTSENT(find));
Use atomics to access find->status
2024-07-16 13:36:54 +10:00
atomic_store(&find->status, astat);
update copyright notice
2008-02-07 23:46:54 +00:00
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
DP(DEF_LEVEL, "cfan: sending find %p to caller", find);
update copyright notice
2008-02-07 23:46:54 +00:00
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
isc_async_run(find->loop, find->cb, find);
Fix a crash due to use-after-free (#38495)
2015-03-18 06:42:54 +05:30
find->flags |= FIND_EVENT_SENT;
update copyright notice
2008-02-07 23:46:54 +00:00
} else {
DP(DEF_LEVEL, "cfan: skipping find %p", find);
}
UNLOCK(&find->lock);
}
DP(ENTER_LEVEL, "EXIT clean_finds_at_name, name %p", name);
snapshot ; external and internal reference counts are separated out, and everything but fetches work.
1999-10-19 01:37:22 +00:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbname_t *
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
new_adbname(dns_adb_t *adb, const dns_name_t *dnsname, unsigned int type) {
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adbname_t *name = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
name = isc_mem_get(adb->mctx, sizeof(*name));
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
*name = (dns_adbname_t){
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
.adb = dns_adb_ref(adb),
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
.expire_v4 = INT_MAX,
.expire_v6 = INT_MAX,
.fetch_err = FIND_ERR_UNEXPECTED,
.fetch6_err = FIND_ERR_UNEXPECTED,
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
.v4 = ISC_LIST_INITIALIZER,
.v6 = ISC_LIST_INITIALIZER,
.finds = ISC_LIST_INITIALIZER,
.link = ISC_LINK_INITIALIZER,
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
.type = type,
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
.loop = isc_loop_ref(isc_loop()),
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
.magic = DNS_ADBNAME_MAGIC,
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
};
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#if DNS_ADB_TRACE
fprintf(stderr, "dns_adbname__init:%s:%s:%d:%p->references = 1\n",
__func__, __FILE__, __LINE__ + 1, name);
#endif
isc_refcount_init(&name->references, 1);
isc_mutex_init(&name->lock);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
name->name = dns_fixedname_initname(&name->fname);
dns_name_copy(dnsname, name->name);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
inc_adbstats(adb, dns_adbstats_namescnt);
update copyright notice
2008-02-07 23:46:54 +00:00
return name;
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#if DNS_ADB_TRACE
ISC_REFCOUNT_TRACE_IMPL(dns_adbname, destroy_adbname);
#else
ISC_REFCOUNT_IMPL(dns_adbname, destroy_adbname);
#endif
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
static void
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
destroy_adbname_rcu(struct rcu_head *rcu_head) {
dns_adbname_t *adbname = caa_container_of(rcu_head, dns_adbname_t,
rcu_head);
REQUIRE(DNS_ADBNAME_VALID(adbname));
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adb_t *adb = adbname->adb;
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
REQUIRE(!NAME_HAS_V4(adbname));
REQUIRE(!NAME_HAS_V6(adbname));
REQUIRE(!NAME_FETCH(adbname));
REQUIRE(ISC_LIST_EMPTY(adbname->finds));
REQUIRE(!ISC_LINK_LINKED(adbname, link));
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
adbname->magic = 0;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
isc_mutex_destroy(&adbname->lock);
isc_loop_detach(&adbname->loop);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
isc_mem_put(adb->mctx, adbname, sizeof(*adbname));
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dec_adbstats(adb, dns_adbstats_namescnt);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adb_detach(&adb);
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
static void
destroy_adbname(dns_adbname_t *adbname) {
call_rcu(&adbname->rcu_head, destroy_adbname_rcu);
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbnamehook_t *
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
new_adbnamehook(dns_adb_t *adb) {
dns_adbnamehook_t *nh = isc_mem_get(adb->mctx, sizeof(*nh));
*nh = (dns_adbnamehook_t){
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
.name_link = ISC_LINK_INITIALIZER,
.entry_link = ISC_LINK_INITIALIZER,
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
.magic = DNS_ADBNAMEHOOK_MAGIC,
};
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
return nh;
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
free_adbnamehook(dns_adb_t *adb, dns_adbnamehook_t **namehook) {
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adbnamehook_t *nh = NULL;
REQUIRE(namehook != NULL && DNS_ADBNAMEHOOK_VALID(*namehook));
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
nh = *namehook;
*namehook = NULL;
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(nh->entry == NULL);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
REQUIRE(!ISC_LINK_LINKED(nh, name_link));
REQUIRE(!ISC_LINK_LINKED(nh, entry_link));
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
nh->magic = 0;
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
isc_mem_put(adb->mctx, nh, sizeof(*nh));
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbentry_t *
Always set ADB entry expiration to now + ADB_ENTRY_WINDOW When ADB entry was created it was set to never expire. If we never called any of the functions that adjust the expiration, it could linger in the ADB forever. Set the expiration (.expires) to now + ADB_ENTRY_WINDOW when creating the new ADB entry to ensure the ADB entry will always expire.
2024-04-03 11:33:48 +02:00
new_adbentry(dns_adb_t *adb, const isc_sockaddr_t *addr, isc_stdtime_t now) {
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
dns_adbentry_t *entry = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
entry = isc_mem_get(adb->mctx, sizeof(*entry));
*entry = (dns_adbentry_t){
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
.srtt = isc_random_uniform(0x1f) + 1,
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
.sockaddr = *addr,
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
.link = ISC_LINK_INITIALIZER,
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
.quota = adb->quota,
.references = ISC_REFCOUNT_INITIALIZER(1),
.adb = dns_adb_ref(adb),
Always set ADB entry expiration to now + ADB_ENTRY_WINDOW When ADB entry was created it was set to never expire. If we never called any of the functions that adjust the expiration, it could linger in the ADB forever. Set the expiration (.expires) to now + ADB_ENTRY_WINDOW when creating the new ADB entry to ensure the ADB entry will always expire.
2024-04-03 11:33:48 +02:00
.expires = now + ADB_ENTRY_WINDOW,
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
.loop = isc_loop_ref(isc_loop()),
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
.magic = DNS_ADBENTRY_MAGIC,
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
};
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#if DNS_ADB_TRACE
fprintf(stderr, "dns_adbentry__init:%s:%s:%d:%p->references = 1\n",
__func__, __FILE__, __LINE__ + 1, entry);
#endif
isc_mutex_init(&entry->lock);
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
inc_adbstats(adb, dns_adbstats_entriescnt);
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
return entry;
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
destroy_adbentry_rcu(struct rcu_head *rcu_head) {
dns_adbentry_t *adbentry = caa_container_of(rcu_head, dns_adbentry_t,
rcu_head);
snapshot: dns_adb_done() partial implementation, handles mostly work, but no fetches are started yet.
1999-10-14 00:50:00 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
REQUIRE(DNS_ADBENTRY_VALID(adbentry));
dns_adb_t *adb = adbentry->adb;
INSIST that active quota is 0 in destroy_adbentry() This should catch ADB quota management errors in the resolver.
2022-12-22 13:43:47 +00:00
uint_fast32_t active;
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
adbentry->magic = 0;
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
INSIST(!ISC_LINK_LINKED(adbentry, link));
Don't remove ADB entry from LRU before trying to expire it There was a code flow error that would remove the expired ADB entry from the LRU list and then a check in the expire_entry() would cause assertion error because it expect the ADB entry to be linked. Additionally, the expire mechanism would loop for cases when we would held only a read rwlock; in such case we need to upgrade the lock and try again, not just try again.
2023-02-16 12:26:01 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
INSIST(ISC_LIST_EMPTY(adbentry->nhs));
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
active = atomic_load_acquire(&adbentry->active);
INSIST that active quota is 0 in destroy_adbentry() This should catch ADB quota management errors in the resolver.
2022-12-22 13:43:47 +00:00
INSIST(active == 0);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
if (adbentry->cookie != NULL) {
isc_mem_put(adb->mctx, adbentry->cookie, adbentry->cookielen);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
isc_mutex_destroy(&adbentry->lock);
isc_loop_detach(&adbentry->loop);
isc_mem_put(adb->mctx, adbentry, sizeof(*adbentry));
Use isc_hashmap instead of isc_ht in the dns_adb API Replace the use of isc_ht API with isc_hashmap API in the dns_adb database implementation. This requires extending the dns_adbnamebucket_t and dns_adbentrybucket_t structures to include keysize and copy of the key because the isc_hashmap API needs the raw key in case of resizing the hashmap table.
2022-06-27 12:46:06 +02:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dec_adbstats(adb, dns_adbstats_entriescnt);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adb_detach(&adb);
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
static void
destroy_adbentry(dns_adbentry_t *adbentry) {
call_rcu(&adbentry->rcu_head, destroy_adbentry_rcu);
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#if DNS_ADB_TRACE
ISC_REFCOUNT_TRACE_IMPL(dns_adbentry, destroy_adbentry);
#else
ISC_REFCOUNT_IMPL(dns_adbentry, destroy_adbentry);
#endif
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbfind_t *
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
new_adbfind(dns_adb_t *adb, in_port_t port) {
dns_adbfind_t *find = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
Remove memory limit on ADB finds and fetches Address Database (ADB) shares the memory for the short lived ADB objects (finds, fetches, addrinfo) and the long lived ADB objects (names, entries, namehooks). This could lead to a situation where the resolver-heavy load would force evict ADB objects from the database to point where ADB is completely empty, leading to even more resolver-heavy load. Make the short lived ADB objects use the other memory context that we already created for the hashmaps. This makes the ADB overmem condition to not be triggered by the ongoing resolver fetches.
2025-01-15 10:36:33 +01:00
find = isc_mem_get(adb->hmctx, sizeof(*find));
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
*find = (dns_adbfind_t){
.port = port,
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
.result_v4 = ISC_R_UNEXPECTED,
.result_v6 = ISC_R_UNEXPECTED,
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
.publink = ISC_LINK_INITIALIZER,
.plink = ISC_LINK_INITIALIZER,
.list = ISC_LIST_INITIALIZER,
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
};
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
dns_adb_attach(adb, &find->adb);
isc_mutex_init(&find->lock);
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
find->magic = DNS_ADBFIND_MAGIC;
start on fetch infrastructure
1999-10-25 21:02:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
return find;
start on fetch infrastructure
1999-10-25 21:02:54 +00:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
free_adbfind(dns_adbfind_t **findp) {
dns_adb_t *adb = NULL;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adbfind_t *find = NULL;
REQUIRE(findp != NULL && DNS_ADBFIND_VALID(*findp));
snapshot
1999-10-08 23:12:21 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
find = *findp;
*findp = NULL;
snapshot
1999-10-08 23:12:21 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
adb = find->adb;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(!FIND_HAS_ADDRS(find));
REQUIRE(!ISC_LINK_LINKED(find, publink));
REQUIRE(!ISC_LINK_LINKED(find, plink));
REQUIRE(find->adbname == NULL);
snapshot: dns_adb_done() partial implementation, handles mostly work, but no fetches are started yet.
1999-10-14 00:50:00 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
find->magic = 0;
snapshot
1999-10-08 23:12:21 +00:00
- Make isc_mutex_destroy return void - Make isc_mutexblock_init/destroy return void - Minor cleanups
2018-11-19 10:31:09 +00:00
isc_mutex_destroy(&find->lock);
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
Remove memory limit on ADB finds and fetches Address Database (ADB) shares the memory for the short lived ADB objects (finds, fetches, addrinfo) and the long lived ADB objects (names, entries, namehooks). This could lead to a situation where the resolver-heavy load would force evict ADB objects from the database to point where ADB is completely empty, leading to even more resolver-heavy load. Make the short lived ADB objects use the other memory context that we already created for the hashmaps. This makes the ADB overmem condition to not be triggered by the ongoing resolver fetches.
2025-01-15 10:36:33 +01:00
isc_mem_put(adb->hmctx, find, sizeof(*find));
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
dns_adb_detach(&adb);
}
static dns_adbfetch_t *
new_adbfetch(dns_adb_t *adb) {
dns_adbfetch_t *fetch = NULL;
Remove memory limit on ADB finds and fetches Address Database (ADB) shares the memory for the short lived ADB objects (finds, fetches, addrinfo) and the long lived ADB objects (names, entries, namehooks). This could lead to a situation where the resolver-heavy load would force evict ADB objects from the database to point where ADB is completely empty, leading to even more resolver-heavy load. Make the short lived ADB objects use the other memory context that we already created for the hashmaps. This makes the ADB overmem condition to not be triggered by the ongoing resolver fetches.
2025-01-15 10:36:33 +01:00
fetch = isc_mem_get(adb->hmctx, sizeof(*fetch));
Small cleanup in dns_adb unit
2025-03-17 15:23:00 +01:00
*fetch = (dns_adbfetch_t){
.magic = DNS_ADBFETCH_MAGIC,
};
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
dns_rdataset_init(&fetch->rdataset);
return fetch;
}
static void
free_adbfetch(dns_adb_t *adb, dns_adbfetch_t **fetchp) {
dns_adbfetch_t *fetch = NULL;
REQUIRE(fetchp != NULL && DNS_ADBFETCH_VALID(*fetchp));
fetch = *fetchp;
*fetchp = NULL;
fetch->magic = 0;
if (dns_rdataset_isassociated(&fetch->rdataset)) {
dns_rdataset_disassociate(&fetch->rdataset);
}
Remove memory limit on ADB finds and fetches Address Database (ADB) shares the memory for the short lived ADB objects (finds, fetches, addrinfo) and the long lived ADB objects (names, entries, namehooks). This could lead to a situation where the resolver-heavy load would force evict ADB objects from the database to point where ADB is completely empty, leading to even more resolver-heavy load. Make the short lived ADB objects use the other memory context that we already created for the hashmaps. This makes the ADB overmem condition to not be triggered by the ongoing resolver fetches.
2025-01-15 10:36:33 +01:00
isc_mem_put(adb->hmctx, fetch, sizeof(*fetch));
snapshot
1999-10-08 23:12:21 +00:00
}
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
/*
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* Copy bits from an adbentry into a newly allocated adb_addrinfo structure.
* The entry must be locked, and its reference count must be incremented.
snapshot address.[ch]
1999-09-22 00:26:20 +00:00
*/
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static dns_adbaddrinfo_t *
set the default destination port in the view, and use it when returning addresses from the adb.
2000-06-01 00:30:58 +00:00
new_adbaddrinfo(dns_adb_t *adb, dns_adbentry_t *entry, in_port_t port) {
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adbaddrinfo_t *ai = NULL;
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
Remove memory limit on ADB finds and fetches Address Database (ADB) shares the memory for the short lived ADB objects (finds, fetches, addrinfo) and the long lived ADB objects (names, entries, namehooks). This could lead to a situation where the resolver-heavy load would force evict ADB objects from the database to point where ADB is completely empty, leading to even more resolver-heavy load. Make the short lived ADB objects use the other memory context that we already created for the hashmaps. This makes the ADB overmem condition to not be triggered by the ongoing resolver fetches.
2025-01-15 10:36:33 +01:00
ai = isc_mem_get(adb->hmctx, sizeof(*ai));
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
*ai = (dns_adbaddrinfo_t){
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
.srtt = atomic_load(&entry->srtt),
.flags = atomic_load(&entry->flags),
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
.publink = ISC_LINK_INITIALIZER,
.sockaddr = entry->sockaddr,
.entry = dns_adbentry_ref(entry),
.magic = DNS_ADBADDRINFO_MAGIC,
};
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
isc_sockaddr_setport(&ai->sockaddr, port);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
update copyright notice
2008-02-07 23:46:54 +00:00
return ai;
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
free_adbaddrinfo(dns_adb_t *adb, dns_adbaddrinfo_t **ainfo) {
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adbaddrinfo_t *ai = NULL;
REQUIRE(ainfo != NULL && DNS_ADBADDRINFO_VALID(*ainfo));
snapshot: dns_adb_done() partial implementation, handles mostly work, but no fetches are started yet.
1999-10-14 00:50:00 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
ai = *ainfo;
*ainfo = NULL;
snapshot: dns_adb_done() partial implementation, handles mostly work, but no fetches are started yet.
1999-10-14 00:50:00 +00:00
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(!ISC_LINK_LINKED(ai, publink));
snapshot: dns_adb_done() partial implementation, handles mostly work, but no fetches are started yet.
1999-10-14 00:50:00 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
ai->magic = 0;
snapshot: dns_adb_done() partial implementation, handles mostly work, but no fetches are started yet.
1999-10-14 00:50:00 +00:00
Resolver query forwarding to DoT-enabled upstream servers Implement TLS transport usage in the resolver. Use the configured TLS transport for the forwarders in the resolver.
2022-12-08 14:18:22 +00:00
if (ai->transport != NULL) {
dns_transport_detach(&ai->transport);
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbentry_detach(&ai->entry);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
Remove memory limit on ADB finds and fetches Address Database (ADB) shares the memory for the short lived ADB objects (finds, fetches, addrinfo) and the long lived ADB objects (names, entries, namehooks). This could lead to a situation where the resolver-heavy load would force evict ADB objects from the database to point where ADB is completely empty, leading to even more resolver-heavy load. Make the short lived ADB objects use the other memory context that we already created for the hashmaps. This makes the ADB overmem condition to not be triggered by the ongoing resolver fetches.
2025-01-15 10:36:33 +01:00
isc_mem_put(adb->hmctx, ai, sizeof(*ai));
snapshot: dns_adb_done() partial implementation, handles mostly work, but no fetches are started yet.
1999-10-14 00:50:00 +00:00
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
static int
match_adbname(struct cds_lfht_node *ht_node, const void *key) {
const dns_adbname_t *adbname0 = caa_container_of(ht_node, dns_adbname_t,
ht_node);
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
const dns_adbname_t *adbname1 = key;
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
if (adbname0->type != adbname1->type) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
return 0;
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
}
return dns_name_equal(adbname0->name, adbname1->name);
}
static uint32_t
hash_adbname(const dns_adbname_t *adbname) {
isc_hash32_t hash;
isc_hash32_init(&hash);
isc_hash32_hash(&hash, adbname->name->ndata, adbname->name->length,
false);
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
isc_hash32_hash(&hash, &adbname->type, sizeof(adbname->type), true);
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
return isc_hash32_finalize(&hash);
}
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
/*
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
* Search for the name in the hash table.
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
*/
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
static dns_adbname_t *
Lock the adbname and adbentry prior to unlocking hash locks There was a datarace that could expire a freshly created ADB names and entries between the return from get_attached_{name,entry} and locking it again. Lock the ADB name and ADB entry inside the hash table lock, so they won't get expired until the full initialization has been complete.
2022-12-13 13:48:55 +01:00
get_attached_and_locked_name(dns_adb_t *adb, const dns_name_t *name,
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
unsigned int type, isc_stdtime_t now) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbname_t *adbname = NULL;
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
dns_adbname_t key = {
.name = UNCONST(name),
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
.type = type,
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
};
uint32_t hashval = hash_adbname(&key);
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
if (isc_mem_isovermem(adb->mctx)) {
purge_names_overmem(adb, 2 * sizeof(*adbname));
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
struct cds_lfht_iter iter;
cds_lfht_lookup(adb->names_ht, hashval, match_adbname, &key, &iter);
adbname = cds_lfht_entry(cds_lfht_iter_get_node(&iter), dns_adbname_t,
ht_node);
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
if (adbname == NULL) {
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
create:
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
adbname = new_adbname(adb, name, key.type);
Fix reference counting in get_attached_entry When get_attached_entry() encounters entry that would be expired, it needs to get reference to the entry before calling maybe_expire_entry(), so the ADB entry doesn't get destroyed inside the its own lock.
2022-12-06 15:59:35 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
/*
* We need to lock the adbname before inserting it into the
* hashtable because any other thread could immediately look the
* newly created adbname after it has been inserted but not yet
* properly initialized by the caller.
*/
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
LOCK(&adbname->lock);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
struct cds_lfht_node *ht_node = cds_lfht_add_unique(
adb->names_ht, hashval, match_adbname, &key,
&adbname->ht_node);
if (ht_node == &adbname->ht_node) {
/* Success. */
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
dns_adbname_ref(adbname);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
ISC_SIEVE_INSERT(adb->lru[isc_tid()].names, adbname,
link);
return adbname;
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
/* Somebody was faster */
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
UNLOCK(&adbname->lock);
Lock the adbname and adbentry prior to unlocking hash locks There was a datarace that could expire a freshly created ADB names and entries between the return from get_attached_{name,entry} and locking it again. Lock the ADB name and ADB entry inside the hash table lock, so they won't get expired until the full initialization has been complete.
2022-12-13 13:48:55 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
destroy_adbname_rcu(&adbname->rcu_head);
adbname = caa_container_of(ht_node, dns_adbname_t, ht_node);
}
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
LOCK(&adbname->lock);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
if (cds_lfht_is_node_deleted(&adbname->ht_node)) {
UNLOCK(&adbname->lock);
goto create;
}
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbname_ref(adbname);
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
/* Is the name we found already expired */
if (maybe_expire_name(adbname, now)) {
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
UNLOCK(&adbname->lock);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbname_detach(&adbname);
goto create;
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
ISC_SIEVE_MARK(adbname, visited);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
return adbname;
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
static int
match_adbentry(struct cds_lfht_node *ht_node, const void *key) {
const dns_adbentry_t *adbentry =
caa_container_of(ht_node, dns_adbentry_t, ht_node);
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
return isc_sockaddr_equal(&adbentry->sockaddr, key);
}
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/*
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
* Find the entry in the adb->entries hashtable.
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
*/
static dns_adbentry_t *
Lock the adbname and adbentry prior to unlocking hash locks There was a datarace that could expire a freshly created ADB names and entries between the return from get_attached_{name,entry} and locking it again. Lock the ADB name and ADB entry inside the hash table lock, so they won't get expired until the full initialization has been complete.
2022-12-13 13:48:55 +01:00
get_attached_and_locked_entry(dns_adb_t *adb, isc_stdtime_t now,
const isc_sockaddr_t *addr) {
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
dns_adbentry_t *adbentry = NULL;
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
uint32_t hashval = isc_sockaddr_hash(addr, true);
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
if (isc_mem_isovermem(adb->mctx)) {
purge_entries_overmem(adb, 2 * sizeof(*adbentry));
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
struct cds_lfht_iter iter;
cds_lfht_lookup(adb->entries_ht, hashval, match_adbentry,
(const unsigned char *)addr, &iter);
adbentry = cds_lfht_entry(cds_lfht_iter_get_node(&iter), dns_adbentry_t,
ht_node);
Don't remove ADB entry from LRU before trying to expire it There was a code flow error that would remove the expired ADB entry from the LRU list and then a check in the expire_entry() would cause assertion error because it expect the ADB entry to be linked. Additionally, the expire mechanism would loop for cases when we would held only a read rwlock; in such case we need to upgrade the lock and try again, not just try again.
2023-02-16 12:26:01 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
if (adbentry == NULL) {
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
create:
Always set ADB entry expiration to now + ADB_ENTRY_WINDOW When ADB entry was created it was set to never expire. If we never called any of the functions that adjust the expiration, it could linger in the ADB forever. Set the expiration (.expires) to now + ADB_ENTRY_WINDOW when creating the new ADB entry to ensure the ADB entry will always expire.
2024-04-03 11:33:48 +02:00
adbentry = new_adbentry(adb, addr, now);
Fix reference counting in get_attached_entry When get_attached_entry() encounters entry that would be expired, it needs to get reference to the entry before calling maybe_expire_entry(), so the ADB entry doesn't get destroyed inside the its own lock.
2022-12-06 15:59:35 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
/*
* We need to lock the adbentry before inserting it into the
* hashtable because any other thread could immediately look the
* newly created adbentry after it has been inserted but not yet
* properly initialized by the caller.
*/
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
LOCK(&adbentry->lock);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
struct cds_lfht_node *ht_node = cds_lfht_add_unique(
adb->entries_ht, hashval, match_adbentry,
(const unsigned char *)addr, &adbentry->ht_node);
if (ht_node == &adbentry->ht_node) {
/* Success */
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
dns_adbentry_ref(adbentry);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
ISC_SIEVE_INSERT(adb->lru[isc_tid()].entries, adbentry,
link);
return adbentry;
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
}
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
/* Somebody was faster */
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
UNLOCK(&adbentry->lock);
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
destroy_adbentry_rcu(&adbentry->rcu_head);
adbentry = caa_container_of(ht_node, dns_adbentry_t, ht_node);
}
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
LOCK(&adbentry->lock);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
if (cds_lfht_is_node_deleted(&adbentry->ht_node)) {
UNLOCK(&adbentry->lock);
goto create;
}
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbentry_ref(adbentry);
Don't remove ADB entry from LRU before trying to expire it There was a code flow error that would remove the expired ADB entry from the LRU list and then a check in the expire_entry() would cause assertion error because it expect the ADB entry to be linked. Additionally, the expire mechanism would loop for cases when we would held only a read rwlock; in such case we need to upgrade the lock and try again, not just try again.
2023-02-16 12:26:01 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
/* Is the entry we found already expired */
if (maybe_expire_entry(adbentry, now)) {
Don't remove ADB entry from LRU before trying to expire it There was a code flow error that would remove the expired ADB entry from the LRU list and then a check in the expire_entry() would cause assertion error because it expect the ADB entry to be linked. Additionally, the expire mechanism would loop for cases when we would held only a read rwlock; in such case we need to upgrade the lock and try again, not just try again.
2023-02-16 12:26:01 +01:00
UNLOCK(&adbentry->lock);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbentry_detach(&adbentry);
goto create;
update copyright notice
2008-02-07 23:46:54 +00:00
}
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
ISC_SIEVE_MARK(adbentry, visited);
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
return adbentry;
Start flushing out the address.c stuff
1999-09-23 00:43:10 +00:00
}
snapshot address.[ch]
1999-09-22 00:26:20 +00:00
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
static void
log_quota(dns_adbentry_t *entry, const char *fmt, ...) {
va_list ap;
char msgbuf[2048];
char addrbuf[ISC_NETADDR_FORMATSIZE];
isc_netaddr_t netaddr;
va_start(ap, fmt);
vsnprintf(msgbuf, sizeof(msgbuf), fmt, ap);
va_end(ap);
isc_netaddr_fromsockaddr(&netaddr, &entry->sockaddr);
isc_netaddr_format(&netaddr, addrbuf, sizeof(addrbuf));
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
isc_log_write(DNS_LOGCATEGORY_DATABASE, DNS_LOGMODULE_ADB, ISC_LOG_INFO,
lib/dns/adb.c: Use atomics for adb quota values and reference counting
2019-01-22 10:47:18 +01:00
"adb: quota %s (%" PRIuFAST32 "/%" PRIuFAST32 "): %s",
addrbuf, atomic_load_relaxed(&entry->active),
atomic_load_relaxed(&entry->quota), msgbuf);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
}
snapshot
1999-10-08 23:12:21 +00:00
static void
Remove dns_adblameinfo from dns_adb Keeping the information about lame server in the ADB was done in !322 to fix following security issue: [CVE-2021-25219] Disable "lame-ttl" cache The handling of the lame servers needs to be redesigned and it is not going to be enabled any time soon, and the current code is just dead code that takes up space, code and stands in the way of making ADB work faster. Remove all the internals needed for handling the lame servers in the ADB for now. It might get reintroduced later if and when we redesign ADB.
2023-09-21 15:20:58 +02:00
copy_namehook_lists(dns_adb_t *adb, dns_adbfind_t *find, dns_adbname_t *name) {
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adbentry_t *entry = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
Turn (int & flag) into (int & flag) != 0 when implicitly typed to bool
2018-10-11 11:57:57 +02:00
if ((find->options & DNS_ADBFIND_INET) != 0) {
use ISC_LIST_FOREACH in more places use the ISC_LIST_FOREACH pattern in places where lists had been iterated using a different pattern from the typical `for` loop: for example, `while (!ISC_LIST_EMPTY(...))` or `while ((e = ISC_LIST_HEAD(...)) != NULL)`.
2025-03-22 15:26:16 -07:00
ISC_LIST_FOREACH(name->v4, namehook, name_link) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbaddrinfo_t *addrinfo = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
entry = namehook->entry;
Exempt prefetches from the fetches-per-server quota Give prefetches a free pass through the quota so that the cache entries for popular zones could be updated successfully even if the quota for is already reached.
2024-06-07 16:24:00 +00:00
if ((find->options & DNS_ADBFIND_QUOTAEXEMPT) == 0 &&
adbentry_overquota(entry))
{
Remove dns_adblameinfo from dns_adb Keeping the information about lame server in the ADB was done in !322 to fix following security issue: [CVE-2021-25219] Disable "lame-ttl" cache The handling of the lame servers needs to be redesigned and it is not going to be enabled any time soon, and the current code is just dead code that takes up space, code and stands in the way of making ADB work faster. Remove all the internals needed for handling the lame servers in the ADB for now. It might get reintroduced later if and when we redesign ADB.
2023-09-21 15:20:58 +02:00
find->options |= DNS_ADBFIND_OVERQUOTA;
use ISC_LIST_FOREACH in more places use the ISC_LIST_FOREACH pattern in places where lists had been iterated using a different pattern from the typical `for` loop: for example, `while (!ISC_LIST_EMPTY(...))` or `while ((e = ISC_LIST_HEAD(...)) != NULL)`.
2025-03-22 15:26:16 -07:00
continue;
update copyright notice
2008-02-07 23:46:54 +00:00
}
Rewrite dns_resolver and dns_request to use netmgr timeouts - The `timeout_action` parameter to dns_dispatch_addresponse() been replaced with a netmgr callback that is called when a dispatch read times out. this callback may optionally reset the read timer and resume reading. - Added a function to convert isc_interval to milliseconds; this is used to translate fctx->interval into a value that can be passed to dns_dispatch_addresponse() as the timeout. - Note that netmgr timeouts are accurate to the millisecond, so code to check whether a timeout has been reached cannot rely on microsecond accuracy. - If serve-stale is configured, then a timeout received by the resolver may trigger it to return stale data, and then resume waiting for the read timeout. this is no longer based on a separate stale timer. - The code for canceling requests in request.c has been altered so that it can run asynchronously. - TCP timeout events apply to the dispatch, which may be shared by multiple queries. since in the event of a timeout we have no query ID to use to identify the resp we wanted, we now just send the timeout to the oldest query that was pending. - There was some additional refactoring in the resolver: combining fctx_join() and fctx_try_events() into one function to reduce code duplication, and using fixednames in fetchctx and fetchevent. - Incidental fix: new_adbaddrinfo() can't return NULL anymore, so the code can be simplified.
2021-04-22 18:58:01 -07:00
update copyright notice
2008-02-07 23:46:54 +00:00
addrinfo = new_adbaddrinfo(adb, entry, find->port);
Rewrite dns_resolver and dns_request to use netmgr timeouts - The `timeout_action` parameter to dns_dispatch_addresponse() been replaced with a netmgr callback that is called when a dispatch read times out. this callback may optionally reset the read timer and resume reading. - Added a function to convert isc_interval to milliseconds; this is used to translate fctx->interval into a value that can be passed to dns_dispatch_addresponse() as the timeout. - Note that netmgr timeouts are accurate to the millisecond, so code to check whether a timeout has been reached cannot rely on microsecond accuracy. - If serve-stale is configured, then a timeout received by the resolver may trigger it to return stale data, and then resume waiting for the read timeout. this is no longer based on a separate stale timer. - The code for canceling requests in request.c has been altered so that it can run asynchronously. - TCP timeout events apply to the dispatch, which may be shared by multiple queries. since in the event of a timeout we have no query ID to use to identify the resp we wanted, we now just send the timeout to the oldest query that was pending. - There was some additional refactoring in the resolver: combining fctx_join() and fctx_try_events() into one function to reduce code duplication, and using fixednames in fetchctx and fetchevent. - Incidental fix: new_adbaddrinfo() can't return NULL anymore, so the code can be simplified.
2021-04-22 18:58:01 -07:00
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Found a valid entry. Add it to the find's list.
*/
ISC_LIST_APPEND(find->list, addrinfo, publink);
}
}
Turn (int & flag) into (int & flag) != 0 when implicitly typed to bool
2018-10-11 11:57:57 +02:00
if ((find->options & DNS_ADBFIND_INET6) != 0) {
use ISC_LIST_FOREACH in more places use the ISC_LIST_FOREACH pattern in places where lists had been iterated using a different pattern from the typical `for` loop: for example, `while (!ISC_LIST_EMPTY(...))` or `while ((e = ISC_LIST_HEAD(...)) != NULL)`.
2025-03-22 15:26:16 -07:00
ISC_LIST_FOREACH(name->v6, namehook, name_link) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbaddrinfo_t *addrinfo = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
entry = namehook->entry;
Exempt prefetches from the fetches-per-server quota Give prefetches a free pass through the quota so that the cache entries for popular zones could be updated successfully even if the quota for is already reached.
2024-06-07 16:24:00 +00:00
if ((find->options & DNS_ADBFIND_QUOTAEXEMPT) == 0 &&
adbentry_overquota(entry))
{
Remove dns_adblameinfo from dns_adb Keeping the information about lame server in the ADB was done in !322 to fix following security issue: [CVE-2021-25219] Disable "lame-ttl" cache The handling of the lame servers needs to be redesigned and it is not going to be enabled any time soon, and the current code is just dead code that takes up space, code and stands in the way of making ADB work faster. Remove all the internals needed for handling the lame servers in the ADB for now. It might get reintroduced later if and when we redesign ADB.
2023-09-21 15:20:58 +02:00
find->options |= DNS_ADBFIND_OVERQUOTA;
use ISC_LIST_FOREACH in more places use the ISC_LIST_FOREACH pattern in places where lists had been iterated using a different pattern from the typical `for` loop: for example, `while (!ISC_LIST_EMPTY(...))` or `while ((e = ISC_LIST_HEAD(...)) != NULL)`.
2025-03-22 15:26:16 -07:00
continue;
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
addrinfo = new_adbaddrinfo(adb, entry, find->port);
Rewrite dns_resolver and dns_request to use netmgr timeouts - The `timeout_action` parameter to dns_dispatch_addresponse() been replaced with a netmgr callback that is called when a dispatch read times out. this callback may optionally reset the read timer and resume reading. - Added a function to convert isc_interval to milliseconds; this is used to translate fctx->interval into a value that can be passed to dns_dispatch_addresponse() as the timeout. - Note that netmgr timeouts are accurate to the millisecond, so code to check whether a timeout has been reached cannot rely on microsecond accuracy. - If serve-stale is configured, then a timeout received by the resolver may trigger it to return stale data, and then resume waiting for the read timeout. this is no longer based on a separate stale timer. - The code for canceling requests in request.c has been altered so that it can run asynchronously. - TCP timeout events apply to the dispatch, which may be shared by multiple queries. since in the event of a timeout we have no query ID to use to identify the resp we wanted, we now just send the timeout to the oldest query that was pending. - There was some additional refactoring in the resolver: combining fctx_join() and fctx_try_events() into one function to reduce code duplication, and using fixednames in fetchctx and fetchevent. - Incidental fix: new_adbaddrinfo() can't return NULL anymore, so the code can be simplified.
2021-04-22 18:58:01 -07:00
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Found a valid entry. Add it to the find's list.
*/
ISC_LIST_APPEND(find->list, addrinfo, publink);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
}
}
event-driven shutdown/destroy
1999-10-25 23:07:14 +00:00
}
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
static bool
name_expired(dns_adbname_t *adbname, isc_stdtime_t now) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
REQUIRE(DNS_ADBNAME_VALID(adbname));
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/* Leave this name alone if it still has active namehooks... */
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (NAME_HAS_V4(adbname) || NAME_HAS_V6(adbname)) {
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
return false;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/* ...an active fetch in progres... */
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (NAME_FETCH(adbname)) {
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
return false;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/* ... or is not yet expired. */
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (!EXPIRE_OK(adbname->expire_v4, now) ||
remove 'target' from dns_adb the target name parameter to dns_adb_createfind() was always passed as NULL, so we can safely remove it. relatedly, the 'target' field in the dns_adbname structure was never referenced after being set. the 'expire_target' field was used, but only as a way to check whether an ADB name represents a CNAME or DNAME, and that information can be stored as a single flag.
2025-02-21 01:24:42 -08:00
!EXPIRE_OK(adbname->expire_v6, now))
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
{
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
return false;
}
return true;
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
/*
* The name must be locked and write lock on adb->names_lock must be held.
*/
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
static bool
maybe_expire_name(dns_adbname_t *adbname, isc_stdtime_t now) {
if (name_expired(adbname, now)) {
expire_name(adbname, DNS_ADB_EXPIRED);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
return true;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
return false;
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
}
static void
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
expire_entry_async(void *arg) {
dns_adbentry_t *adbentry = arg;
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
dns_adb_t *adb = adbentry->adb;
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
REQUIRE(adbentry->loop == isc_loop());
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
ISC_SIEVE_UNLINK(adb->lru[isc_tid()].entries, adbentry, link);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
dns_adbentry_detach(&adbentry);
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
static void
expire_entry(dns_adbentry_t *adbentry) {
dns_adb_t *adb = adbentry->adb;
if (cds_lfht_del(adb->entries_ht, &adbentry->ht_node) == 0) {
isc_async_run(adbentry->loop, expire_entry_async, adbentry);
}
}
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
static bool
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
entry_expired(dns_adbentry_t *adbentry, isc_stdtime_t now) {
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
if (!ISC_LIST_EMPTY(adbentry->nhs)) {
return false;
}
Always set ADB entry expiration to now + ADB_ENTRY_WINDOW When ADB entry was created it was set to never expire. If we never called any of the functions that adjust the expiration, it could linger in the ADB forever. Set the expiration (.expires) to now + ADB_ENTRY_WINDOW when creating the new ADB entry to ensure the ADB entry will always expire.
2024-04-03 11:33:48 +02:00
if (!EXPIRE_OK(adbentry->expires, now)) {
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
return false;
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
return true;
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
}
Use C-RW-WP lock in the dns_adb unit Replace the isc_mutex in the dns_adb unit with isc_rwlock for better performance. Both ADB names and ADB entries hashtables and LRU are now using isc_rwlock.
2023-02-13 15:52:51 +01:00
static bool
maybe_expire_entry(dns_adbentry_t *adbentry, isc_stdtime_t now) {
REQUIRE(DNS_ADBENTRY_VALID(adbentry));
if (entry_expired(adbentry, now)) {
expire_entry(adbentry);
return true;
}
return false;
}
When overmem, clean enough memory when adding new ADB names/entries The purge_stale_names()/purge_stale_entries() is opportunistic even when we are under memory pressure (overmem). Split the opportunistic LRU cleaning and overmem cleaning. This makes the stale purging much simpler as we don't have to try that hard and makes the overmem cleaning always cleanup double the amount of the newly allocated ADB name/entry.
2025-06-25 12:05:14 +02:00
static void
purge_names_overmem(dns_adb_t *adb, size_t requested) {
size_t expired = 0;
Purge stale ADB names globaly, not per bucket Before the refactoring, there was only few buckets with many names in them, so cleaning up stale ADB names per-bucket made sense. After the refactoring, each bucket directly maps to ADB name, so purging has been effectively disabled. Create a global LRU list for ADB names (and ADB entries) and purge the stale ADB names globally.
2022-08-01 09:07:38 +02:00
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
do {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbname_t *adbname = ISC_SIEVE_NEXT(
adb->lru[isc_tid()].names, visited, link);
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
if (adbname == NULL) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
break;
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbname_ref(adbname);
LOCK(&adbname->lock);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
/*
* Remove the name if it's expired or unused,
* has no address data.
*/
When overmem, clean enough memory when adding new ADB names/entries The purge_stale_names()/purge_stale_entries() is opportunistic even when we are under memory pressure (overmem). Split the opportunistic LRU cleaning and overmem cleaning. This makes the stale purging much simpler as we don't have to try that hard and makes the overmem cleaning always cleanup double the amount of the newly allocated ADB name/entry.
2025-06-25 12:05:14 +02:00
maybe_expire_namehooks(adbname, INT_MAX);
expire_name(adbname, DNS_ADB_CANCELED);
expired += sizeof(*adbname);
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&adbname->lock);
dns_adbname_detach(&adbname);
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
} while (expired < requested);
add dns_adb_findaddrinfo() and dns_adb_freeaddrinfo()
2000-01-12 03:00:33 +00:00
}
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
static void
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
cleanup_names(dns_adb_t *adb, isc_stdtime_t now) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbname_t *adbname = NULL;
struct cds_lfht_iter iter;
cds_lfht_for_each_entry(adb->names_ht, &iter, adbname, ht_node) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbname_ref(adbname);
LOCK(&adbname->lock);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/*
* Name hooks expire after the address record's TTL
* or 30 minutes, whichever is shorter. If after cleaning
* those up there are no name hooks left, and no active
* fetches, we can remove this name from the bucket.
*/
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
maybe_expire_namehooks(adbname, now);
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
(void)maybe_expire_name(adbname, now);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&adbname->lock);
dns_adbname_detach(&adbname);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
}
}
When overmem, clean enough memory when adding new ADB names/entries The purge_stale_names()/purge_stale_entries() is opportunistic even when we are under memory pressure (overmem). Split the opportunistic LRU cleaning and overmem cleaning. This makes the stale purging much simpler as we don't have to try that hard and makes the overmem cleaning always cleanup double the amount of the newly allocated ADB name/entry.
2025-06-25 12:05:14 +02:00
static void
purge_entries_overmem(dns_adb_t *adb, size_t requested) {
size_t expired = 0;
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
do {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbentry_t *adbentry = ISC_SIEVE_NEXT(
adb->lru[isc_tid()].entries, visited, link);
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
if (adbentry == NULL) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
break;
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
}
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
dns_adbentry_ref(adbentry);
LOCK(&adbentry->lock);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
expire_entry(adbentry);
When overmem, clean enough memory when adding new ADB names/entries The purge_stale_names()/purge_stale_entries() is opportunistic even when we are under memory pressure (overmem). Split the opportunistic LRU cleaning and overmem cleaning. This makes the stale purging much simpler as we don't have to try that hard and makes the overmem cleaning always cleanup double the amount of the newly allocated ADB name/entry.
2025-06-25 12:05:14 +02:00
expired += sizeof(*adbentry);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
UNLOCK(&adbentry->lock);
dns_adbentry_detach(&adbentry);
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
} while (expired < requested);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
}
static void
cleanup_entries(dns_adb_t *adb, isc_stdtime_t now) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbentry_t *adbentry = NULL;
struct cds_lfht_iter iter;
cds_lfht_for_each_entry(adb->entries_ht, &iter, adbentry, ht_node) {
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
dns_adbentry_ref(adbentry);
LOCK(&adbentry->lock);
maybe_expire_entry(adbentry, now);
UNLOCK(&adbentry->lock);
dns_adbentry_detach(&adbentry);
}
}
use isc_mutexblock_init/destroy()
1999-09-24 23:54:42 +00:00
static void
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adb_destroy(dns_adb_t *adb) {
Remove several superfluous newlines in log messages
2022-04-05 13:16:00 +01:00
DP(DEF_LEVEL, "destroying ADB %p", adb);
Add detailed ADB and entry attach/detach tracing To turn on detailed debug tracing of dns_adb and dns_adbentry reference counting, #define ADB_TRACE at the top of adb.c. This is off by default.
2022-03-24 15:58:26 -07:00
update copyright notice
2008-02-07 23:46:54 +00:00
adb->magic = 0;
commit lruttl to the mainline. A tag was set called skan_lruttl-mainline-base, and I will tag this as skan_lruttl-mainline-merge after this commit
2007-10-19 17:15:53 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
RUNTIME_CHECK(!cds_lfht_destroy(adb->names_ht, NULL));
adb->names_ht = NULL;
RUNTIME_CHECK(!cds_lfht_destroy(adb->entries_ht, NULL));
adb->entries_ht = NULL;
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
isc_mem_cput(adb->hmctx, adb->lru, adb->nloops, sizeof(adb->lru[0]));
remove ADB whenshutdown events weakly attaching and detaching the view when creating or destroying the ADB obviates the need for a whenshutdown callback event to do the detaching. remove the dns_adb_whenshutdown() mechanism, since it is no longer needed.
2022-05-11 13:55:01 -07:00
Refactor isc_stats_create() and its downstream users to return void The isc_stats_create() can no longer return anything else than ISC_R_SUCCESS. Refactor isc_stats_create() and its variants in libdns, libns and named to just return void.
2023-06-26 10:58:30 +02:00
isc_mem_detach(&adb->hmctx);
Exclude the ADB hashmaps from ADB overmem accounting The ADB overmem accounting would include the memory used by hashtables thus vastly reducing the space that can be used for ADB names and entries when the hashtables would grow. Create own memory context for the ADB names and entries hash tables.
2022-12-13 14:14:21 +01:00
- Make isc_mutex_destroy return void - Make isc_mutexblock_init/destroy return void - Minor cleanups
2018-11-19 10:31:09 +00:00
isc_mutex_destroy(&adb->lock);
use isc_mutexblock_init/destroy()
1999-09-24 23:54:42 +00:00
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
isc_stats_detach(&adb->stats);
attach the resolver to the ADB the ADB depends on the resolver, but previously only accessed it via the view. as view->resolver may now be detached before the ADB finishes, a shutdown race was possible. attaching to the resolver directly prevents this.
2022-05-12 15:51:10 -07:00
dns_resolver_detach(&adb->res);
remove ADB whenshutdown events weakly attaching and detaching the view when creating or destroying the ADB obviates the need for a whenshutdown callback event to do the detaching. remove the dns_adb_whenshutdown() mechanism, since it is no longer needed.
2022-05-11 13:55:01 -07:00
dns_view_weakdetach(&adb->view);
update copyright notice
2008-02-07 23:46:54 +00:00
isc_mem_putanddetach(&adb->mctx, adb, sizeof(dns_adb_t));
use isc_mutexblock_init/destroy()
1999-09-24 23:54:42 +00:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#if DNS_ADB_TRACE
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
ISC_REFCOUNT_TRACE_IMPL(dns_adb, dns_adb_destroy);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#else
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
ISC_REFCOUNT_IMPL(dns_adb, dns_adb_destroy);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#endif
snapshot address.[ch]
1999-09-22 00:26:20 +00:00
/*
* Public functions.
*/
Refactor dns_adb_create() to return void After isc_stats_create() change, the dns_adb_create() cannot fail, so refactor the function to return void and fix all its uses.
2023-06-26 11:04:33 +02:00
void
Change the loopmgr to be singleton All the applications built on top of the loop manager were required to create just a single instance of the loop manager. Refactor the loop manager to not expose this instance to the callers and keep the loop manager object internal to the isc_loop compilation unit. This significantly simplifies a number of data structures and calls to the isc_loop API.
2025-07-14 10:50:21 +02:00
dns_adb_create(isc_mem_t *mem, dns_view_t *view, dns_adb_t **adbp) {
update copyright notice
2008-02-07 23:46:54 +00:00
REQUIRE(mem != NULL);
REQUIRE(view != NULL);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
REQUIRE(adbp != NULL && *adbp == NULL);
update copyright notice
2008-02-07 23:46:54 +00:00
Change the loopmgr to be singleton All the applications built on top of the loop manager were required to create just a single instance of the loop manager. Refactor the loop manager to not expose this instance to the callers and keep the loop manager object internal to the isc_loop compilation unit. This significantly simplifies a number of data structures and calls to the isc_loop API.
2025-07-14 10:50:21 +02:00
uint32_t nloops = isc_loopmgr_nloops();
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adb_t *adb = isc_mem_get(mem, sizeof(dns_adb_t));
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
*adb = (dns_adb_t){
Rewrite dns_adb LRU to SIEVE The dns_adb cleaning is little bit muddled as it mixes the "TTL" based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for adbentry) with overmem cleaning. Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem cleaning only with a requirement to always cleanup at least 2-times the size of the newly added entry.
2025-06-12 11:14:51 +02:00
.references = 1,
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
.nloops = nloops,
.magic = DNS_ADB_MAGIC,
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
};
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Initialize things here that cannot fail, and especially things
* that must be NULL for the error return to work properly.
*/
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
#if DNS_ADB_TRACE
fprintf(stderr, "dns_adb__init:%s:%s:%d:%p->references = 1\n", __func__,
__FILE__, __LINE__ + 1, adb);
#endif
remove ADB whenshutdown events weakly attaching and detaching the view when creating or destroying the ADB obviates the need for a whenshutdown callback event to do the detaching. remove the dns_adb_whenshutdown() mechanism, since it is no longer needed.
2022-05-11 13:55:01 -07:00
dns_view_weakattach(view, &adb->view);
attach the resolver to the ADB the ADB depends on the resolver, but previously only accessed it via the view. as view->resolver may now be detached before the ADB finishes, a shutdown race was possible. attaching to the resolver directly prevents this.
2022-05-12 15:51:10 -07:00
dns_resolver_attach(view->resolver, &adb->res);
update copyright notice
2008-02-07 23:46:54 +00:00
isc_mem_attach(mem, &adb->mctx);
Give every memory context a name Instead of giving the memory context names with an explicit call to isc_mem_setname(), add the name to isc_mem_create() call to have all the memory contexts an unconditional name.
2025-02-21 12:45:08 +01:00
isc_mem_create("ADB_dynamic", &adb->hmctx);
Exclude the ADB hashmaps from ADB overmem accounting The ADB overmem accounting would include the memory used by hashtables thus vastly reducing the space that can be used for ADB names and entries when the hashtables would grow. Create own memory context for the ADB names and entries hash tables.
2022-12-13 14:14:21 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
adb->names_ht = cds_lfht_new(ADB_HASH_SIZE, ADB_HASH_SIZE, 0,
CDS_LFHT_AUTO_RESIZE | CDS_LFHT_ACCOUNTING,
NULL);
INSIST(adb->names_ht != NULL);
adb->entries_ht =
cds_lfht_new(ADB_HASH_SIZE, ADB_HASH_SIZE, 0,
CDS_LFHT_AUTO_RESIZE | CDS_LFHT_ACCOUNTING, NULL);
INSIST(adb->entries_ht != NULL);
2993. [func] Dynamically grow adb hash tables. [RT #21186]
2010-12-21 03:11:42 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
adb->lru = isc_mem_cget(adb->hmctx, adb->nloops, sizeof(adb->lru[0]));
for (size_t i = 0; i < adb->nloops; i++) {
ISC_SIEVE_INIT(adb->lru[i].names);
ISC_SIEVE_INIT(adb->lru[i].entries);
}
- Make isc_mutex_destroy return void - Make isc_mutexblock_init/destroy return void - Minor cleanups
2018-11-19 10:31:09 +00:00
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
isc_mutex_init(&adb->lock);
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor isc_stats_create() and its downstream users to return void The isc_stats_create() can no longer return anything else than ISC_R_SUCCESS. Refactor isc_stats_create() and its variants in libdns, libns and named to just return void.
2023-06-26 10:58:30 +02:00
isc_stats_create(adb->mctx, &adb->stats, dns_adbstats_max);
3353. [bug] Use a single task for task exclusive operations. [RT #29872]
2012-07-19 23:00:21 +10:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
set_adbstat(adb, 0, dns_adbstats_nnames);
set_adbstat(adb, 0, dns_adbstats_nentries);
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
*adbp = adb;
}
static void
dns_adb_shutdown_async(void *arg) {
dns_adb_t *adb = arg;
synchronize_rcu();
rcu_read_lock();
shutdown_names(adb);
shutdown_entries(adb);
rcu_read_unlock();
dns_adb_detach(&adb);
Start on address.c
1999-09-21 22:46:42 +00:00
}
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
void
dns_adb_shutdown(dns_adb_t *adb) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
if (!atomic_compare_exchange_strong(&adb->shuttingdown,
&(bool){ false }, true))
Update sources to Clang 15 formatting
2022-11-02 19:33:14 +01:00
{
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
return;
}
[rt36341] 3905. [bug] Address deadlock between view.c and adb.c. [RT #36341]
2014-07-31 11:38:11 +10:00
Remove several superfluous newlines in log messages
2022-04-05 13:16:00 +01:00
DP(DEF_LEVEL, "shutting down ADB %p", adb);
Add detailed ADB and entry attach/detach tracing To turn on detailed debug tracing of dns_adb and dns_adbentry reference counting, #define ADB_TRACE at the top of adb.c. This is off by default.
2022-03-24 15:58:26 -07:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
isc_mem_clearwater(adb->mctx);
[rt36341] 3905. [bug] Address deadlock between view.c and adb.c. [RT #36341]
2014-07-31 11:38:11 +10:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
/*
* dns_adb_shutdown() can get called from call_rcu thread, so we need to
* pass the control over synchronize_rcu() back to main loop thread when
* shutting down ADB.
*/
dns_adb_ref(adb);
Change the loopmgr to be singleton All the applications built on top of the loop manager were required to create just a single instance of the loop manager. Refactor the loop manager to not expose this instance to the callers and keep the loop manager object internal to the isc_loop compilation unit. This significantly simplifies a number of data structures and calls to the isc_loop API.
2025-07-14 10:50:21 +02:00
isc_async_run(isc_loop_main(), dns_adb_shutdown_async, adb);
Start on address.c
1999-09-21 22:46:42 +00:00
}
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/*
* Look up the name in our internal database.
*
* There are three possibilities. Note that these are not always exclusive.
*
* - No name found. In this case, allocate a new name header and
* an initial namehook or two.
*
* - Name found, valid addresses present. Allocate one addrinfo
* structure for each found and append it to the linked list
* of addresses for this header.
*
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
* - Name found, queries pending. In this case, if a loop was
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* passed in, allocate a job id, attach it to the name's job
* list and remember to tell the caller that there will be
* more info coming later.
*/
Start on address.c
1999-09-21 22:46:42 +00:00
isc_result_t
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
dns_adb_createfind(dns_adb_t *adb, isc_loop_t *loop, isc_job_cb cb, void *cbarg,
const dns_name_t *name, const dns_name_t *qname,
Remove dns_adblameinfo from dns_adb Keeping the information about lame server in the ADB was done in !322 to fix following security issue: [CVE-2021-25219] Disable "lame-ttl" cache The handling of the lame servers needs to be redesigned and it is not going to be enabled any time soon, and the current code is just dead code that takes up space, code and stands in the way of making ADB work faster. Remove all the internals needed for handling the lame servers in the ADB for now. It might get reintroduced later if and when we redesign ADB.
2023-09-21 15:20:58 +02:00
dns_rdatatype_t qtype ISC_ATTR_UNUSED, unsigned int options,
remove 'target' from dns_adb the target name parameter to dns_adb_createfind() was always passed as NULL, so we can safely remove it. relatedly, the 'target' field in the dns_adbname structure was never referenced after being set. the 'expire_target' field was used, but only as a way to check whether an ADB name represents a CNAME or DNAME, and that information can be stored as a single flag.
2025-02-21 01:24:42 -08:00
isc_stdtime_t now, in_port_t port, unsigned int depth,
isc_counter_t *qc, isc_counter_t *gqc,
libdns refactoring: get rid of two versions of dns_adb_createfind and dns_adb_probesize
2018-04-03 13:10:03 +02:00
dns_adbfind_t **findp) {
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
isc_result_t result = ISC_R_UNEXPECTED;
incidental cleanup The NAME_FETCH_A and NAME_FETCH_AAAA macros were meant to be boolean, indicating whether the pointers were set or not, while the NAME_FETCH_V4 and NAME_FETCH_V6 macros were meant to return the pointer values. The latter were only used as booleans, so they've been removed in favor of the former. Also did some style cleanup and removed an unreachable code block.
2021-10-18 17:57:58 -07:00
dns_adbfind_t *find = NULL;
dns_adbname_t *adbname = NULL;
bool want_event = true;
bool start_at_zone = false;
bool alias = false;
bool have_address = false;
unsigned int wanted_addresses = (options & DNS_ADBFIND_ADDRESSMASK);
unsigned int wanted_fetches = 0;
unsigned int query_pending = 0;
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
char namebuf[DNS_NAME_FORMATSIZE] = { 0 };
update copyright notice
2008-02-07 23:46:54 +00:00
REQUIRE(DNS_ADB_VALID(adb));
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
if (loop != NULL) {
REQUIRE(cb != NULL);
update copyright notice
2008-02-07 23:46:54 +00:00
}
REQUIRE(name != NULL);
REQUIRE(qname != NULL);
REQUIRE(findp != NULL && *findp == NULL);
REQUIRE((options & DNS_ADBFIND_ADDRESSMASK) != 0);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_lock();
Fix the data race when shutting down dns_adb When dns_adb is shutting down, first the adb->shutting_down flag is set and then task is created that runs shutdown_stage2() that sets the shutdown flag on names and entries. However, when dns_adb_createfind() is called, only the individual shutdown flags are being checked, and the global adb->shutting_down flag was not checked. Because of that it was possible for a different thread to slip in and create new find between the dns_adb_shutdown() and dns_adb_detach(), but before the shutdown_stage2() task is complete. This was detected by ThreadSanitizer as data race because the zonetable might have been already detached by dns_view shutdown process and simultaneously accessed by dns_adb_createfind(). This commit converts the adb->shutting_down to atomic_bool to prevent the global adb lock when creating the find.
2021-11-15 12:13:22 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
if (atomic_load(&adb->shuttingdown)) {
rcu_read_unlock();
Fix the data race when shutting down dns_adb When dns_adb is shutting down, first the adb->shutting_down flag is set and then task is created that runs shutdown_stage2() that sets the shutdown flag on names and entries. However, when dns_adb_createfind() is called, only the individual shutdown flags are being checked, and the global adb->shutting_down flag was not checked. Because of that it was possible for a different thread to slip in and create new find between the dns_adb_shutdown() and dns_adb_detach(), but before the shutdown_stage2() task is complete. This was detected by ThreadSanitizer as data race because the zonetable might have been already detached by dns_view shutdown process and simultaneously accessed by dns_adb_createfind(). This commit converts the adb->shutting_down to atomic_bool to prevent the global adb lock when creating the find.
2021-11-15 12:13:22 +01:00
return ISC_R_SHUTTINGDOWN;
}
update copyright notice
2008-02-07 23:46:54 +00:00
if (now == 0) {
Apply the semantic patch to remove isc_stdtime_get() This is a simple replacement using the semantic patch from the previous commit and as added bonus, one removal of previously undetected unused variable in named/server.c.
2023-03-30 21:13:41 +02:00
now = isc_stdtime_now();
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
Store static-stub addresses seperately in the adb Static-stub address and addresses from other sources where being mixed together resulting in static-stub queries going to addresses not specified in the configuration or alternatively static-stub addresses being used instead of the real addresses.
2024-08-14 23:46:44 +10:00
/*
* If STATICSTUB is set we always want to have STARTATZONE set.
*/
if (options & DNS_ADBFIND_STATICSTUB) {
options |= DNS_ADBFIND_STARTATZONE;
}
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Remember what types of addresses we are interested in.
*/
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
find = new_adbfind(adb, port);
update copyright notice
2008-02-07 23:46:54 +00:00
find->options = options;
find->flags |= wanted_addresses;
if (FIND_WANTEVENT(find)) {
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
REQUIRE(loop != NULL);
update copyright notice
2008-02-07 23:46:54 +00:00
}
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
if (isc_log_wouldlog(DEF_LEVEL)) {
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
dns_name_format(name, namebuf, sizeof(namebuf));
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
/* Try to see if we know anything about this name at all. */
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
adbname = get_attached_and_locked_name(
adb, name, ADBNAME_TYPE(find->options), now);
update copyright notice
2008-02-07 23:46:54 +00:00
/*
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* Name hooks expire after the address record's TTL or 30 minutes,
* whichever is shorter. If there are expired name hooks, remove
* them so we'll send a new fetch.
update copyright notice
2008-02-07 23:46:54 +00:00
*/
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
maybe_expire_namehooks(adbname, now);
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Do we know that the name is an alias?
*/
remove 'target' from dns_adb the target name parameter to dns_adb_createfind() was always passed as NULL, so we can safely remove it. relatedly, the 'target' field in the dns_adbname structure was never referenced after being set. the 'expire_target' field was used, but only as a way to check whether an ADB name represents a CNAME or DNAME, and that information can be stored as a single flag.
2025-02-21 01:24:42 -08:00
if (NAME_ALIAS(adbname) && !EXPIRE_OK(adbname->expire_v4, now)) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
/* Yes, it is. */
update copyright notice
2008-02-07 23:46:54 +00:00
DP(DEF_LEVEL,
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
"dns_adb_createfind: name %s (%p) is an alias (cached)",
namebuf, adbname);
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
alias = true;
update copyright notice
2008-02-07 23:46:54 +00:00
goto post_copy;
}
/*
* Try to populate the name from the database and/or
* start fetches. First try looking for an A record
* in the database.
*/
if (!NAME_HAS_V4(adbname) && EXPIRE_OK(adbname->expire_v4, now) &&
WANT_INET(wanted_addresses))
{
result = dbfind_name(adbname, now, dns_rdatatype_a);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
switch (result) {
case ISC_R_SUCCESS:
/* Found an A; now we proceed to check for AAAA */
update copyright notice
2008-02-07 23:46:54 +00:00
DP(DEF_LEVEL,
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
"dns_adb_createfind: found A for name %s (%p) in db",
namebuf, adbname);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
break;
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
case DNS_R_ALIAS:
/* Got a CNAME or DNAME. */
update copyright notice
2008-02-07 23:46:54 +00:00
DP(DEF_LEVEL,
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
"dns_adb_createfind: name %s (%p) is an alias",
namebuf, adbname);
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
alias = true;
update copyright notice
2008-02-07 23:46:54 +00:00
goto post_copy;
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
case DNS_R_NXDOMAIN:
case DNS_R_NCACHENXDOMAIN:
/*
* If the name doesn't exist at all, don't bother with
* v6 queries; they won't work.
*/
update copyright notice
2008-02-07 23:46:54 +00:00
goto fetch;
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
case DNS_R_NXRRSET:
case DNS_R_NCACHENXRRSET:
case DNS_R_HINTNXRRSET:
/*
* The name does exist but we didn't get our data, go
* ahead and try AAAA.
*/
break;
default:
/*
* Any other result, start a fetch for A, then fall
* through to AAAA.
*/
Store static-stub addresses seperately in the adb Static-stub address and addresses from other sources where being mixed together resulting in static-stub queries going to addresses not specified in the configuration or alternatively static-stub addresses being used instead of the real addresses.
2024-08-14 23:46:44 +10:00
if (!NAME_FETCH_A(adbname) && !FIND_STATICSTUB(find)) {
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
wanted_fetches |= DNS_ADBFIND_INET;
}
break;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
}
fetch contexts
1999-10-26 01:12:20 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/*
* Now look up or start fetches for AAAA.
*/
update copyright notice
2008-02-07 23:46:54 +00:00
if (!NAME_HAS_V6(adbname) && EXPIRE_OK(adbname->expire_v6, now) &&
WANT_INET6(wanted_addresses))
{
result = dbfind_name(adbname, now, dns_rdatatype_aaaa);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
switch (result) {
case ISC_R_SUCCESS:
update copyright notice
2008-02-07 23:46:54 +00:00
DP(DEF_LEVEL,
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
"dns_adb_createfind: found AAAA for name %s (%p)",
namebuf, adbname);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
break;
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
case DNS_R_ALIAS:
/* Got a CNAME or DNAME. */
update copyright notice
2008-02-07 23:46:54 +00:00
DP(DEF_LEVEL,
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
"dns_adb_createfind: name %s (%p) is an alias",
namebuf, adbname);
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
alias = true;
update copyright notice
2008-02-07 23:46:54 +00:00
goto post_copy;
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
case DNS_R_NXDOMAIN:
case DNS_R_NCACHENXDOMAIN:
case DNS_R_NXRRSET:
case DNS_R_NCACHENXRRSET:
/*
* Name doens't exist or was found in the negative
* cache to have no AAAA, don't bother fetching.
*/
break;
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
default:
/*
* Any other result, start a fetch for AAAA.
*/
Store static-stub addresses seperately in the adb Static-stub address and addresses from other sources where being mixed together resulting in static-stub queries going to addresses not specified in the configuration or alternatively static-stub addresses being used instead of the real addresses.
2024-08-14 23:46:44 +10:00
if (!NAME_FETCH_AAAA(adbname) && !FIND_STATICSTUB(find))
{
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
wanted_fetches |= DNS_ADBFIND_INET6;
}
break;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
}
Fix a lot of negative caching issues. I believe the code will use ncache responses from fetches and database lookups correctly now, but there needs to be a little more cleanup here and there. Bugs exist.
1999-11-24 19:23:27 +00:00
IPv6 query flood suppression
1999-12-14 00:10:06 +00:00
fetch:
update copyright notice
2008-02-07 23:46:54 +00:00
if ((WANT_INET(wanted_addresses) && NAME_HAS_V4(adbname)) ||
(WANT_INET6(wanted_addresses) && NAME_HAS_V6(adbname)))
{
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
have_address = true;
update copyright notice
2008-02-07 23:46:54 +00:00
} else {
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
have_address = false;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Reduce the number of fetches we make when looking up addresses If there are more that 5 NS record for a zone only perform a maximum of 4 address lookups for all the name servers. This limits the amount of remote lookup performed for server addresses at each level for a given query.
2020-02-06 17:19:10 +11:00
if (wanted_fetches != 0 && !(FIND_AVOIDFETCHES(find) && have_address) &&
!FIND_NOFETCH(find))
{
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
bool no_validate = FIND_NOVALIDATE(find);
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* We're missing at least one address family. Either the
* caller hasn't instructed us to avoid fetches, or we don't
* know anything about any of the address families that would
* be acceptable so we have to launch fetches.
*/
if (FIND_STARTATZONE(find)) {
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
start_at_zone = true;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Start V4.
*/
if (WANT_INET(wanted_fetches) &&
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
fetch_name(adbname, start_at_zone, no_validate, depth, qc,
gqc, dns_rdatatype_a) == ISC_R_SUCCESS)
checkpoint
2018-09-28 16:51:55 +10:00
{
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
DP(DEF_LEVEL,
"dns_adb_createfind: "
"started A fetch for name %s (%p)",
namebuf, adbname);
update copyright notice
2008-02-07 23:46:54 +00:00
}
/*
* Start V6.
*/
if (WANT_INET6(wanted_fetches) &&
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
fetch_name(adbname, start_at_zone, no_validate, depth, qc,
gqc, dns_rdatatype_aaaa) == ISC_R_SUCCESS)
checkpoint
2018-09-28 16:51:55 +10:00
{
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
DP(DEF_LEVEL,
"dns_adb_createfind: "
"started AAAA fetch for name %s (%p)",
namebuf, adbname);
update copyright notice
2008-02-07 23:46:54 +00:00
}
}
/*
* Run through the name and copy out the bits we are
* interested in.
*/
Remove dns_adblameinfo from dns_adb Keeping the information about lame server in the ADB was done in !322 to fix following security issue: [CVE-2021-25219] Disable "lame-ttl" cache The handling of the lame servers needs to be redesigned and it is not going to be enabled any time soon, and the current code is just dead code that takes up space, code and stands in the way of making ADB work faster. Remove all the internals needed for handling the lame servers in the ADB for now. It might get reintroduced later if and when we redesign ADB.
2023-09-21 15:20:58 +02:00
copy_namehook_lists(adb, find, adbname);
clean up from Bob's quick code walk-through
1999-10-21 01:37:57 +00:00
always set find->query_pending correctly
2000-01-27 02:14:53 +00:00
post_copy:
incidental cleanup The NAME_FETCH_A and NAME_FETCH_AAAA macros were meant to be boolean, indicating whether the pointers were set or not, while the NAME_FETCH_V4 and NAME_FETCH_V6 macros were meant to return the pointer values. The latter were only used as booleans, so they've been removed in favor of the former. Also did some style cleanup and removed an unreachable code block.
2021-10-18 17:57:58 -07:00
if (NAME_FETCH_A(adbname)) {
update copyright notice
2008-02-07 23:46:54 +00:00
query_pending |= DNS_ADBFIND_INET;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
incidental cleanup The NAME_FETCH_A and NAME_FETCH_AAAA macros were meant to be boolean, indicating whether the pointers were set or not, while the NAME_FETCH_V4 and NAME_FETCH_V6 macros were meant to return the pointer values. The latter were only used as booleans, so they've been removed in favor of the former. Also did some style cleanup and removed an unreachable code block.
2021-10-18 17:57:58 -07:00
if (NAME_FETCH_AAAA(adbname)) {
update copyright notice
2008-02-07 23:46:54 +00:00
query_pending |= DNS_ADBFIND_INET6;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
add find->result_v[46]
2000-08-26 02:21:45 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Attach to the name's query list if there are queries
* already running, and we have been asked to.
*/
if (!FIND_WANTEVENT(find)) {
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
want_event = false;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
if (FIND_WANTEMPTYEVENT(find) && FIND_HAS_ADDRS(find)) {
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
want_event = false;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
if ((wanted_addresses & query_pending) == 0) {
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
want_event = false;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
if (alias) {
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
want_event = false;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
if (want_event) {
incidental cleanup The NAME_FETCH_A and NAME_FETCH_AAAA macros were meant to be boolean, indicating whether the pointers were set or not, while the NAME_FETCH_V4 and NAME_FETCH_V6 macros were meant to return the pointer values. The latter were only used as booleans, so they've been removed in favor of the former. Also did some style cleanup and removed an unreachable code block.
2021-10-18 17:57:58 -07:00
bool empty;
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
update copyright notice
2008-02-07 23:46:54 +00:00
find->adbname = adbname;
incidental cleanup The NAME_FETCH_A and NAME_FETCH_AAAA macros were meant to be boolean, indicating whether the pointers were set or not, while the NAME_FETCH_V4 and NAME_FETCH_V6 macros were meant to return the pointer values. The latter were only used as booleans, so they've been removed in favor of the former. Also did some style cleanup and removed an unreachable code block.
2021-10-18 17:57:58 -07:00
empty = ISC_LIST_EMPTY(adbname->finds);
update copyright notice
2008-02-07 23:46:54 +00:00
ISC_LIST_APPEND(adbname->finds, find, plink);
find->query_pending = (query_pending & wanted_addresses);
find->flags &= ~DNS_ADBFIND_ADDRESSMASK;
find->flags |= (find->query_pending & DNS_ADBFIND_ADDRESSMASK);
incidental cleanup The NAME_FETCH_A and NAME_FETCH_AAAA macros were meant to be boolean, indicating whether the pointers were set or not, while the NAME_FETCH_V4 and NAME_FETCH_V6 macros were meant to return the pointer values. The latter were only used as booleans, so they've been removed in favor of the former. Also did some style cleanup and removed an unreachable code block.
2021-10-18 17:57:58 -07:00
DP(DEF_LEVEL, "createfind: attaching find %p to adbname %p %d",
checkpoint
2018-09-28 16:51:55 +10:00
find, adbname, empty);
update copyright notice
2008-02-07 23:46:54 +00:00
} else {
/*
* Remove the flag so the caller knows there will never
* be an event, and set internal flags to fake that
* the event was sent and freed, so dns_adb_destroyfind() will
* do the right thing.
*/
find->query_pending = (query_pending & wanted_addresses);
find->options &= ~DNS_ADBFIND_WANTEVENT;
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
find->flags |= FIND_EVENT_SENT;
update copyright notice
2008-02-07 23:46:54 +00:00
find->flags &= ~DNS_ADBFIND_ADDRESSMASK;
}
snapshot
1999-10-08 23:12:21 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
find->partial_result |= (adbname->partial_result & wanted_addresses);
if (alias) {
result = DNS_R_ALIAS;
} else {
result = ISC_R_SUCCESS;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
snapshot; everything works other than fetches and database lookups.
1999-10-16 00:38:21 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Copy out error flags from the name structure into the find.
*/
find->result_v4 = find_err_map[adbname->fetch_err];
find->result_v6 = find_err_map[adbname->fetch6_err];
snapshot; everything works other than fetches and database lookups.
1999-10-16 00:38:21 +00:00
Remove unnecessary NULL test leading to REVERSE_INULL false positive *** CID 351371: Null pointer dereferences (REVERSE_INULL) /lib/dns/adb.c: 2615 in dns_adb_createfind() 2609 /* 2610 * Copy out error flags from the name structure into the find. 2611 */ 2612 find->result_v4 = find_err_map[adbname->fetch_err]; 2613 find->result_v6 = find_err_map[adbname->fetch6_err]; 2614 >>> CID 351371: Null pointer dereferences (REVERSE_INULL) >>> Null-checking "find" suggests that it may be null, but it has already been dereferenced on all paths leading to the check. 2615 if (find != NULL) { 2616 if (want_event) { 2617 INSIST((find->flags & DNS_ADBFIND_ADDRESSMASK) != 0); 2618 isc_task_attach(task, &(isc_task_t *){ NULL }); 2619 find->event.ev_sender = task; 2620 find->event.ev_action = action;
2022-04-06 12:54:08 +10:00
if (want_event) {
INSIST((find->flags & DNS_ADBFIND_ADDRESSMASK) != 0);
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
find->loop = loop;
Use atomics to access find->status
2024-07-16 13:36:54 +10:00
atomic_store(&find->status, DNS_ADB_UNSET);
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
find->cb = cb;
find->cbarg = cbarg;
update copyright notice
2008-02-07 23:46:54 +00:00
}
Remove unnecessary NULL test leading to REVERSE_INULL false positive *** CID 351371: Null pointer dereferences (REVERSE_INULL) /lib/dns/adb.c: 2615 in dns_adb_createfind() 2609 /* 2610 * Copy out error flags from the name structure into the find. 2611 */ 2612 find->result_v4 = find_err_map[adbname->fetch_err]; 2613 find->result_v6 = find_err_map[adbname->fetch6_err]; 2614 >>> CID 351371: Null pointer dereferences (REVERSE_INULL) >>> Null-checking "find" suggests that it may be null, but it has already been dereferenced on all paths leading to the check. 2615 if (find != NULL) { 2616 if (want_event) { 2617 INSIST((find->flags & DNS_ADBFIND_ADDRESSMASK) != 0); 2618 isc_task_attach(task, &(isc_task_t *){ NULL }); 2619 find->event.ev_sender = task; 2620 find->event.ev_action = action;
2022-04-06 12:54:08 +10:00
*findp = find;
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&adbname->lock);
dns_adbname_detach(&adbname);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_unlock();
update copyright notice
2008-02-07 23:46:54 +00:00
return result;
add dns_adb_refresh()
1999-09-25 01:56:10 +00:00
}
Start on address.c
1999-09-21 22:46:42 +00:00
void
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
dns_adb_destroyfind(dns_adbfind_t **findp) {
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adbfind_t *find = NULL;
dns_adb_t *adb = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
REQUIRE(findp != NULL && DNS_ADBFIND_VALID(*findp));
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
update copyright notice
2008-02-07 23:46:54 +00:00
find = *findp;
*findp = NULL;
DP(DEF_LEVEL, "dns_adb_destroyfind on find %p", find);
adb = find->adb;
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
LOCK(&find->lock);
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
REQUIRE(find->adbname == NULL);
update copyright notice
2008-02-07 23:46:54 +00:00
/*
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* Free the addrinfo objects on the find's list. Note that
* we also need to decrement the reference counter in the
* associated adbentry every time we remove one from the list.
update copyright notice
2008-02-07 23:46:54 +00:00
*/
make all ISC_LIST_FOREACH calls safe previously, ISC_LIST_FOREACH and ISC_LIST_FOREACH_SAFE were two separate macros, with the _SAFE version allowing entries to be unlinked during the loop. ISC_LIST_FOREACH is now also safe, and the separate _SAFE macro has been removed. similarly, the ISC_LIST_FOREACH_REV macro is now safe, and ISC_LIST_FOREACH_REV_SAFE has also been removed.
2025-05-23 13:02:22 -07:00
ISC_LIST_FOREACH(find->list, ai, publink) {
update copyright notice
2008-02-07 23:46:54 +00:00
ISC_LIST_UNLINK(find->list, ai, publink);
free_adbaddrinfo(adb, &ai);
}
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
UNLOCK(&find->lock);
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
free_adbfind(&find);
Start on address.c
1999-09-21 22:46:42 +00:00
}
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
/*
* Caller must hold find lock.
*/
static void
find_sendevent(dns_adbfind_t *find) {
if (!FIND_EVENTSENT(find)) {
Use atomics to access find->status
2024-07-16 13:36:54 +10:00
atomic_store(&find->status, DNS_ADB_CANCELED);
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
DP(DEF_LEVEL, "sending find %p to caller", find);
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
isc_async_run(find->loop, find->cb, find);
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
}
}
implement dns_adb_cancelfind(), rename _done() to _destroyfind(), and _lookup to _createfind()
1999-10-29 18:00:31 +00:00
void
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
dns_adb_cancelfind(dns_adbfind_t *find) {
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
dns_adbname_t *adbname = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
DP(DEF_LEVEL, "dns_adb_cancelfind on find %p", find);
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
REQUIRE(DNS_ADBFIND_VALID(find));
REQUIRE(DNS_ADB_VALID(find->adb));
update copyright notice
2008-02-07 23:46:54 +00:00
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
LOCK(&find->lock);
update copyright notice
2008-02-07 23:46:54 +00:00
REQUIRE(FIND_WANTEVENT(find));
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
adbname = find->adbname;
update copyright notice
2008-02-07 23:46:54 +00:00
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
if (adbname == NULL) {
find_sendevent(find);
UNLOCK(&find->lock);
} else {
/*
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
* Release the find lock, then acquire the name and find
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
* locks in that order, to match locking hierarchy
* elsewhere.
*/
Revalidate the adbname when canceling the ADB find When canceling the ADB find, the lock on the find gets released for a brief period of time to be locked again inside adbname lock. During the brief period that the ADB find is unlocked, it can get canceled by other means removing it from the adbname list which in turn causes assertion failure due to a double removal from the adbname list. Recheck if the find->adbname is still valid after acquiring the lock again and if not just skip the double removal. Additionally, attach to the adbname as in the worst case, the adbname might also cease to exist if the scheduler would block this particular thread for a longer period of time invalidating the lock we are going to acquire and release.
2024-11-05 14:59:16 +01:00
dns_adbname_ref(adbname);
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
UNLOCK(&find->lock);
Revalidate the adbname when canceling the ADB find When canceling the ADB find, the lock on the find gets released for a brief period of time to be locked again inside adbname lock. During the brief period that the ADB find is unlocked, it can get canceled by other means removing it from the adbname list which in turn causes assertion failure due to a double removal from the adbname list. Recheck if the find->adbname is still valid after acquiring the lock again and if not just skip the double removal. Additionally, attach to the adbname as in the worst case, the adbname might also cease to exist if the scheduler would block this particular thread for a longer period of time invalidating the lock we are going to acquire and release.
2024-11-05 14:59:16 +01:00
/*
* Other thread could cancel the find between the unlock and
* lock, so we need to recheck whether the adbname is still
* valid and reference the adbname, so it does not vanish before
* we have a chance to lock it again.
*/
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
LOCK(&adbname->lock);
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
LOCK(&find->lock);
implement dns_adb_cancelfind(), rename _done() to _destroyfind(), and _lookup to _createfind()
1999-10-29 18:00:31 +00:00
Revalidate the adbname when canceling the ADB find When canceling the ADB find, the lock on the find gets released for a brief period of time to be locked again inside adbname lock. During the brief period that the ADB find is unlocked, it can get canceled by other means removing it from the adbname list which in turn causes assertion failure due to a double removal from the adbname list. Recheck if the find->adbname is still valid after acquiring the lock again and if not just skip the double removal. Additionally, attach to the adbname as in the worst case, the adbname might also cease to exist if the scheduler would block this particular thread for a longer period of time invalidating the lock we are going to acquire and release.
2024-11-05 14:59:16 +01:00
if (find->adbname != NULL) {
ISC_LIST_UNLINK(find->adbname->finds, find, plink);
find->adbname = NULL;
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
find_sendevent(find);
implement dns_adb_cancelfind(), rename _done() to _destroyfind(), and _lookup to _createfind()
1999-10-29 18:00:31 +00:00
lock find when unlinking adbname->finds in dns_adb_cancelfind() In dns_adb_cancelfind(), we need to release the find lock and then acquire the bucket and find locks in that order, for consistency with locking hierarchy elsehwere. Previously we were only acquiring the bucket lock. Also rewrote the function for better readability.
2022-04-19 19:14:49 -07:00
UNLOCK(&find->lock);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&adbname->lock);
Revalidate the adbname when canceling the ADB find When canceling the ADB find, the lock on the find gets released for a brief period of time to be locked again inside adbname lock. During the brief period that the ADB find is unlocked, it can get canceled by other means removing it from the adbname list which in turn causes assertion failure due to a double removal from the adbname list. Recheck if the find->adbname is still valid after acquiring the lock again and if not just skip the double removal. Additionally, attach to the adbname as in the worst case, the adbname might also cease to exist if the scheduler would block this particular thread for a longer period of time invalidating the lock we are going to acquire and release.
2024-11-05 14:59:16 +01:00
dns_adbname_detach(&adbname);
update copyright notice
2008-02-07 23:46:54 +00:00
}
implement dns_adb_cancelfind(), rename _done() to _destroyfind(), and _lookup to _createfind()
1999-10-29 18:00:31 +00:00
}
Use an accessor fuction to access find->status find->status is marked as private and access is controlled by find->lock.
2024-08-28 11:37:05 +10:00
unsigned int
dns_adb_findstatus(dns_adbfind_t *find) {
REQUIRE(DNS_ADBFIND_VALID(find));
Use atomics to access find->status
2024-07-16 13:36:54 +10:00
return atomic_load(&find->status);
Use an accessor fuction to access find->status find->status is marked as private and access is controlled by find->lock.
2024-08-28 11:37:05 +10:00
}
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
void
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
dns_adb_dump(dns_adb_t *adb, FILE *f) {
Apply the semantic patch to remove isc_stdtime_get() This is a simple replacement using the semantic patch from the previous commit and as added bonus, one removal of previously undetected unused variable in named/server.c.
2023-03-30 21:13:41 +02:00
isc_stdtime_t now = isc_stdtime_now();
1761. [bug] 'rndc dumpdb' didn't report unassociated entries. [RT #12971]
2004-11-10 22:33:18 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(f != NULL);
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_lock();
if (atomic_load(&adb->shuttingdown)) {
rcu_read_unlock();
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
return;
}
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
cleanup_names(adb, now);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
cleanup_entries(adb, now);
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
dump_adb(adb, f, false, now);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_unlock();
snapshot; nearly all fetch infrastructure is in place.
1999-10-27 19:36:58 +00:00
}
static void
dump the ADB in a more user-friendly format (unless a debug flag is set)
2000-12-18 20:03:33 +00:00
dump_ttl(FILE *f, const char *legend, isc_stdtime_t value, isc_stdtime_t now) {
update copyright notice
2008-02-07 23:46:54 +00:00
if (value == INT_MAX) {
return;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
formally cast to int
2018-02-14 19:13:56 +11:00
fprintf(f, " [%s TTL %d]", legend, (int)(value - now));
dump the ADB in a more user-friendly format (unless a debug flag is set)
2000-12-18 20:03:33 +00:00
}
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/*
* Both rwlocks for the hash tables need to be held by the caller.
*/
dump the ADB in a more user-friendly format (unless a debug flag is set)
2000-12-18 20:03:33 +00:00
static void
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
dump_adb(dns_adb_t *adb, FILE *f, bool debug, isc_stdtime_t now) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
struct cds_lfht_iter iter;
update copyright notice
2008-02-07 23:46:54 +00:00
fprintf(f, ";\n; Address database dump\n;\n");
Simplify the EDNS buffer size logic for DNS Flag Day 2020 The DNS Flag Day 2020 aims to remove the IP fragmentation problem from the UDP DNS communication. In this commit, we implement the required changes and simplify the logic for picking the EDNS Buffer Size. 1. The defaults for `edns-udp-size`, `max-udp-size` and `nocookie-udp-size` have been changed to `1232` (the value picked by DNS Flag Day 2020). 2. The probing heuristics that would try 512->4096->1432->1232 buffer sizes has been removed and the resolver will always use just the `edns-udp-size` value. 3. Instead of just disabling the PMTUD mechanism on the UDP sockets, we now set IP_DONTFRAG (IPV6_DONTFRAG) flag. That means that the UDP packets won't get ever fragmented. If the ICMP packets are lost the UDP will just timeout and eventually be retried over TCP.
2020-09-23 14:47:26 +02:00
fprintf(f, "; [edns success/timeout]\n");
add partial legend for Address database dump
2013-11-13 12:10:43 +11:00
fprintf(f, "; [plain success/timeout]\n;\n");
Fix missing adb->{e,i}refcnt locking
2019-11-19 13:49:28 +08:00
if (debug) {
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
fprintf(f, "; addr %p, references %" PRIuFAST32 "\n", adb,
isc_refcount_current(&adb->references));
Fix missing adb->{e,i}refcnt locking
2019-11-19 13:49:28 +08:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
/*
* Ensure this operation is applied to both hash tables at once.
*/
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbname_t *adbname = NULL;
cds_lfht_for_each_entry(adb->names_ht, &iter, adbname, ht_node) {
LOCK(&adbname->lock);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
/*
* Dump the names
*/
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (debug) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
fprintf(f, "; name %p (flags %08x)\n", adbname,
adbname->flags);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
}
fprintf(f, "; ");
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_name_print(adbname->name, f);
update copyright notice
2008-02-07 23:46:54 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dump_ttl(f, "v4", adbname->expire_v4, now);
dump_ttl(f, "v6", adbname->expire_v6, now);
update copyright notice
2008-02-07 23:46:54 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
fprintf(f, " [v4 %s] [v6 %s]", errnames[adbname->fetch_err],
errnames[adbname->fetch6_err]);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
fprintf(f, "\n");
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
print_namehook_list(f, "v4", adb, &adbname->v4, debug, now);
print_namehook_list(f, "v6", adb, &adbname->v6, debug, now);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (debug) {
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
print_fetch_list(f, adbname);
print_find_list(f, adbname);
update copyright notice
2008-02-07 23:46:54 +00:00
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
UNLOCK(&adbname->lock);
lib/dns/adb.c: don't use more than 64 lock simultaneously when run under TSAN - TSAN can't handle more than 64 locks in one thread, lock ADB bucket-by-bucket in TSAN mode. This means that the dump won't be consistent but it's good enough for testing - Use proper order when unlocking adb->namelocks and adb->entrylocks when dumping ADB.
2019-01-22 10:47:18 +01:00
}
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbentry_t *adbentry = NULL;
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
fprintf(f, ";\n; Unassociated entries\n;\n");
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
cds_lfht_for_each_entry(adb->entries_ht, &iter, adbentry, ht_node) {
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
LOCK(&adbentry->lock);
if (ISC_LIST_EMPTY(adbentry->nhs)) {
dump_entry(f, adb, adbentry, debug, now);
}
UNLOCK(&adbentry->lock);
}
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
}
dump the ADB in a more user-friendly format (unless a debug flag is set)
2000-12-18 20:03:33 +00:00
static void
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
dump_entry(FILE *f, dns_adb_t *adb, dns_adbentry_t *entry, bool debug,
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
isc_stdtime_t now) {
update copyright notice
2008-02-07 23:46:54 +00:00
char addrbuf[ISC_NETADDR_FORMATSIZE];
isc_netaddr_t netaddr;
isc_netaddr_fromsockaddr(&netaddr, &entry->sockaddr);
isc_netaddr_format(&netaddr, addrbuf, sizeof(addrbuf));
if (debug) {
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
fprintf(f, ";\t%p: refcnt %" PRIuFAST32 "\n", entry,
isc_refcount_current(&entry->references));
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
fprintf(f,
Simplify the EDNS buffer size logic for DNS Flag Day 2020 The DNS Flag Day 2020 aims to remove the IP fragmentation problem from the UDP DNS communication. In this commit, we implement the required changes and simplify the logic for picking the EDNS Buffer Size. 1. The defaults for `edns-udp-size`, `max-udp-size` and `nocookie-udp-size` have been changed to `1232` (the value picked by DNS Flag Day 2020). 2. The probing heuristics that would try 512->4096->1432->1232 buffer sizes has been removed and the resolver will always use just the `edns-udp-size` value. 3. Instead of just disabling the PMTUD mechanism on the UDP sockets, we now set IP_DONTFRAG (IPV6_DONTFRAG) flag. That means that the UDP packets won't get ever fragmented. If the ICMP packets are lost the UDP will just timeout and eventually be retried over TCP.
2020-09-23 14:47:26 +02:00
";\t%s [srtt %u] [flags %08x] [edns %u/%u] "
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
"[plain %u/%u]",
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
addrbuf, atomic_load(&entry->srtt), atomic_load(&entry->flags),
entry->edns, entry->ednsto, entry->plain, entry->plainto);
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
if (entry->udpsize != 0U) {
fprintf(f, " [udpsize %u]", entry->udpsize);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
4152. [func] Implement DNS COOKIE option. This replaces the experimental SIT option of BIND 9.10. The following named.conf directives are avaliable: send-cookie, cookie-secret, cookie-algorithm and nocookie-udp-size. The following dig options are available: +[no]cookie[=value] and +[no]badcookie. [RT #39928]
2015-07-06 09:44:24 +10:00
if (entry->cookie != NULL) {
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
unsigned int i;
4152. [func] Implement DNS COOKIE option. This replaces the experimental SIT option of BIND 9.10. The following named.conf directives are avaliable: send-cookie, cookie-secret, cookie-algorithm and nocookie-udp-size. The following dig options are available: +[no]cookie[=value] and +[no]badcookie. [RT #39928]
2015-07-06 09:44:24 +10:00
fprintf(f, " [cookie=");
for (i = 0; i < entry->cookielen; i++) {
fprintf(f, "%02x", entry->cookie[i]);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
fprintf(f, "]");
}
Always set ADB entry expiration to now + ADB_ENTRY_WINDOW When ADB entry was created it was set to never expire. If we never called any of the functions that adjust the expiration, it could linger in the ADB forever. Set the expiration (.expires) to now + ADB_ENTRY_WINDOW when creating the new ADB entry to ensure the ADB entry will always expire.
2024-04-03 11:33:48 +02:00
fprintf(f, " [ttl %d]", entry->expires - now);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
if (adb != NULL && adb->quota != 0 && adb->atr_freq != 0) {
Workaround MSVC warning C4477 Due to a way the stdatomic.h shim is implemented on Windows, the MSVC always things that the outside type is the largest - atomic_(u)int_fast64_t. This can lead to false positives as this one: lib\dns\adb.c(3678): warning C4477: 'fprintf' : format string '%u' requires an argument of type 'unsigned int', but variadic argument 2 has type 'unsigned __int64' We workaround the issue by loading the value in a scoped local variable with correct type first.
2020-04-15 12:36:12 +02:00
uint_fast32_t quota = atomic_load_relaxed(&entry->quota);
lib/dns/adb.c: Use atomics for adb quota values and reference counting
2019-01-22 10:47:18 +01:00
fprintf(f, " [atr %0.2f] [quota %" PRIuFAST32 "]", entry->atr,
Workaround MSVC warning C4477 Due to a way the stdatomic.h shim is implemented on Windows, the MSVC always things that the outside type is the largest - atomic_(u)int_fast64_t. This can lead to false positives as this one: lib\dns\adb.c(3678): warning C4477: 'fprintf' : format string '%u' requires an argument of type 'unsigned int', but variadic argument 2 has type 'unsigned __int64' We workaround the issue by loading the value in a scoped local variable with correct type first.
2020-04-15 12:36:12 +02:00
quota);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
fprintf(f, "\n");
dump the ADB in a more user-friendly format (unless a debug flag is set)
2000-12-18 20:03:33 +00:00
}
clean up unused API the dns_adb_dumpfind() function was only used inside adb.c and can be static. dns_view_dumpdbtostream() was not used anywhere.
2022-05-25 12:02:18 -07:00
static void
dumpfind(dns_adbfind_t *find, FILE *f) {
update copyright notice
2008-02-07 23:46:54 +00:00
char tmp[512];
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
const char *tmpp = NULL;
isc_sockaddr_t *sa = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Not used currently, in the API Just In Case we
* want to dump out the name and/or entries too.
*/
LOCK(&find->lock);
fprintf(f, ";Find %p\n", find);
fprintf(f, ";\tqpending %08x partial %08x options %08x flags %08x\n",
find->query_pending, find->partial_result, find->options,
find->flags);
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
fprintf(f, ";\tname %p\n", find->adbname);
update copyright notice
2008-02-07 23:46:54 +00:00
use ISC_LIST_FOREACH in more places use the ISC_LIST_FOREACH pattern in places where lists had been iterated using a different pattern from the typical `for` loop: for example, `while (!ISC_LIST_EMPTY(...))` or `while ((e = ISC_LIST_HEAD(...)) != NULL)`.
2025-03-22 15:26:16 -07:00
if (!ISC_LIST_EMPTY(find->list)) {
update copyright notice
2008-02-07 23:46:54 +00:00
fprintf(f, "\tAddresses:\n");
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
use ISC_LIST_FOREACH in more places use the ISC_LIST_FOREACH pattern in places where lists had been iterated using a different pattern from the typical `for` loop: for example, `while (!ISC_LIST_EMPTY(...))` or `while ((e = ISC_LIST_HEAD(...)) != NULL)`.
2025-03-22 15:26:16 -07:00
ISC_LIST_FOREACH(find->list, ai, publink) {
update copyright notice
2008-02-07 23:46:54 +00:00
sa = &ai->sockaddr;
switch (sa->type.sa.sa_family) {
case AF_INET:
tmpp = inet_ntop(AF_INET, &sa->type.sin.sin_addr, tmp,
sizeof(tmp));
break;
case AF_INET6:
tmpp = inet_ntop(AF_INET6, &sa->type.sin6.sin6_addr,
tmp, sizeof(tmp));
break;
default:
tmpp = "UnkFamily";
}
if (tmpp == NULL) {
tmpp = "BadAddress";
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
fprintf(f,
"\t\tentry %p, flags %08x"
" srtt %u addr %s\n",
ai->entry, ai->flags, ai->srtt, tmpp);
}
UNLOCK(&find->lock);
snapshot; everything works other than fetches and database lookups.
1999-10-16 00:38:21 +00:00
}
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
static void
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
print_namehook_list(FILE *f, const char *legend, dns_adb_t *adb,
dns_adbnamehooklist_t *list, bool debug,
isc_stdtime_t now) {
switch to ISC_LIST_FOREACH everywhere the pattern `for (x = ISC_LIST_HEAD(...); x != NULL; ISC_LIST_NEXT(...)` has been changed to `ISC_LIST_FOREACH` throughout BIND, except in a few cases where the change would be excessively complex. in most cases this was a straightforward change. in some places, however, the list element variable was referenced after the loop ended, and the code was refactored to avoid this necessity. also, because `ISC_LIST_FOREACH` uses typeof(list.head) to declare the list elements, compilation failures can occur if the list object has a `const` qualifier. some `const` qualifiers have been removed from function parameters to avoid this problem, and where that was not possible, `UNCONST` was used.
2025-03-20 22:25:56 -07:00
ISC_LIST_FOREACH(*list, nh, name_link) {
update copyright notice
2008-02-07 23:46:54 +00:00
if (debug) {
fprintf(f, ";\tHook(%s) %p\n", legend, nh);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
LOCK(&nh->entry->lock);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
dump_entry(f, adb, nh->entry, debug, now);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&nh->entry->lock);
update copyright notice
2008-02-07 23:46:54 +00:00
}
Many changes. o added adb_test.c to test the functions already written. o added code to delete names and all associated addresses. o added a database dumper which will currently print the adb in a human (or at least developer) readable format. o found and fixed many twisted little locking issues.
1999-10-02 01:59:09 +00:00
}
snapshot; everything works other than fetches and database lookups.
1999-10-16 00:38:21 +00:00
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Megacommit of many files. Mostly, several functions that take pointers as arguments, almost always char * pointers, had those pointers qualified with "const". Those that returned pointers to previously const-qualified arguments had their return values qualified as const. Some structure members were qualified as const to retain that attribute from the variables from which they were assigned. The macro DE_CONST is used to deal with a handful of very special places where something is qualified as const but really needs to have its const qualifier removed. rdata.c now defines macros for the prototypes of the basic rdata functions, and all of the lib/dns/rdata/**/*.c files now use them. Some minor integer-compatibility issues. (IE, ~0x03 is a signed int, so assigning it to an unsigned int should use a cast. The type of an enum member is int, so there are some conversion issues there, too.) A pointers-to-function should not be cast to a pointer-to-object. Variables should not be named for C reserved identifiers. One or two set-but-not-used variables removed. Minor other ISC style cleanups.
2000-06-01 18:26:56 +00:00
print_fetch(FILE *f, dns_adbfetch_t *ft, const char *type) {
2544. [cleanup] Removed unused structure members in adb.c. [RT #19225]
2009-02-03 22:33:13 +00:00
fprintf(f, "\t\tFetch(%s): %p -> { fetch %p }\n", type, ft, ft->fetch);
kill name->query_pending, since that bitmask is really represented more accurately by other means. Reverse the sense of fetch checking from NO_FETCH_A() and NO_FETCH_AAAA(), etc to NAME_FETCH_A(), NAME_FETCH_AAAA(), and add address family checks like NO_FETCH_V4() and _V6(). Fix a database dump bug where the A fetches were printed twice rather than printing the AAAA fetches. Add printing of A6 fetches.
1999-11-02 18:36:54 +00:00
}
fetch change: A -> pointer, AAAA -> pointer, A6 -> list
1999-10-29 21:48:51 +00:00
static void
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
print_fetch_list(FILE *f, dns_adbname_t *n) {
update copyright notice
2008-02-07 23:46:54 +00:00
if (NAME_FETCH_A(n)) {
print_fetch(f, n->fetch_a, "A");
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
if (NAME_FETCH_AAAA(n)) {
print_fetch(f, n->fetch_aaaa, "AAAA");
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
snapshot; nearly all fetch infrastructure is in place.
1999-10-27 19:36:58 +00:00
}
snapshot; everything works other than fetches and database lookups.
1999-10-16 00:38:21 +00:00
static void
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
print_find_list(FILE *f, dns_adbname_t *name) {
use ISC_LIST_FOREACH in more places use the ISC_LIST_FOREACH pattern in places where lists had been iterated using a different pattern from the typical `for` loop: for example, `while (!ISC_LIST_EMPTY(...))` or `while ((e = ISC_LIST_HEAD(...)) != NULL)`.
2025-03-22 15:26:16 -07:00
ISC_LIST_FOREACH(name->finds, find, plink) {
clean up unused API the dns_adb_dumpfind() function was only used inside adb.c and can be static. dns_view_dumpdbtostream() was not used anywhere.
2022-05-25 12:02:18 -07:00
dumpfind(find, f);
update copyright notice
2008-02-07 23:46:54 +00:00
}
snapshot; everything works other than fetches and database lookups.
1999-10-16 00:38:21 +00:00
}
Add code to check for a lame zone, and to expire lame entries as needed.
1999-10-19 20:55:04 +00:00
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
static isc_result_t
putstr(isc_buffer_t **b, const char *str) {
isc_result_t result;
Change the isc_buffer_reserve() to take just buffer pointer The isc_buffer_reserve() would be passed a reference to the buffer pointer, which was unnecessary as the pointer would never be changed in the current implementation. Remove the extra dereference.
2022-12-15 11:54:51 +01:00
result = isc_buffer_reserve(*b, strlen(str));
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
if (result != ISC_R_SUCCESS) {
return result;
}
isc_buffer_putstr(*b, str);
return ISC_R_SUCCESS;
}
isc_result_t
dns_adb_dumpquota(dns_adb_t *adb, isc_buffer_t **buf) {
REQUIRE(DNS_ADB_VALID(adb));
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbentry_t *adbentry = NULL;
struct cds_lfht_iter iter;
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_lock();
if (atomic_load(&adb->shuttingdown)) {
rcu_read_unlock();
return ISC_R_SHUTTINGDOWN;
}
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
cds_lfht_for_each_entry(adb->entries_ht, &iter, adbentry, ht_node) {
LOCK(&adbentry->lock);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
char addrbuf[ISC_NETADDR_FORMATSIZE];
char text[ISC_NETADDR_FORMATSIZE + BUFSIZ];
isc_netaddr_t netaddr;
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
if (adbentry->atr == 0.0 && adbentry->quota == adb->quota) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
goto unlock;
}
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
isc_netaddr_fromsockaddr(&netaddr, &adbentry->sockaddr);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
isc_netaddr_format(&netaddr, addrbuf, sizeof(addrbuf));
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
snprintf(text, sizeof(text),
"\n- quota %s (%" PRIuFAST32 "/%d) atr %0.2f", addrbuf,
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
atomic_load_relaxed(&adbentry->quota), adb->quota,
adbentry->atr);
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
putstr(buf, text);
unlock:
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
UNLOCK(&adbentry->lock);
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_unlock();
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
Purge stale ADB names globaly, not per bucket Before the refactoring, there was only few buckets with many names in them, so cleaning up stale ADB names per-bucket made sense. After the refactoring, each bucket directly maps to ADB name, so purging has been effectively disabled. Create a global LRU list for ADB names (and ADB entries) and purge the stale ADB names globally.
2022-08-01 09:07:38 +02:00
return ISC_R_SUCCESS;
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
}
Add code to check for a lame zone, and to expire lame entries as needed.
1999-10-19 20:55:04 +00:00
static isc_result_t
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
dbfind_name(dns_adbname_t *adbname, isc_stdtime_t now, dns_rdatatype_t rdtype) {
update copyright notice
2008-02-07 23:46:54 +00:00
isc_result_t result;
dns_rdataset_t rdataset;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adb_t *adb = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
dns_fixedname_t foundname;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_name_t *fname = NULL;
Set PENDINGOK if STARTATZONE is set When there are parent and child zones on the same server, the DNSKEY lookup was failing as the pending record we are validating is needed to fetch the DNSKEY records. This change allows that to happen. The caller is already setting STARTATZONE when the name being looked up is a subdomain of the current domain.
2025-01-22 20:58:33 +11:00
unsigned int options = DNS_DBFIND_GLUEOK | DNS_DBFIND_ADDITIONALOK;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(DNS_ADBNAME_VALID(adbname));
update copyright notice
2008-02-07 23:46:54 +00:00
adb = adbname->adb;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(DNS_ADB_VALID(adb));
use new dns_rdatatype classification functions modify code to use dns_rdatatype_ismulti(), dns_rdatatype_issig(), dns_rdatatype_isaddr(), and dns_rdatatype_isalias() where applicable.
2025-03-04 15:51:49 -08:00
REQUIRE(dns_rdatatype_isaddr(rdtype));
update copyright notice
2008-02-07 23:46:54 +00:00
Use dns_fixedname_initname() where possible Replace dns_fixedname_init() calls followed by dns_fixedname_name() calls with calls to dns_fixedname_initname() where it is possible without affecting current behavior and/or performance. This patch was mostly prepared using Coccinelle and the following semantic patch: @@ expression fixedname, name; @@ - dns_fixedname_init(&fixedname); ... - name = dns_fixedname_name(&fixedname); + name = dns_fixedname_initname(&fixedname); The resulting set of changes was then manually reviewed to exclude false positives and apply minor tweaks. It is likely that more occurrences of this pattern can be refactored in an identical way. This commit only takes care of the low-hanging fruit.
2018-03-28 14:38:09 +02:00
fname = dns_fixedname_initname(&foundname);
update copyright notice
2008-02-07 23:46:54 +00:00
dns_rdataset_init(&rdataset);
if (rdtype == dns_rdatatype_a) {
adbname->fetch_err = FIND_ERR_UNEXPECTED;
} else {
adbname->fetch6_err = FIND_ERR_UNEXPECTED;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
2947. [func] Add new zone type "static-stub". It's like a stub zone, but the nameserver names and/or their IP addresses are statically configured. [RT #21474] (for 9.8.0)
2010-12-16 09:51:30 +00:00
/*
* We need to specify whether to search static-stub zones (if
* configured) depending on whether this is a "start at zone" lookup,
* i.e., whether it's a "bailiwick" glue. If it's bailiwick (in which
clean up numbering of FETCHOPT and ADDRINFO flags in the past there was overlap between the fields used as resolver fetch options and ADB addrinfo flags. this has mostly been eliminated; now we can clean up the rest of it and remove some confusing comments.
2023-07-03 12:43:17 -07:00
* case DNS_ADBFIND_STARTATZONE is set) we need to stop the search at
* any matching static-stub zone without looking into the cache to honor
2947. [func] Add new zone type "static-stub". It's like a stub zone, but the nameserver names and/or their IP addresses are statically configured. [RT #21474] (for 9.8.0)
2010-12-16 09:51:30 +00:00
* the configuration on which server we should send queries to.
*/
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
if ((adbname->type & DNS_ADBFIND_STARTATZONE) != 0) {
Set PENDINGOK if STARTATZONE is set When there are parent and child zones on the same server, the DNSKEY lookup was failing as the pending record we are validating is needed to fetch the DNSKEY records. This change allows that to happen. The caller is already setting STARTATZONE when the name being looked up is a subdomain of the current domain.
2025-01-22 20:58:33 +11:00
options |= DNS_DBFIND_PENDINGOK;
}
result = dns_view_find(adb->view, adbname->name, rdtype, now, options,
true,
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
(adbname->type & DNS_ADBFIND_STARTATZONE) != 0,
Set PENDINGOK if STARTATZONE is set When there are parent and child zones on the same server, the DNSKEY lookup was failing as the pending record we are validating is needed to fetch the DNSKEY records. This change allows that to happen. The caller is already setting STARTATZONE when the name being looked up is a subdomain of the current domain.
2025-01-22 20:58:33 +11:00
NULL, NULL, fname, &rdataset, NULL);
update copyright notice
2008-02-07 23:46:54 +00:00
switch (result) {
case DNS_R_GLUE:
case DNS_R_HINT:
import_rdataset() can't fail the import_rdataset() function can't return any value other than ISC_R_SUCCESS, so it's been changed to void and its callers don't rely on its return value any longer.
2025-03-21 23:48:01 -07:00
result = ISC_R_SUCCESS;
FALLTHROUGH;
update copyright notice
2008-02-07 23:46:54 +00:00
case ISC_R_SUCCESS:
/*
* Found in the database. Even if we can't copy out
* any information, return success, or else a fetch
* will be made, which will only make things worse.
*/
if (rdtype == dns_rdatatype_a) {
adbname->fetch_err = FIND_ERR_SUCCESS;
} else {
adbname->fetch6_err = FIND_ERR_SUCCESS;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
import_rdataset() can't fail the import_rdataset() function can't return any value other than ISC_R_SUCCESS, so it's been changed to void and its callers don't rely on its return value any longer.
2025-03-21 23:48:01 -07:00
import_rdataset(adbname, &rdataset, now);
update copyright notice
2008-02-07 23:46:54 +00:00
break;
case DNS_R_NXDOMAIN:
case DNS_R_NXRRSET:
/*
* We're authoritative and the data doesn't exist.
* Make up a negative cache entry so we don't ask again
* for a while.
*
* XXXRTH What time should we use? I'm putting in 30 seconds
* for now.
*/
if (rdtype == dns_rdatatype_a) {
adbname->expire_v4 = now + 30;
DP(NCACHE_LEVEL,
"adb name %p: Caching auth negative entry for A",
adbname);
if (result == DNS_R_NXDOMAIN) {
adbname->fetch_err = FIND_ERR_NXDOMAIN;
} else {
adbname->fetch_err = FIND_ERR_NXRRSET;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
} else {
DP(NCACHE_LEVEL,
"adb name %p: Caching auth negative entry for AAAA",
adbname);
adbname->expire_v6 = now + 30;
if (result == DNS_R_NXDOMAIN) {
adbname->fetch6_err = FIND_ERR_NXDOMAIN;
} else {
adbname->fetch6_err = FIND_ERR_NXRRSET;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
}
break;
case DNS_R_NCACHENXDOMAIN:
case DNS_R_NCACHENXRRSET:
/*
* We found a negative cache entry. Pull the TTL from it
* so we won't ask again for a while.
*/
rdataset.ttl = ttlclamp(rdataset.ttl);
if (rdtype == dns_rdatatype_a) {
adbname->expire_v4 = rdataset.ttl + now;
if (result == DNS_R_NCACHENXDOMAIN) {
adbname->fetch_err = FIND_ERR_NXDOMAIN;
} else {
adbname->fetch_err = FIND_ERR_NXRRSET;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
DP(NCACHE_LEVEL,
"adb name %p: Caching negative entry for A (ttl %u)",
adbname, rdataset.ttl);
} else {
DP(NCACHE_LEVEL,
"adb name %p: Caching negative entry for AAAA (ttl "
"%u)",
adbname, rdataset.ttl);
adbname->expire_v6 = rdataset.ttl + now;
if (result == DNS_R_NCACHENXDOMAIN) {
adbname->fetch6_err = FIND_ERR_NXDOMAIN;
} else {
adbname->fetch6_err = FIND_ERR_NXRRSET;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
}
break;
case DNS_R_CNAME:
case DNS_R_DNAME:
remove 'target' from dns_adb the target name parameter to dns_adb_createfind() was always passed as NULL, so we can safely remove it. relatedly, the 'target' field in the dns_adbname structure was never referenced after being set. the 'expire_target' field was used, but only as a way to check whether an ADB name represents a CNAME or DNAME, and that information can be stored as a single flag.
2025-02-21 01:24:42 -08:00
/*
* We found a CNAME or DNAME. Mark this as an
* alias (not to be used) and mark the expiry
* for both address families so we won't ask again
* for a while.
*/
update copyright notice
2008-02-07 23:46:54 +00:00
rdataset.ttl = ttlclamp(rdataset.ttl);
remove 'target' from dns_adb the target name parameter to dns_adb_createfind() was always passed as NULL, so we can safely remove it. relatedly, the 'target' field in the dns_adbname structure was never referenced after being set. the 'expire_target' field was used, but only as a way to check whether an ADB name represents a CNAME or DNAME, and that information can be stored as a single flag.
2025-02-21 01:24:42 -08:00
result = DNS_R_ALIAS;
adbname->flags |= NAME_IS_ALIAS;
adbname->expire_v4 = adbname->expire_v6 =
ADJUSTED_EXPIRE(INT_MAX, now, rdataset.ttl);
update copyright notice
2008-02-07 23:46:54 +00:00
if (rdtype == dns_rdatatype_a) {
adbname->fetch_err = FIND_ERR_SUCCESS;
} else {
adbname->fetch6_err = FIND_ERR_SUCCESS;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
break;
Make isc_result a static enum Remove the dynamic registration of result codes. Convert isc_result_t from unsigned + #defines into 32-bit enum type in grand unified <isc/result.h> header. Keep the existing values of the result codes even at the expense of the description and identifier tables being unnecessary large. Additionally, add couple of: switch (result) { [...] default: break; } statements where compiler now complains about missing enum values in the switch statement.
2021-10-04 17:14:53 +02:00
default:
break;
update copyright notice
2008-02-07 23:46:54 +00:00
}
if (dns_rdataset_isassociated(&rdataset)) {
dns_rdataset_disassociate(&rdataset);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
return result;
ipv6 AAAA queries work now.
1999-10-29 19:20:36 +00:00
}
snapshot; nearly all fetch infrastructure is in place.
1999-10-27 19:36:58 +00:00
static void
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
fetch_callback(void *arg) {
dns_fetchresponse_t *resp = (dns_fetchresponse_t *)arg;
dns_adbname_t *name = resp->arg;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adb_t *adb = NULL;
dns_adbfetch_t *fetch = NULL;
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
dns_adbstatus_t astat = DNS_ADB_NOMOREADDRESSES;
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
isc_stdtime_t now = isc_stdtime_now();
update copyright notice
2008-02-07 23:46:54 +00:00
unsigned int address_type;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(DNS_ADBNAME_VALID(name));
remove ADB whenshutdown events weakly attaching and detaching the view when creating or destroying the ADB obviates the need for a whenshutdown callback event to do the detaching. remove the dns_adb_whenshutdown() mechanism, since it is no longer needed.
2022-05-11 13:55:01 -07:00
dns_adb_attach(name->adb, &adb);
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(DNS_ADB_VALID(adb));
update copyright notice
2008-02-07 23:46:54 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_lock();
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
LOCK(&name->lock);
update copyright notice
2008-02-07 23:46:54 +00:00
INSIST(NAME_FETCH_A(name) || NAME_FETCH_AAAA(name));
address_type = 0;
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
if (NAME_FETCH_A(name) && (name->fetch_a->fetch == resp->fetch)) {
update copyright notice
2008-02-07 23:46:54 +00:00
address_type = DNS_ADBFIND_INET;
fetch = name->fetch_a;
name->fetch_a = NULL;
} else if (NAME_FETCH_AAAA(name) &&
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
(name->fetch_aaaa->fetch == resp->fetch))
Update sources to Clang 15 formatting
2022-11-02 19:33:14 +01:00
{
update copyright notice
2008-02-07 23:46:54 +00:00
address_type = DNS_ADBFIND_INET6;
fetch = name->fetch_aaaa;
name->fetch_aaaa = NULL;
silence coverity false positive
2008-10-15 04:22:30 +00:00
} else {
fetch = NULL;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
silence coverity false positive
2008-10-15 04:22:30 +00:00
INSIST(address_type != 0 && fetch != NULL);
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Cleanup things we don't care about.
*/
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
if (resp->node != NULL) {
Decouple database and node lifetimes by adding node-specific vtables All databases in the codebase follow the same structure: a database is an associative container from DNS names to nodes, and each node is an associative container from RR types to RR data. Each database implementation (qpzone, qpcache, sdlz, builtin, dyndb) has its own corresponding node type (qpznode, qpcnode, etc). However, some code needs to work with nodes generically regardless of their specific type - for example, to acquire locks, manage references, or register/unregister slabs from the heap. Currently, these generic node operations are implemented as methods in the database vtable, which creates problematic coupling between database and node lifetimes. If a node outlives its parent database, the node destructor will destroy all RR data, and each RR data destructor will try to unregister from heaps by calling a virtual function from the database vtable. Since the database was already freed, this causes a crash. This commit breaks the coupling by standardizing the layout of all database nodes, adding a dedicated vtable for node operations, and moving node-specific methods from the database vtable to the node vtable.
2025-06-05 11:51:29 +02:00
dns_db_detachnode(&resp->node);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
if (resp->db != NULL) {
dns_db_detach(&resp->db);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
if (atomic_load(&adb->shuttingdown)) {
astat = DNS_ADB_SHUTTINGDOWN;
goto out;
}
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* If this name is marked as dead, clean up, throwing away
* potentially good data.
*/
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
if (cds_lfht_is_node_deleted(&name->ht_node)) {
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
astat = DNS_ADB_CANCELED;
remove ADB whenshutdown events weakly attaching and detaching the view when creating or destroying the ADB obviates the need for a whenshutdown callback event to do the detaching. remove the dns_adb_whenshutdown() mechanism, since it is no longer needed.
2022-05-11 13:55:01 -07:00
goto out;
update copyright notice
2008-02-07 23:46:54 +00:00
}
/*
* If we got a negative cache response, remember it.
*/
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
if (NCACHE_RESULT(resp->result)) {
resp->rdataset->ttl = ttlclamp(resp->rdataset->ttl);
update copyright notice
2008-02-07 23:46:54 +00:00
if (address_type == DNS_ADBFIND_INET) {
Unify the expiration time handling for all ADB expiration The algorithm from the previous commit[1] is now used to calculate all the expiration values through the code (ncache results, cname/dname targets). 1. ISC_MIN(cur, ISC_MAX(now + ADB_ENTRY_WINDOW, now + rdataset->ttl))
2024-04-03 11:51:58 +02:00
name->expire_v4 = ADJUSTED_EXPIRE(name->expire_v4, now,
resp->rdataset->ttl);
update copyright notice
2008-02-07 23:46:54 +00:00
DP(NCACHE_LEVEL,
"adb fetch name %p: "
"caching negative entry for A (ttl %u)",
Unify the expiration time handling for all ADB expiration The algorithm from the previous commit[1] is now used to calculate all the expiration values through the code (ncache results, cname/dname targets). 1. ISC_MIN(cur, ISC_MAX(now + ADB_ENTRY_WINDOW, now + rdataset->ttl))
2024-04-03 11:51:58 +02:00
name, name->expire_v4);
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
if (resp->result == DNS_R_NCACHENXDOMAIN) {
update copyright notice
2008-02-07 23:46:54 +00:00
name->fetch_err = FIND_ERR_NXDOMAIN;
} else {
name->fetch_err = FIND_ERR_NXRRSET;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
inc_resstats(adb, dns_resstatscounter_gluefetchv4fail);
update copyright notice
2008-02-07 23:46:54 +00:00
} else {
Unify the expiration time handling for all ADB expiration The algorithm from the previous commit[1] is now used to calculate all the expiration values through the code (ncache results, cname/dname targets). 1. ISC_MIN(cur, ISC_MAX(now + ADB_ENTRY_WINDOW, now + rdataset->ttl))
2024-04-03 11:51:58 +02:00
name->expire_v6 = ADJUSTED_EXPIRE(name->expire_v6, now,
resp->rdataset->ttl);
update copyright notice
2008-02-07 23:46:54 +00:00
DP(NCACHE_LEVEL,
"adb fetch name %p: "
"caching negative entry for AAAA (ttl %u)",
Unify the expiration time handling for all ADB expiration The algorithm from the previous commit[1] is now used to calculate all the expiration values through the code (ncache results, cname/dname targets). 1. ISC_MIN(cur, ISC_MAX(now + ADB_ENTRY_WINDOW, now + rdataset->ttl))
2024-04-03 11:51:58 +02:00
name, name->expire_v6);
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
if (resp->result == DNS_R_NCACHENXDOMAIN) {
update copyright notice
2008-02-07 23:46:54 +00:00
name->fetch6_err = FIND_ERR_NXDOMAIN;
} else {
name->fetch6_err = FIND_ERR_NXRRSET;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
inc_resstats(adb, dns_resstatscounter_gluefetchv6fail);
update copyright notice
2008-02-07 23:46:54 +00:00
}
goto out;
}
/*
* Handle CNAME/DNAME.
*/
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
if (resp->result == DNS_R_CNAME || resp->result == DNS_R_DNAME) {
resp->rdataset->ttl = ttlclamp(resp->rdataset->ttl);
remove 'target' from dns_adb the target name parameter to dns_adb_createfind() was always passed as NULL, so we can safely remove it. relatedly, the 'target' field in the dns_adbname structure was never referenced after being set. the 'expire_target' field was used, but only as a way to check whether an ADB name represents a CNAME or DNAME, and that information can be stored as a single flag.
2025-02-21 01:24:42 -08:00
name->flags |= NAME_IS_ALIAS;
name->expire_v4 = name->expire_v6 =
ADJUSTED_EXPIRE(INT_MAX, now, resp->rdataset->ttl);
import_rdataset() can't fail the import_rdataset() function can't return any value other than ISC_R_SUCCESS, so it's been changed to void and its callers don't rely on its return value any longer.
2025-03-21 23:48:01 -07:00
goto moreaddrs;
update copyright notice
2008-02-07 23:46:54 +00:00
}
/*
* Did we get back junk? If so, and there are no more fetches
* sitting out there, tell all the finds about it.
*/
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
if (resp->result != ISC_R_SUCCESS) {
update copyright notice
2008-02-07 23:46:54 +00:00
char buf[DNS_NAME_FORMATSIZE];
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
dns_name_format(name->name, buf, sizeof(buf));
update copyright notice
2008-02-07 23:46:54 +00:00
DP(DEF_LEVEL, "adb: fetch of '%s' %s failed: %s", buf,
address_type == DNS_ADBFIND_INET ? "A" : "AAAA",
refactor dns_resolver to use loop callbacks callback events from dns_resolver_createfetch() are now posted using isc_async_run. other modules which called the resolver and maintained task/taskmgr objects for this purpose have been cleaned up.
2022-10-28 01:33:40 -07:00
isc_result_totext(resp->result));
[master] limit recursion depth and iterative queries 4006. [security] A flaw in delegation handling could be exploited to put named into an infinite loop. This has been addressed by placing limits on the number of levels of recursion named will allow (default 7), and the number of iterative queries that it will send (default 50) before terminating a recursive query (CVE-2014-8500). The recursion depth limit is configured via the "max-recursion-depth" option. [RT #35780]
2014-11-17 23:24:44 -08:00
/*
* Don't record a failure unless this is the initial
* fetch of a chain.
*/
if (fetch->depth > 1) {
goto out;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
/* XXXMLG Don't pound on bad servers. */
if (address_type == DNS_ADBFIND_INET) {
[master] adjust max-recursion-queries 4021. [bug] Adjust max-recursion-queries to accommodate the need for more queries when the cache is empty. [RT #38104]
2014-12-15 22:28:06 -08:00
name->expire_v4 = ISC_MIN(name->expire_v4, now + 10);
update copyright notice
2008-02-07 23:46:54 +00:00
name->fetch_err = FIND_ERR_FAILURE;
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
inc_resstats(adb, dns_resstatscounter_gluefetchv4fail);
update copyright notice
2008-02-07 23:46:54 +00:00
} else {
[master] adjust max-recursion-queries 4021. [bug] Adjust max-recursion-queries to accommodate the need for more queries when the cache is empty. [RT #38104]
2014-12-15 22:28:06 -08:00
name->expire_v6 = ISC_MIN(name->expire_v6, now + 10);
update copyright notice
2008-02-07 23:46:54 +00:00
name->fetch6_err = FIND_ERR_FAILURE;
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
inc_resstats(adb, dns_resstatscounter_gluefetchv6fail);
update copyright notice
2008-02-07 23:46:54 +00:00
}
goto out;
}
/*
* We got something potentially useful.
*/
import_rdataset() can't fail the import_rdataset() function can't return any value other than ISC_R_SUCCESS, so it's been changed to void and its callers don't rely on its return value any longer.
2025-03-21 23:48:01 -07:00
import_rdataset(name, &fetch->rdataset, now);
CNAME/DNAME chaining (initial support; A6 still missing)
2000-01-21 02:50:27 +00:00
import_rdataset() can't fail the import_rdataset() function can't return any value other than ISC_R_SUCCESS, so it's been changed to void and its callers don't rely on its return value any longer.
2025-03-21 23:48:01 -07:00
moreaddrs:
astat = DNS_ADB_MOREADDRESSES;
if (address_type == DNS_ADBFIND_INET) {
name->fetch_err = FIND_ERR_SUCCESS;
} else {
name->fetch6_err = FIND_ERR_SUCCESS;
update copyright notice
2008-02-07 23:46:54 +00:00
}
ipv6 AAAA queries work now.
1999-10-29 19:20:36 +00:00
out:
remove ADB whenshutdown events weakly attaching and detaching the view when creating or destroying the ADB obviates the need for a whenshutdown callback event to do the detaching. remove the dns_adb_whenshutdown() mechanism, since it is no longer needed.
2022-05-11 13:55:01 -07:00
dns_resolver_destroyfetch(&fetch->fetch);
update copyright notice
2008-02-07 23:46:54 +00:00
free_adbfetch(adb, &fetch);
Add EDE 22 No reachable authority code Add support for Extended DNS Errors (EDE) error 22: No reachable authority. This occurs when after a timeout delay when the resolver is trying to query an authority server.
2024-11-08 18:18:30 +01:00
dns_resolver_freefresp(&resp);
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
if (astat != DNS_ADB_CANCELED) {
clean_finds_at_name(name, astat, address_type);
remove ADB whenshutdown events weakly attaching and detaching the view when creating or destroying the ADB obviates the need for a whenshutdown callback event to do the detaching. remove the dns_adb_whenshutdown() mechanism, since it is no longer needed.
2022-05-11 13:55:01 -07:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&name->lock);
dns_adbname_detach(&name);
remove ADB whenshutdown events weakly attaching and detaching the view when creating or destroying the ADB obviates the need for a whenshutdown callback event to do the detaching. remove the dns_adb_whenshutdown() mechanism, since it is no longer needed.
2022-05-11 13:55:01 -07:00
dns_adb_detach(&adb);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_unlock();
snapshot; nearly all fetch infrastructure is in place.
1999-10-27 19:36:58 +00:00
}
fetch contexts
1999-10-26 01:12:20 +00:00
static isc_result_t
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
fetch_name(dns_adbname_t *adbname, bool start_at_zone, bool no_validation,
unsigned int depth, isc_counter_t *qc, isc_counter_t *gqc,
dns_rdatatype_t type) {
update copyright notice
2008-02-07 23:46:54 +00:00
isc_result_t result;
dns_adbfetch_t *fetch = NULL;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adb_t *adb = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
dns_fixedname_t fixed;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_name_t *name = NULL;
update copyright notice
2008-02-07 23:46:54 +00:00
dns_rdataset_t rdataset;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_rdataset_t *nameservers = NULL;
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
unsigned int options = no_validation ? DNS_FETCHOPT_NOVALIDATE : 0;
update copyright notice
2008-02-07 23:46:54 +00:00
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(DNS_ADBNAME_VALID(adbname));
update copyright notice
2008-02-07 23:46:54 +00:00
adb = adbname->adb;
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE((type == dns_rdatatype_a && !NAME_FETCH_A(adbname)) ||
(type == dns_rdatatype_aaaa && !NAME_FETCH_AAAA(adbname)));
update copyright notice
2008-02-07 23:46:54 +00:00
adbname->fetch_err = FIND_ERR_NOTFOUND;
dns_rdataset_init(&rdataset);
if (start_at_zone) {
DP(ENTER_LEVEL, "fetch_name: starting at zone for name %p",
adbname);
Use dns_fixedname_initname() where possible Replace dns_fixedname_init() calls followed by dns_fixedname_name() calls with calls to dns_fixedname_initname() where it is possible without affecting current behavior and/or performance. This patch was mostly prepared using Coccinelle and the following semantic patch: @@ expression fixedname, name; @@ - dns_fixedname_init(&fixedname); ... - name = dns_fixedname_name(&fixedname); + name = dns_fixedname_initname(&fixedname); The resulting set of changes was then manually reviewed to exclude false positives and apply minor tweaks. It is likely that more occurrences of this pattern can be refactored in an identical way. This commit only takes care of the low-hanging fruit.
2018-03-28 14:38:09 +02:00
name = dns_fixedname_initname(&fixed);
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
result = dns_view_findzonecut(adb->view, adbname->name, name,
QNAME miminimization should create a separate fetch context for each fetch - this makes the cache more efficient and eliminates duplicates queries.
2018-06-13 09:29:40 +02:00
NULL, 0, 0, true, false,
libdns refactoring: get rid of multiple versions of dns_view_find, dns_view_findzonecut and dns_view_flushcache
2018-04-04 09:51:42 +02:00
&rdataset, NULL);
update copyright notice
2008-02-07 23:46:54 +00:00
if (result != ISC_R_SUCCESS && result != DNS_R_HINT) {
goto cleanup;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
nameservers = &rdataset;
options |= DNS_FETCHOPT_UNSHARED;
Turn on QNAME minimisation when fetching nameserver addresses
2023-07-18 12:18:05 +10:00
} else if (adb->view->qminimization) {
options |= DNS_FETCHOPT_QMINIMIZE | DNS_FETCHOPT_QMIN_SKIP_IP6A;
if (adb->view->qmin_strict) {
options |= DNS_FETCHOPT_QMIN_STRICT;
}
update copyright notice
2008-02-07 23:46:54 +00:00
}
rename test to qmin; add it to conf.sh.in and Makefile.in; fix copyrights
2018-05-18 13:07:31 -07:00
update copyright notice
2008-02-07 23:46:54 +00:00
fetch = new_adbfetch(adb);
[master] limit recursion depth and iterative queries 4006. [security] A flaw in delegation handling could be exploited to put named into an infinite loop. This has been addressed by placing limits on the number of levels of recursion named will allow (default 7), and the number of iterative queries that it will send (default 50) before terminating a recursive query (CVE-2014-8500). The recursion depth limit is configured via the "max-recursion-depth" option. [RT #35780]
2014-11-17 23:24:44 -08:00
fetch->depth = depth;
update copyright notice
2008-02-07 23:46:54 +00:00
QNAME miminimization should create a separate fetch context for each fetch - this makes the cache more efficient and eliminates duplicates queries.
2018-06-13 09:29:40 +02:00
/*
* We're not minimizing this query, as nothing user-related should
* be leaked here.
* However, if we'd ever want to change it we'd have to modify
* createfetch to find deepest cached name when we're providing
* domain and nameservers.
*/
Fix gaining adbname reference Call dns_adbname_ref before calling dns_resolver_createfetch to ensure adbname->name remains stable for the life of the fetch.
2025-03-19 13:31:35 +11:00
dns_adbname_ref(adbname);
libdns refactoring: get rid of 3 versions of dns_resolver_createfetch
2018-04-03 16:05:11 +02:00
result = dns_resolver_createfetch(
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
adb->res, adbname->name, type, name, nameservers, NULL, NULL, 0,
Implement global limit for outgoing queries This global limit is not reset on query restarts and is a hard limit for any client request.
2024-11-11 11:01:50 +01:00
options, depth, qc, gqc, isc_loop(), fetch_callback, adbname,
Split and simplify the use of EDE list implementation Instead of mixing the dns_resolver and dns_validator units directly with the EDE code, split-out the dns_ede functionality into own separate compilation unit and hide the implementation details behind abstraction. Additionally, the EDE codes are directly copied into the ns_client buffers by passing the EDE context to dns_resolver_createfetch(). This makes the dns_ede implementation simpler to use, although sligtly more complicated on the inside. Co-authored-by: Colin Vidal <colin@isc.org> Co-authored-by: Ondřej Surý <ondrej@isc.org>
2025-01-29 11:11:32 +01:00
NULL, &fetch->rdataset, NULL, &fetch->fetch);
Fix looping issues
2018-07-14 20:11:03 +02:00
if (result != ISC_R_SUCCESS) {
DP(ENTER_LEVEL, "fetch_name: createfetch failed with %s",
isc_result_totext(result));
Fix gaining adbname reference Call dns_adbname_ref before calling dns_resolver_createfetch to ensure adbname->name remains stable for the life of the fetch.
2025-03-19 13:31:35 +11:00
dns_adbname_unref(adbname);
update copyright notice
2008-02-07 23:46:54 +00:00
goto cleanup;
Fix looping issues
2018-07-14 20:11:03 +02:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
update copyright notice
2008-04-03 06:09:05 +00:00
if (type == dns_rdatatype_a) {
update copyright notice
2008-02-07 23:46:54 +00:00
adbname->fetch_a = fetch;
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
inc_resstats(adb, dns_resstatscounter_gluefetchv4);
update copyright notice
2008-04-03 06:09:05 +00:00
} else {
update copyright notice
2008-02-07 23:46:54 +00:00
adbname->fetch_aaaa = fetch;
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
inc_resstats(adb, dns_resstatscounter_gluefetchv6);
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
fetch = NULL; /* Keep us from cleaning this up below. */
snapshot; nearly all fetch infrastructure is in place.
1999-10-27 19:36:58 +00:00
cleanup:
update copyright notice
2008-02-07 23:46:54 +00:00
if (fetch != NULL) {
free_adbfetch(adb, &fetch);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
update copyright notice
2008-02-07 23:46:54 +00:00
if (dns_rdataset_isassociated(&rdataset)) {
dns_rdataset_disassociate(&rdataset);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
snapshot; a6 seems to work?
1999-10-30 02:26:03 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
return result;
Add the API to allow marking an address as lame for a zone
1999-10-19 21:10:29 +00:00
}
Add a way to adjust goodness.
1999-10-19 22:06:08 +00:00
Allow the srtt value to be adjusted, and update the goodness and srtt when they are modified
1999-10-19 22:21:27 +00:00
void
dns_adb_adjustsrtt(dns_adb_t *adb, dns_adbaddrinfo_t *addr, unsigned int rtt,
update copyright notice
2008-02-07 23:46:54 +00:00
unsigned int factor) {
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
REQUIRE(factor <= 10);
Allow the srtt value to be adjusted, and update the goodness and srtt when they are modified
1999-10-19 22:21:27 +00:00
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
isc_stdtime_t now = 0;
Always set ADB entry expiration to now + ADB_ENTRY_WINDOW When ADB entry was created it was set to never expire. If we never called any of the functions that adjust the expiration, it could linger in the ADB forever. Set the expiration (.expires) to now + ADB_ENTRY_WINDOW when creating the new ADB entry to ensure the ADB entry will always expire.
2024-04-03 11:33:48 +02:00
if (factor == DNS_ADB_RTTADJAGE) {
Apply the semantic patch to remove isc_stdtime_get() This is a simple replacement using the semantic patch from the previous commit and as added bonus, one removal of previously undetected unused variable in named/server.c.
2023-03-30 21:13:41 +02:00
now = isc_stdtime_now();
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
3855. [bug] Limit smoothed round trip time aging to no more than once a second. [RT #32909]
2014-05-21 10:08:52 +10:00
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
adjustsrtt(addr, rtt, factor, now);
3855. [bug] Limit smoothed round trip time aging to no more than once a second. [RT #32909]
2014-05-21 10:08:52 +10:00
}
void
dns_adb_agesrtt(dns_adb_t *adb, dns_adbaddrinfo_t *addr, isc_stdtime_t now) {
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
adjustsrtt(addr, 0, DNS_ADB_RTTADJAGE, now);
}
static void
adjustsrtt(dns_adbaddrinfo_t *addr, unsigned int rtt, unsigned int factor,
isc_stdtime_t now) {
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
unsigned int new_srtt;
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
if (factor == DNS_ADB_RTTADJAGE) {
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
if (atomic_load(&addr->entry->lastage) != now) {
Use mul and div instead of bitshifts to calculate srtt There was a microoptimization for smoothing srtt with bitshifts. Revert the code to use * 98 / 100, it doesn't really make that difference on modern CPUs, for comparison here: muldiv: imul eax, edi, 98 imul rax, rax, 1374389535 shr rax, 37 ret shift: mov eax, edi sal eax, 9 sub eax, edi shr eax, 9 ret
2023-10-12 09:20:42 +02:00
new_srtt = (uint64_t)atomic_load(&addr->entry->srtt) *
98 / 100;
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
atomic_store(&addr->entry->lastage, now);
Skip the no-op code in adjustsrtt() If factor == DNS_ADB_RTTADJAGE and addr->entry->lastage == now we would load value into new_srtt and then immediatelly store it back which triggers the synchronization between threads using .srtt values.
2023-10-12 09:17:40 +02:00
atomic_store(&addr->entry->srtt, new_srtt);
addr->srtt = new_srtt;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
} else {
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
new_srtt = ((uint64_t)atomic_load(&addr->entry->srtt) / 10 *
factor) +
Replace custom isc_u?intNN_t types with C99 u?intNN_t types
2018-03-28 14:19:37 +02:00
((uint64_t)rtt / 10 * (10 - factor));
Skip the no-op code in adjustsrtt() If factor == DNS_ADB_RTTADJAGE and addr->entry->lastage == now we would load value into new_srtt and then immediatelly store it back which triggers the synchronization between threads using .srtt values.
2023-10-12 09:17:40 +02:00
atomic_store(&addr->entry->srtt, new_srtt);
addr->srtt = new_srtt;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Allow the srtt value to be adjusted, and update the goodness and srtt when they are modified
1999-10-19 22:21:27 +00:00
}
add dns_adb_changeflags()
1999-11-16 06:53:29 +00:00
void
dns_adb_changeflags(dns_adb_t *adb, dns_adbaddrinfo_t *addr, unsigned int bits,
update copyright notice
2008-02-07 23:46:54 +00:00
unsigned int mask) {
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
add dns_adb_changeflags()
1999-11-16 06:53:29 +00:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbentry_t *entry = addr->entry;
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
unsigned int flags = atomic_load(&entry->flags);
while (!atomic_compare_exchange_strong(&entry->flags, &flags,
(flags & ~mask) | (bits & mask)))
{
/* repeat */
}
add dns_adb_changeflags()
1999-11-16 06:53:29 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
/*
* Note that we do not update the other bits in addr->flags with
* the most recent values from addr->entry->flags.
*/
addr->flags = (addr->flags & ~mask) | (bits & mask);
add dns_adb_changeflags()
1999-11-16 06:53:29 +00:00
}
add dns_adb_findaddrinfo() and dns_adb_freeaddrinfo()
2000-01-12 03:00:33 +00:00
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
/*
Describe the polynomial backoff curve used in the quota adjustment
2019-11-01 12:22:56 -05:00
* The polynomial backoff curve (10000 / ((10 + n) / 10)^(3/2)) <0..99> drops
* fairly aggressively at first, then slows down and tails off at around 2-3%.
*
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
* These will be used to make quota adjustments.
*/
static int quota_adj[] = {
10000, 8668, 7607, 6747, 6037, 5443, 4941, 4512, 4141, 3818, 3536,
3286, 3065, 2867, 2690, 2530, 2385, 2254, 2134, 2025, 1925, 1832,
1747, 1668, 1595, 1527, 1464, 1405, 1350, 1298, 1250, 1205, 1162,
1121, 1083, 1048, 1014, 981, 922, 894, 868, 843, 820, 797,
775, 755, 735, 716, 698, 680, 664, 648, 632, 618, 603,
590, 577, 564, 552, 540, 529, 518, 507, 497, 487, 477,
468, 459, 450, 442, 434, 426, 418, 411, 404, 397, 390,
383, 377, 370, 364, 358, 353, 347, 342, 336, 331, 326,
321, 316, 312, 307, 303, 298, 294, 290, 286, 282, 278
};
4719. [bug] Address PVS static analyzer warnings. [RT #45946]
2017-09-13 09:50:51 +10:00
#define QUOTA_ADJ_SIZE (sizeof(quota_adj) / sizeof(quota_adj[0]))
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
/*
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
* The adb entry associated with 'addr' must be locked.
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
*/
static void
maybe_adjust_quota(dns_adb_t *adb, dns_adbaddrinfo_t *addr, bool timeout) {
double tr;
UNUSED(adb);
if (adb->quota == 0 || adb->atr_freq == 0) {
return;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
if (timeout) {
addr->entry->timeouts++;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
if (addr->entry->completed++ <= adb->atr_freq) {
return;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
/*
* Calculate an exponential rolling average of the timeout ratio
*
* XXX: Integer arithmetic might be better than floating point
*/
tr = (double)addr->entry->timeouts / addr->entry->completed;
addr->entry->timeouts = addr->entry->completed = 0;
INSIST(addr->entry->atr >= 0.0);
INSIST(addr->entry->atr <= 1.0);
INSIST(adb->atr_discount >= 0.0);
INSIST(adb->atr_discount <= 1.0);
addr->entry->atr *= 1.0 - adb->atr_discount;
addr->entry->atr += tr * adb->atr_discount;
addr->entry->atr = ISC_CLAMP(addr->entry->atr, 0.0, 1.0);
if (addr->entry->atr < adb->atr_low && addr->entry->mode > 0) {
lib/dns/adb.c: Use atomics for adb quota values and reference counting
2019-01-22 10:47:18 +01:00
uint_fast32_t new_quota =
adb->quota * quota_adj[--addr->entry->mode] / 10000;
atomic_store_release(&addr->entry->quota,
Fix the fetches-per-server quota calculation Since commit bad5a523c2e, when the fetches-per-server quota was increased or decreased, instead of the value being set to the newly calculated quota, it was set to the *minimum* of the new quota or 1 - which effectively meant it was always set to 1. it should instead have been the maximum, to prevent the value from ever dropping to zero.
2022-05-04 17:27:56 -07:00
ISC_MAX(1, new_quota));
lib/dns/adb.c: Use atomics for adb quota values and reference counting
2019-01-22 10:47:18 +01:00
log_quota(addr->entry,
"atr %0.2f, quota increased to %" PRIuFAST32,
addr->entry->atr, new_quota);
4719. [bug] Address PVS static analyzer warnings. [RT #45946]
2017-09-13 09:50:51 +10:00
} else if (addr->entry->atr > adb->atr_high &&
addr->entry->mode < (QUOTA_ADJ_SIZE - 1))
{
lib/dns/adb.c: Use atomics for adb quota values and reference counting
2019-01-22 10:47:18 +01:00
uint_fast32_t new_quota =
adb->quota * quota_adj[++addr->entry->mode] / 10000;
atomic_store_release(&addr->entry->quota,
Fix the fetches-per-server quota calculation Since commit bad5a523c2e, when the fetches-per-server quota was increased or decreased, instead of the value being set to the newly calculated quota, it was set to the *minimum* of the new quota or 1 - which effectively meant it was always set to 1. it should instead have been the maximum, to prevent the value from ever dropping to zero.
2022-05-04 17:27:56 -07:00
ISC_MAX(1, new_quota));
lib/dns/adb.c: Use atomics for adb quota values and reference counting
2019-01-22 10:47:18 +01:00
log_quota(addr->entry,
"atr %0.2f, quota decreased to %" PRIuFAST32,
addr->entry->atr, new_quota);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
}
}
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
#define EDNSTOS 3U
void
dns_adb_plainresponse(dns_adb_t *adb, dns_adbaddrinfo_t *addr) {
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbentry_t *entry = addr->entry;
LOCK(&entry->lock);
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
maybe_adjust_quota(adb, addr, false);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
entry->plain++;
if (entry->plain == 0xff) {
entry->edns >>= 1;
entry->ednsto >>= 1;
entry->plain >>= 1;
entry->plainto >>= 1;
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&entry->lock);
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
}
void
dns_adb_timeout(dns_adb_t *adb, dns_adbaddrinfo_t *addr) {
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbentry_t *entry = addr->entry;
LOCK(&entry->lock);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
maybe_adjust_quota(adb, addr, true);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
addr->entry->plainto++;
if (addr->entry->plainto == 0xff) {
addr->entry->edns >>= 1;
Simplify the EDNS buffer size logic for DNS Flag Day 2020 The DNS Flag Day 2020 aims to remove the IP fragmentation problem from the UDP DNS communication. In this commit, we implement the required changes and simplify the logic for picking the EDNS Buffer Size. 1. The defaults for `edns-udp-size`, `max-udp-size` and `nocookie-udp-size` have been changed to `1232` (the value picked by DNS Flag Day 2020). 2. The probing heuristics that would try 512->4096->1432->1232 buffer sizes has been removed and the resolver will always use just the `edns-udp-size` value. 3. Instead of just disabling the PMTUD mechanism on the UDP sockets, we now set IP_DONTFRAG (IPV6_DONTFRAG) flag. That means that the UDP packets won't get ever fragmented. If the ICMP packets are lost the UDP will just timeout and eventually be retried over TCP.
2020-09-23 14:47:26 +02:00
addr->entry->ednsto >>= 1;
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
addr->entry->plain >>= 1;
addr->entry->plainto >>= 1;
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&entry->lock);
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
}
void
Simplify the EDNS buffer size logic for DNS Flag Day 2020 The DNS Flag Day 2020 aims to remove the IP fragmentation problem from the UDP DNS communication. In this commit, we implement the required changes and simplify the logic for picking the EDNS Buffer Size. 1. The defaults for `edns-udp-size`, `max-udp-size` and `nocookie-udp-size` have been changed to `1232` (the value picked by DNS Flag Day 2020). 2. The probing heuristics that would try 512->4096->1432->1232 buffer sizes has been removed and the resolver will always use just the `edns-udp-size` value. 3. Instead of just disabling the PMTUD mechanism on the UDP sockets, we now set IP_DONTFRAG (IPV6_DONTFRAG) flag. That means that the UDP packets won't get ever fragmented. If the ICMP packets are lost the UDP will just timeout and eventually be retried over TCP.
2020-09-23 14:47:26 +02:00
dns_adb_ednsto(dns_adb_t *adb, dns_adbaddrinfo_t *addr) {
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbentry_t *entry = addr->entry;
LOCK(&entry->lock);
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
maybe_adjust_quota(adb, addr, true);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
entry->ednsto++;
Simplify the EDNS buffer size logic for DNS Flag Day 2020 The DNS Flag Day 2020 aims to remove the IP fragmentation problem from the UDP DNS communication. In this commit, we implement the required changes and simplify the logic for picking the EDNS Buffer Size. 1. The defaults for `edns-udp-size`, `max-udp-size` and `nocookie-udp-size` have been changed to `1232` (the value picked by DNS Flag Day 2020). 2. The probing heuristics that would try 512->4096->1432->1232 buffer sizes has been removed and the resolver will always use just the `edns-udp-size` value. 3. Instead of just disabling the PMTUD mechanism on the UDP sockets, we now set IP_DONTFRAG (IPV6_DONTFRAG) flag. That means that the UDP packets won't get ever fragmented. If the ICMP packets are lost the UDP will just timeout and eventually be retried over TCP.
2020-09-23 14:47:26 +02:00
if (addr->entry->ednsto == 0xff) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
entry->edns >>= 1;
entry->ednsto >>= 1;
entry->plain >>= 1;
entry->plainto >>= 1;
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&entry->lock);
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
}
void
dns_adb_setudpsize(dns_adb_t *adb, dns_adbaddrinfo_t *addr, unsigned int size) {
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbentry_t *entry = addr->entry;
LOCK(&entry->lock);
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
if (size < 512U) {
size = 512U;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
if (size > addr->entry->udpsize) {
addr->entry->udpsize = size;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
maybe_adjust_quota(adb, addr, false);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
entry->edns++;
if (entry->edns == 0xff) {
entry->edns >>= 1;
entry->ednsto >>= 1;
entry->plain >>= 1;
entry->plainto >>= 1;
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&entry->lock);
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
}
unsigned int
dns_adb_getudpsize(dns_adb_t *adb, dns_adbaddrinfo_t *addr) {
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
unsigned int size;
dns_adbentry_t *entry = addr->entry;
LOCK(&entry->lock);
size = entry->udpsize;
UNLOCK(&entry->lock);
3593. [func] Update EDNS processing to better track remote server capabilities. [RT #30655]
2013-06-12 11:31:30 +10:00
return size;
}
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
void
4152. [func] Implement DNS COOKIE option. This replaces the experimental SIT option of BIND 9.10. The following named.conf directives are avaliable: send-cookie, cookie-secret, cookie-algorithm and nocookie-udp-size. The following dig options are available: +[no]cookie[=value] and +[no]badcookie. [RT #39928]
2015-07-06 09:44:24 +10:00
dns_adb_setcookie(dns_adb_t *adb, dns_adbaddrinfo_t *addr,
const unsigned char *cookie, size_t len) {
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbentry_t *entry = addr->entry;
LOCK(&entry->lock);
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (entry->cookie != NULL &&
(cookie == NULL || len != entry->cookielen))
4152. [func] Implement DNS COOKIE option. This replaces the experimental SIT option of BIND 9.10. The following named.conf directives are avaliable: send-cookie, cookie-secret, cookie-algorithm and nocookie-udp-size. The following dig options are available: +[no]cookie[=value] and +[no]badcookie. [RT #39928]
2015-07-06 09:44:24 +10:00
{
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
isc_mem_put(adb->mctx, entry->cookie, entry->cookielen);
entry->cookielen = 0;
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (entry->cookie == NULL && cookie != NULL && len != 0U) {
entry->cookie = isc_mem_get(adb->mctx, len);
entry->cookielen = (uint16_t)len;
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
if (entry->cookie != NULL) {
memmove(entry->cookie, cookie, len);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&entry->lock);
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
}
size_t
Clean up and refactor dns_adb_getcookie() The dns_adb_getcookie() doesn't use the 'adb' parameter, remove it. Refactor the dns_adb_getcookie() function to just return the size of the cookie when the caller passes 'NULL' as the 'cookie' argument.
2022-12-09 15:09:12 +00:00
dns_adb_getcookie(dns_adbaddrinfo_t *addr, unsigned char *cookie, size_t len) {
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbentry_t *entry = addr->entry;
LOCK(&entry->lock);
Clean up and refactor dns_adb_getcookie() The dns_adb_getcookie() doesn't use the 'adb' parameter, remove it. Refactor the dns_adb_getcookie() function to just return the size of the cookie when the caller passes 'NULL' as the 'cookie' argument.
2022-12-09 15:09:12 +00:00
if (entry->cookie == NULL) {
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
len = 0;
Clean up and refactor dns_adb_getcookie() The dns_adb_getcookie() doesn't use the 'adb' parameter, remove it. Refactor the dns_adb_getcookie() function to just return the size of the cookie when the caller passes 'NULL' as the 'cookie' argument.
2022-12-09 15:09:12 +00:00
goto unlock;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Clean up and refactor dns_adb_getcookie() The dns_adb_getcookie() doesn't use the 'adb' parameter, remove it. Refactor the dns_adb_getcookie() function to just return the size of the cookie when the caller passes 'NULL' as the 'cookie' argument.
2022-12-09 15:09:12 +00:00
if (cookie != NULL) {
if (len < entry->cookielen) {
len = 0;
goto unlock;
}
memmove(cookie, entry->cookie, entry->cookielen);
}
len = entry->cookielen;
unlock:
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&entry->lock);
3744. [experimental] SIT: send and process Source Identity Tokens (which are similar to DNS Cookies by Donald Eastlake) and are designed to help clients detect off path spoofed responses and for servers to detect legitimate clients. SIT use a experimental EDNS option code (65001). SIT can be enabled via --enable-developer or --enable-sit. It is on by default in Windows. RRL processing as been updated to know about SIT with legitimate clients not being rate limited. [RT #35389]
2014-02-19 12:53:42 +11:00
return len;
}
add dns_adb_findaddrinfo() and dns_adb_freeaddrinfo()
2000-01-12 03:00:33 +00:00
isc_result_t
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adb_findaddrinfo(dns_adb_t *adb, const isc_sockaddr_t *addr,
dns_adbaddrinfo_t **adbaddrp, isc_stdtime_t now) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
REQUIRE(DNS_ADB_VALID(adb));
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
REQUIRE(adbaddrp != NULL && *adbaddrp == NULL);
update copyright notice
2008-02-07 23:46:54 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_lock();
if (atomic_load(&adb->shuttingdown)) {
rcu_read_unlock();
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
return ISC_R_SHUTTINGDOWN;
update copyright notice
2008-02-07 23:46:54 +00:00
}
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbentry_t *adbentry = get_attached_and_locked_entry(adb, now,
addr);
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
in_port_t port = isc_sockaddr_getport(addr);
*adbaddrp = new_adbaddrinfo(adb, adbentry, port);
Replace some ADB entry locking with atomics to reduce ADB contention Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and .lastage) to remove ADB entry locking from couple of hot spots. The most prominent place is copy_namehook_lists() that gets called under ADB name lock and if the namehook list is long it acquires-releases quite a few ADB entry locks. Changing those ADB entry members to atomics allowed us to new_adbaddrinfo() not require locked ADB entry and since adbentry_overquota() already used atomics and handling lame information was dropped in the previous commit, we could not make the copy_namehook_lists() lockless. The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can now use atomics because .srtt is already atomic_uint. And the last place that could now use atomics is dns_adb_changeflags().
2023-09-21 11:59:01 +02:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
UNLOCK(&adbentry->lock);
dns_adbentry_detach(&adbentry);
add dns_adb_findaddrinfo() and dns_adb_freeaddrinfo()
2000-01-12 03:00:33 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_unlock();
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
return ISC_R_SUCCESS;
add dns_adb_findaddrinfo() and dns_adb_freeaddrinfo()
2000-01-12 03:00:33 +00:00
}
void
Megacommit of dozens of files. Cleanup of redundant/useless header file inclusion. ISC style lint, primarily for function declarations and standalone comments -- ie, those that appear on a line without any code, which should be written as follows: /* * This is a comment. */
2000-05-08 14:38:29 +00:00
dns_adb_freeaddrinfo(dns_adb_t *adb, dns_adbaddrinfo_t **addrp) {
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adbaddrinfo_t *addr = NULL;
dns_adbentry_t *entry = NULL;
add dns_adb_findaddrinfo() and dns_adb_freeaddrinfo()
2000-01-12 03:00:33 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(addrp != NULL);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
update copyright notice
2008-02-07 23:46:54 +00:00
addr = *addrp;
Clear the pointer to destroyed object early using the semantic patch Also disable the semantic patch as the code needs tweaks here and there because some destroy functions might not destroy the object and return early if the object is still in use.
2020-02-08 04:37:54 -08:00
*addrp = NULL;
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
update copyright notice
2008-02-07 23:46:54 +00:00
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
update copyright notice
2008-02-07 23:46:54 +00:00
entry = addr->entry;
add dns_adb_findaddrinfo() and dns_adb_freeaddrinfo()
2000-01-12 03:00:33 +00:00
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
REQUIRE(DNS_ADBENTRY_VALID(entry));
add dns_adb_findaddrinfo() and dns_adb_freeaddrinfo()
2000-01-12 03:00:33 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
free_adbaddrinfo(adb, &addr);
add dns_adb_findaddrinfo() and dns_adb_freeaddrinfo()
2000-01-12 03:00:33 +00:00
}
808. [func] Add 'rndc flush' to flush the server's cache.
2001-04-11 20:37:50 +00:00
void
dns_adb_flush(dns_adb_t *adb) {
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(DNS_ADB_VALID(adb));
808. [func] Add 'rndc flush' to flush the server's cache.
2001-04-11 20:37:50 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_lock();
if (atomic_load(&adb->shuttingdown)) {
rcu_read_unlock();
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
return;
}
808. [func] Add 'rndc flush' to flush the server's cache.
2001-04-11 20:37:50 +00:00
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
cleanup_names(adb, INT_MAX);
Keep the unlink adb entries until expiration Currently, the ADB uses TTL of 0 for ADB names that the server is authoritative for and TTL of 10 seconds for HINT and GLUE ADB names. This requires the unlinked ADB entries to be kept around, because they would disappear too quickly. This especially affect the root zone as the trust level is "ultimate" for the root zone nameservers. This commit restores the ability to keep the unlinked ADB entries in the database for later reuse, restores printing the unlinked entries and adds some extra cleaning of the unlinked ADB entries on the tail of the LRU list (similar to what we are doing for the ADB names).
2022-11-23 09:56:19 +01:00
cleanup_entries(adb, INT_MAX);
808. [func] Add 'rndc flush' to flush the server's cache.
2001-04-11 20:37:50 +00:00
#ifdef DUMP_ADB_AFTER_CLEANING
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
dump_adb(adb, stdout, true, INT_MAX);
808. [func] Add 'rndc flush' to flush the server's cache.
2001-04-11 20:37:50 +00:00
#endif /* ifdef DUMP_ADB_AFTER_CLEANING */
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_unlock();
808. [func] Add 'rndc flush' to flush the server's cache.
2001-04-11 20:37:50 +00:00
}
1073. [bug] The adb cache should also be space driven, max-adb-size. [RT #1915, #1938]
2001-10-25 04:57:46 +00:00
1137. [func] It is now possible to flush given names from the adb cache dns_adb_flushname().
2001-11-27 03:00:50 +00:00
void
4546. [func] Extend the use of const declarations. [RT #43379]
2016-12-30 15:45:08 +11:00
dns_adb_flushname(dns_adb_t *adb, const dns_name_t *name) {
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
dns_adbname_t *adbname = NULL;
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
bool start_at_zone = false;
Store static-stub addresses seperately in the adb Static-stub address and addresses from other sources where being mixed together resulting in static-stub queries going to addresses not specified in the configuration or alternatively static-stub addresses being used instead of the real addresses.
2024-08-14 23:46:44 +10:00
bool static_stub = false;
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
bool novalidate = false;
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
dns_adbname_t key = { .name = UNCONST(name) };
update copyright notice
2008-02-07 23:46:54 +00:00
[master] "rndc flushtree -all <name>" 3606. [func] "rndc flushtree -all" flushes matching records in the ADB and bad cache as well as the DNS cache. (Without the "-all" option, flushtree will still only flush records from the DNS cache.) [RT #33970]
2013-06-26 14:59:32 -07:00
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(name != NULL);
update copyright notice
2008-02-07 23:46:54 +00:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_lock();
if (atomic_load(&adb->shuttingdown)) {
rcu_read_unlock();
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
return;
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
again:
/*
Store static-stub addresses seperately in the adb Static-stub address and addresses from other sources where being mixed together resulting in static-stub queries going to addresses not specified in the configuration or alternatively static-stub addresses being used instead of the real addresses.
2024-08-14 23:46:44 +10:00
* Delete all entries - with and without DNS_ADBFIND_STARTATZONE set
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
* with and without DNS_ADBFIND_STATICSTUB set and with and without
* DNS_ADBFIND_NOVALIDATE set.
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
*/
Separate out adbname flags that are hashed There are three adbname flags that are used to identify different types of adbname lookups when hashing rather than using multiple hash tables. Separate these to their own structure element as these need to be able to be read without locking the adbname structure.
2025-07-01 16:45:39 +10:00
key.type = ((static_stub) ? DNS_ADBFIND_STATICSTUB : 0) |
((start_at_zone) ? DNS_ADBFIND_STARTATZONE : 0) |
((novalidate) ? DNS_ADBFIND_NOVALIDATE : 0);
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
uint32_t hashval = hash_adbname(&key);
struct cds_lfht_iter iter;
cds_lfht_lookup(adb->names_ht, hashval, match_adbname, &key, &iter);
adbname = cds_lfht_entry(cds_lfht_iter_get_node(&iter), dns_adbname_t,
ht_node);
if (adbname != NULL) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbname_ref(adbname);
LOCK(&adbname->lock);
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
if (dns_name_equal(name, adbname->name)) {
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
expire_name(adbname, DNS_ADB_CANCELED);
update copyright notice
2008-02-07 23:46:54 +00:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&adbname->lock);
dns_adbname_detach(&adbname);
}
if (!start_at_zone) {
start_at_zone = true;
goto again;
update copyright notice
2008-02-07 23:46:54 +00:00
}
Store static-stub addresses seperately in the adb Static-stub address and addresses from other sources where being mixed together resulting in static-stub queries going to addresses not specified in the configuration or alternatively static-stub addresses being used instead of the real addresses.
2024-08-14 23:46:44 +10:00
if (!static_stub) {
static_stub = true;
goto again;
}
Validate address lookups from ADB The address lookups from ADB were not being validated, allowing spoofed responses to be accepted and used for other lookups. Validate the answers except when CD=1 is set in the triggering request. Separate ADB names looked up with CD=1 from those without CD=1, to prevent the use of unvalidated answers in the normal lookup case (CD=0). Set the TTL on unvalidated (pending) responses to ADB_CACHE_MINIMUM when adding them to the ADB.
2025-01-17 19:32:28 +11:00
if (!novalidate) {
start_at_zone = false;
static_stub = false;
novalidate = true;
goto again;
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_unlock();
1137. [func] It is now possible to flush given names from the adb cache dns_adb_flushname().
2001-11-27 03:00:50 +00:00
}
[master] "rndc flushtree -all <name>" 3606. [func] "rndc flushtree -all" flushes matching records in the ADB and bad cache as well as the DNS cache. (Without the "-all" option, flushtree will still only flush records from the DNS cache.) [RT #33970]
2013-06-26 14:59:32 -07:00
void
4546. [func] Extend the use of const declarations. [RT #43379]
2016-12-30 15:45:08 +11:00
dns_adb_flushnames(dns_adb_t *adb, const dns_name_t *name) {
[master] "rndc flushtree -all <name>" 3606. [func] "rndc flushtree -all" flushes matching records in the ADB and bad cache as well as the DNS cache. (Without the "-all" option, flushtree will still only flush records from the DNS cache.) [RT #33970]
2013-06-26 14:59:32 -07:00
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(name != NULL);
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_lock();
if (atomic_load(&adb->shuttingdown)) {
rcu_read_unlock();
Refactor ADB reference counting, shutdown and locking The ADB previously used separate reference counters for internal and external references, plus additional counters for ABD find and namehook objects, and used all these counters to coordinate its shutdown process, which was a multi-stage affair involving a sequence of control events. It also used a complex interlocking set of static functions for referencing, deferencing, linking, unlinking, and cleaning up various internal objects; these functions returned boolean values to their callers to indicate what additional processing was needed. The changes in the previous two commits destabilized this fragile system in a way that was difficult to recover from, so in this commit we refactor all of it. The dns_adb and dns_adbentry objects now use conventional attach and detach functions for reference counting, and the shutdown process is much more straightforward. Instead of handling shutdown asynchronously, we can just destroy the ADB when references reach zero In addition, ADB locking has been simplified. Instead of a single `find_{name,entry}_and_lock()` function which searches for a name or entry's hash bucket, locks it, and then searches for the name or entry in the bucket, we now use one function to find the bucket (leaving it to the caller to do the locking) and another find the name or entry. Instead of locking the entire ADB when modifying hash tables, we now use read-write locks around the specific hash table. The only remaining need for adb->lock is when modifying the `whenshutdown` list. Comments throughout the module have been improved.
2022-03-21 12:48:52 -07:00
return;
}
Refactor how ADB names and entries are stored in the dns_adb Replace adb->{names,entries} and related arrays (indexed by hashed bucket) with a isc_ht hash tables storing the new struct adb{name,entry}bucket_t that wraps all the variables that were originally stored in arrays indexed by "bucket" number stored directly in the struct dns_adb. Previously, the task exclusive mode has been used to grow the internal arrays used to store the named and entries objects. The isc_ht hash tables are now protected by the isc_rwlock instead and thus the usage of the task exclusive mode has been removed from the dns_adb. Co-authored-by: Ondřej Surý <ondrej@isc.org>
2022-03-16 21:58:55 +01:00
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
dns_adbname_t *adbname = NULL;
struct cds_lfht_iter iter;
cds_lfht_for_each_entry(adb->names_ht, &iter, adbname, ht_node) {
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
dns_adbname_ref(adbname);
LOCK(&adbname->lock);
Refactor isc_hashmap to accept custom match function Refactor isc_hashmap to allow custom matching functions. This allows us to have better tailored keys that don't require fixed uint8_t arrays, but can be composed of more fields from the stored data structure.
2023-08-28 10:08:59 +02:00
if (dns_name_issubdomain(adbname->name, name)) {
refactor dns_adb to use loop callbacks The callbacks from dns_abd_createfind() are now posted using isc_async_run() instead of isc_task_send(). ADB event types have been replaced with a new dns_adbstatus_t type which is included as find->status. (The ADB still uses a task for dns_resolver_createfetch().)
2022-10-27 22:55:47 -07:00
expire_name(adbname, DNS_ADB_CANCELED);
[master] "rndc flushtree -all <name>" 3606. [func] "rndc flushtree -all" flushes matching records in the ADB and bad cache as well as the DNS cache. (Without the "-all" option, flushtree will still only flush records from the DNS cache.) [RT #33970]
2013-06-26 14:59:32 -07:00
}
Refactor the dns_adb unit The dns_adb unit has been refactored to be much simpler. Following changes have been made: 1. Simplify the ADB to always allow GLUE and hints There were only two places where dns_adb_createfind() was used - in the dns_resolver unit where hints and GLUE addresses were ok, and in the dns_zone where dns_adb_createfind() would be called without DNS_ADBFIND_HINTOK and DNS_ADBFIND_GLUEOK set. Simplify the logic by allowing hint and GLUE addresses when looking up the nameserver addresses to notify. The difference is negligible and would cause a difference in the notified addresses only when there's mismatch between the parent and child addresses and we haven't cached the child addresses yet. 2. Drop the namebuckets and entrybuckets Formerly, the namebuckets and entrybuckets were used to reduced the lock contention when accessing the double-linked lists stored in each bucket. In the previous refactoring, the custom hashtable for the buckets has been replaced with isc_ht/isc_hashmap, so only a single item (mostly, see below) would end up in each bucket. Removing the entrybuckets has been straightforward, the only matching was done on the isc_sockaddr_t member of the dns_adbentry. Removing the zonebuckets required GLUEOK and HINTOK bits to be removed because the find could match entries with-or-without the bits set, and creating a custom key that stores the DNS_ADBFIND_STARTATZONE in the first byte of the key, so we can do a straightforward lookup into the hashtable without traversing a list that contains items with different flags. 3. Remove unassociated entries from ADB database Previously, the adbentries could live in the ADB database even after unlinking them from dns_adbnames. Such entries would show up as "Unassociated entries" in the ADB dump. The benefit of keeping such entries is little - the chance that we link such entry to a adbname is small, and it's simpler to evict unlinked entries from the ADB cache (and the hashtable) than create second LRU cleaning mechanism. Unlinked ADB entries are now directly deleted from the hash table (hashmap) upon destruction. 4. Cleanup expired entries from the hash table When buckets were still in place, the code would keep the buckets always allocated and never shrink the hash table (hashmap). With proper reference counting in place, we can delete the adbnames from the hash table and the LRU list. 5. Stop purging the names early when we hit the time limit Because the LRU list is now time ordered, we can stop purging the names when we find a first entry that doesn't fullfil our time-based eviction criteria because no further entry on the LRU list will meet the criteria. Future work: 1. Lock contention In this commit, the focus was on correctness of the data structure, but in the future, the lock contention in the ADB database needs to be addressed. Currently, we use simple mutex to lock the hash tables, because we almost always need to use a write lock for properly purging the hashtables. The ADB database needs to be sharded (similar to the effect that buckets had in the past). Each shard would contain own hashmap and own LRU list. 2. Time-based purging The ADB names and entries stay intact when there are no lookups. When we add separate shards, a timer needs to be added for time-based cleaning in case there's no traffic hashing to the inactive shard. 3. Revisit the 30 minutes limit The ADB cache is capped at 30 minutes. This needs to be revisited, and at least the limit should be configurable (in both directions).
2022-10-13 12:00:54 +02:00
UNLOCK(&adbname->lock);
dns_adbname_detach(&adbname);
[master] "rndc flushtree -all <name>" 3606. [func] "rndc flushtree -all" flushes matching records in the ADB and bad cache as well as the DNS cache. (Without the "-all" option, flushtree will still only flush records from the DNS cache.) [RT #33970]
2013-06-26 14:59:32 -07:00
}
Use cds_lfht for lock-free hashtables in dns_adb Replace the read-write locked isc_hashmap with lock-free cds_lfht hashtable and replace the singular LRU tables for ADB names and entries with a per-thread LRU tables. These changes allowed to remove all the read-write locking on the names and entries tables.
2025-06-25 19:02:37 +02:00
rcu_read_unlock();
[master] "rndc flushtree -all <name>" 3606. [func] "rndc flushtree -all" flushes matching records in the ADB and bad cache as well as the DNS cache. (Without the "-all" option, flushtree will still only flush records from the DNS cache.) [RT #33970]
2013-06-26 14:59:32 -07:00
}
1073. [bug] The adb cache should also be space driven, max-adb-size. [RT #1915, #1938]
2001-10-25 04:57:46 +00:00
void
[master] accept >4g max-{,a}cache-size 3506. [func] When setting "max-cache-size" and "max-acache-size", the keyword "unlimited" is no longer defined as equal to 4 gigabytes (except on 32-bit platforms); it means literally unlimited. [RT #32358] 3505. [bug] When setting "max-cache-size" and "max-acache-size", larger values than 4 gigabytes could not be set explicitly, though larger sizes were available when setting cache size to 0. This has been corrected; the full range is now available. [RT #32358]
2013-02-28 09:29:12 -08:00
dns_adb_setadbsize(dns_adb_t *adb, size_t size) {
size_t hiwater, lowater;
1073. [bug] The adb cache should also be space driven, max-adb-size. [RT #1915, #1938]
2001-10-25 04:57:46 +00:00
minor pre-refactoring cleanups the use of "result" as a variable name for a boolean return value was confusing; all 'result' variables that are not isc_result_t have been renamed to 'ret'. The static function print_dns_name() was a duplicate of dns_name_print(), so it has been replaced with that. Changed INSIST to REQUIRE where appropriate, and added NULL initialization for pointer variables.
2022-03-24 16:06:48 -07:00
REQUIRE(DNS_ADB_VALID(adb));
1073. [bug] The adb cache should also be space driven, max-adb-size. [RT #1915, #1938]
2001-10-25 04:57:46 +00:00
silence compiler warnings
2013-03-05 23:41:22 +11:00
if (size != 0U && size < DNS_ADB_MINADBSIZE) {
update copyright notice
2008-02-07 23:46:54 +00:00
size = DNS_ADB_MINADBSIZE;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
1073. [bug] The adb cache should also be space driven, max-adb-size. [RT #1915, #1938]
2001-10-25 04:57:46 +00:00
update copyright notice
2008-02-07 23:46:54 +00:00
hiwater = size - (size >> 3); /* Approximately 7/8ths. */
lowater = size - (size >> 2); /* Approximately 3/4ths. */
1073. [bug] The adb cache should also be space driven, max-adb-size. [RT #1915, #1938]
2001-10-25 04:57:46 +00:00
silence compiler warnings
2013-03-05 23:41:22 +11:00
if (size == 0U || hiwater == 0U || lowater == 0U) {
Use static storage for isc_mem water_t On the isc_mem water change the old water_t structure could be used after free. Instead of introducing reference counting on the hot-path we are going to introduce additional constraints on the isc_mem_setwater. Once it's set for the first time, the additional calls have to be made with the same water and water_arg arguments.
2021-07-22 06:14:32 +02:00
isc_mem_clearwater(adb->mctx);
update copyright notice
2008-02-07 23:46:54 +00:00
} else {
Refactor the handling of isc_mem overmem condition Previously, there were two methods of working with the overmem condition: 1. hi/lo water callback - when the overmem condition was reached for the first time, the water callback was called with HIWATER mark and .is_overmem boolean was set internally. Similarly, when the used memory went below the lo water mark, the water callback would be called with LOWATER mark and .is_overmem was reset. This check would be called **every** time memory was allocated or freed. 2. isc_mem_isovermem() - a simple getter for the internal .is_overmem flag This commit refactors removes the first method and move the hi/lo water checks to the isc_mem_isovermem() function, thus we now have only a single method of checking overmem condition and the check for hi/lo water is removed from the hot path for memory contexts that doesn't use overmem checks.
2023-11-29 09:01:56 +01:00
isc_mem_setwater(adb->mctx, hiwater, lowater);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
1073. [bug] The adb cache should also be space driven, max-adb-size. [RT #1915, #1938]
2001-10-25 04:57:46 +00:00
}
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
void
Replace custom isc_u?intNN_t types with C99 u?intNN_t types
2018-03-28 14:19:37 +02:00
dns_adb_setquota(dns_adb_t *adb, uint32_t quota, uint32_t freq, double low,
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
double high, double discount) {
REQUIRE(DNS_ADB_VALID(adb));
adb->quota = quota;
adb->atr_freq = freq;
adb->atr_low = low;
adb->atr_high = high;
adb->atr_discount = discount;
}
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
void
dns_adb_getquota(dns_adb_t *adb, uint32_t *quotap, uint32_t *freqp,
double *lowp, double *highp, double *discountp) {
REQUIRE(DNS_ADB_VALID(adb));
Apply the SET_IF_NOT_NULL() semantic patch spatch --sp-file cocci/set_if_not_null.spatch --use-gitgrep --dir "." --include-headers --in-place
2023-04-06 16:32:16 +01:00
SET_IF_NOT_NULL(quotap, adb->quota);
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
Apply the SET_IF_NOT_NULL() semantic patch spatch --sp-file cocci/set_if_not_null.spatch --use-gitgrep --dir "." --include-headers --in-place
2023-04-06 16:32:16 +01:00
SET_IF_NOT_NULL(freqp, adb->atr_freq);
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
Apply the SET_IF_NOT_NULL() semantic patch spatch --sp-file cocci/set_if_not_null.spatch --use-gitgrep --dir "." --include-headers --in-place
2023-04-06 16:32:16 +01:00
SET_IF_NOT_NULL(lowp, adb->atr_low);
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
Apply the SET_IF_NOT_NULL() semantic patch spatch --sp-file cocci/set_if_not_null.spatch --use-gitgrep --dir "." --include-headers --in-place
2023-04-06 16:32:16 +01:00
SET_IF_NOT_NULL(highp, adb->atr_high);
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
Apply the SET_IF_NOT_NULL() semantic patch spatch --sp-file cocci/set_if_not_null.spatch --use-gitgrep --dir "." --include-headers --in-place
2023-04-06 16:32:16 +01:00
SET_IF_NOT_NULL(discountp, adb->atr_discount);
add "rndc fetchlimit" to show fetchlimited servers this command runs dns_adb_dumpquota() to display all servers in the ADB that are being actively fetchlimited by the fetches-per-server controls (i.e, servers with a nonzero average timeout ratio or with the quota having been reduced from the default value). the "fetchlimit" system test has been updated to use the new command to check quota values instead of "rndc dumpdb".
2022-05-26 02:33:52 -07:00
}
Change dns_adbentry_overquota() to dns_adb_overquota() The dns_adbentry_overquota() was violating the layers accessing the adbentry struct members directly. Change it to dns_adb_overquota() to match the dns_adb API.
2023-03-30 21:19:17 +02:00
static bool
adbentry_overquota(dns_adbentry_t *entry) {
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
REQUIRE(DNS_ADBENTRY_VALID(entry));
lib/dns/adb.c: Use atomics for adb quota values and reference counting
2019-01-22 10:47:18 +01:00
Change dns_adbentry_overquota() to dns_adb_overquota() The dns_adbentry_overquota() was violating the layers accessing the adbentry struct members directly. Change it to dns_adb_overquota() to match the dns_adb API.
2023-03-30 21:19:17 +02:00
uint_fast32_t quota = atomic_load_relaxed(&entry->quota);
uint_fast32_t active = atomic_load_acquire(&entry->active);
lib/dns/adb.c: Use atomics for adb quota values and reference counting
2019-01-22 10:47:18 +01:00
return quota != 0 && active >= quota;
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
}
Change dns_adbentry_overquota() to dns_adb_overquota() The dns_adbentry_overquota() was violating the layers accessing the adbentry struct members directly. Change it to dns_adb_overquota() to match the dns_adb API.
2023-03-30 21:19:17 +02:00
bool
dns_adb_overquota(dns_adb_t *adb ISC_ATTR_UNUSED, dns_adbaddrinfo_t *addrinfo) {
REQUIRE(DNS_ADBADDRINFO_VALID(addrinfo));
return adbentry_overquota(addrinfo->entry);
}
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
void
dns_adb_beginudpfetch(dns_adb_t *adb, dns_adbaddrinfo_t *addr) {
REQUIRE should not have side effects it's a style violation to have REQUIRE or INSIST contain code that must run for the server to work. this was being done with some atomic_compare_exchange calls. these have been cleaned up. uses of atomic_compare_exchange in assertions have been replaced with a new macro atomic_compare_exchange_enforced, which uses RUNTIME_CHECK to ensure that the exchange was successful.
2021-10-21 02:28:48 -07:00
uint_fast32_t active;
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
REQUIRE should not have side effects it's a style violation to have REQUIRE or INSIST contain code that must run for the server to work. this was being done with some atomic_compare_exchange calls. these have been cleaned up. uses of atomic_compare_exchange in assertions have been replaced with a new macro atomic_compare_exchange_enforced, which uses RUNTIME_CHECK to ensure that the exchange was successful.
2021-10-21 02:28:48 -07:00
active = atomic_fetch_add_relaxed(&addr->entry->active, 1);
INSIST(active != UINT32_MAX);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
}
void
dns_adb_endudpfetch(dns_adb_t *adb, dns_adbaddrinfo_t *addr) {
REQUIRE should not have side effects it's a style violation to have REQUIRE or INSIST contain code that must run for the server to work. this was being done with some atomic_compare_exchange calls. these have been cleaned up. uses of atomic_compare_exchange in assertions have been replaced with a new macro atomic_compare_exchange_enforced, which uses RUNTIME_CHECK to ensure that the exchange was successful.
2021-10-21 02:28:48 -07:00
uint_fast32_t active;
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
REQUIRE(DNS_ADB_VALID(adb));
REQUIRE(DNS_ADBADDRINFO_VALID(addr));
REQUIRE should not have side effects it's a style violation to have REQUIRE or INSIST contain code that must run for the server to work. this was being done with some atomic_compare_exchange calls. these have been cleaned up. uses of atomic_compare_exchange in assertions have been replaced with a new macro atomic_compare_exchange_enforced, which uses RUNTIME_CHECK to ensure that the exchange was successful.
2021-10-21 02:28:48 -07:00
active = atomic_fetch_sub_release(&addr->entry->active, 1);
INSIST(active != 0);
[master] DDoS mitigation features 3938. [func] Added quotas to be used in recursive resolvers that are under high query load for names in zones whose authoritative servers are nonresponsive or are experiencing a denial of service attack. - "fetches-per-server" limits the number of simultaneous queries that can be sent to any single authoritative server. The configured value is a starting point; it is automatically adjusted downward if the server is partially or completely non-responsive. The algorithm used to adjust the quota can be configured via the "fetch-quota-params" option. - "fetches-per-zone" limits the number of simultaneous queries that can be sent for names within a single domain. (Note: Unlike "fetches-per-server", this value is not self-tuning.) - New stats counters have been added to count queries spilled due to these quotas. See the ARM for details of these options. [RT #37125]
2015-07-08 22:53:39 -07:00
}
move ADB and resolver stats out of the view object for better object separation, ADB and resolver statistics counters are now stored in the ADB and resolver objects themsevles, rather than in the associated view.
2022-05-11 15:38:54 -07:00
isc_stats_t *
dns_adb_getstats(dns_adb_t *adb) {
REQUIRE(DNS_ADB_VALID(adb));
return adb->stats;
}
Reference in New Issue Copy Permalink