2
0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-30 22:15:20 +00:00

dns/rbt.c: Implement incremental hash table resizing

Originally, the hash table used in RBT database would be resized when it
reached certain number of elements (defined by overcommit).  This was
causing resolution brownouts for busy resolvers, because the rehashing
could take several seconds to complete.  This was mitigated by
pre-allocating the hash table in the RBT database used for caching to be
large-enough as determined by max-cache-size.  The downside of this
solution was that the pre-allocated hash table could take a significant
chunk of the memory even when the resolver cache would be otherwise
empty because the default value for max-cache-size is 90% of available
memory.

Implement incremental resizing[1] to perform the rehashing gradually:

 1. During the resize, allocate the new hash table, but keep the old
    table unchanged.
 2. In each lookup or delete operation, check both tables.
 3. Perform insertion operations only in the new table.
 4. At each insertion also move r elements from the old table to the new
    table.
 5. When all elements are removed from the old table, deallocate it.

To ensure that the old table is completely copied over before the new
table itself needs to be enlarged, it is necessary to increase the
size of the table by a factor of at least (r + 1)/r during resizing.

In our implementation r is equal to 1.

The downside of this approach is that the old table and the new table
could stay in memory for longer when there are no new insertions into
the hash table for prolonged periods of time as the incremental
rehashing happens only during the insertions.

The upside of this approach is that it's no longer necessary to
pre-allocate large hash table, because the RBT hash table rehashing
doesn't cause resolution brownouts anymore and thus we can use the
memory as needed.

1. https://en.m.wikipedia.org/wiki/Hash_table#Dynamic_resizing
This commit is contained in:
Ondřej Surý
2021-10-07 18:41:02 +02:00
parent 0590d71977
commit 8c819ec366
10 changed files with 318 additions and 371 deletions

View File

@@ -1234,56 +1234,34 @@ getoriginnode(dns_db_t *db, dns_dbnode_t **nodep) {
}
static dns_dbmethods_t sdlzdb_methods = {
attach,
detach,
beginload,
endload,
dump,
currentversion,
newversion,
attachversion,
closeversion,
findnode,
find,
findzonecut,
attachnode,
detachnode,
expirenode,
printnode,
createiterator,
findrdataset,
allrdatasets,
addrdataset,
subtractrdataset,
deleterdataset,
issecure,
nodecount,
ispersistent,
overmem,
settask,
getoriginnode,
NULL, /* transfernode */
NULL, /* getnsec3parameters */
NULL, /* findnsec3node */
NULL, /* setsigningtime */
NULL, /* getsigningtime */
NULL, /* resigned */
NULL, /* isdnssec */
NULL, /* getrrsetstats */
NULL, /* rpz_attach */
NULL, /* rpz_ready */
findnodeext,
findext,
NULL, /* setcachestats */
NULL, /* hashsize */
NULL, /* nodefullname */
NULL, /* getsize */
NULL, /* setservestalettl */
NULL, /* getservestalettl */
NULL, /* setservestalerefresh */
NULL, /* getservestalerefresh */
NULL, /* setgluecachestats */
NULL /* adjusthashsize */
attach, detach, beginload,
endload, dump, currentversion,
newversion, attachversion, closeversion,
findnode, find, findzonecut,
attachnode, detachnode, expirenode,
printnode, createiterator, findrdataset,
allrdatasets, addrdataset, subtractrdataset,
deleterdataset, issecure, nodecount,
ispersistent, overmem, settask,
getoriginnode, NULL, /* transfernode */
NULL, /* getnsec3parameters */
NULL, /* findnsec3node */
NULL, /* setsigningtime */
NULL, /* getsigningtime */
NULL, /* resigned */
NULL, /* isdnssec */
NULL, /* getrrsetstats */
NULL, /* rpz_attach */
NULL, /* rpz_ready */
findnodeext, findext, NULL, /* setcachestats */
NULL, /* hashsize */
NULL, /* nodefullname */
NULL, /* getsize */
NULL, /* setservestalettl */
NULL, /* getservestalettl */
NULL, /* setservestalerefresh */
NULL, /* getservestalerefresh */
NULL, /* setgluecachestats */
};
/*