mir/bind
2
0
Fork 0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-23 10:39:16 +00:00
Code Issues Releases Wiki Activity
bind/lib/isc/loop.c

650 lines
15 KiB
C
Raw Normal View History

New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
/*
* Copyright (C) Internet Systems Consortium, Inc. ("ISC")
*
* SPDX-License-Identifier: MPL-2.0
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, you can obtain one at https://mozilla.org/MPL/2.0/.
*
* See the COPYRIGHT file distributed with this work for additional
* information regarding copyright ownership.
*/
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <isc/async.h>
#include <isc/atomic.h>
#include <isc/barrier.h>
#include <isc/condition.h>
#include <isc/job.h>
#include <isc/list.h>
remove isc_task completely as there is no further use of isc_task in BIND, this commit removes it, along with isc_taskmgr, isc_event, and all other related types. functions that accepted taskmgr as a parameter have been cleaned up. as a result of this change, some functions can no longer fail, so they've been changed to type void, and their callers have been updated accordingly. the tasks table has been removed from the statistics channel and the stats version has been updated. dns_dyndbctx has been changed to reference the loopmgr instead of taskmgr, and DNS_DYNDB_VERSION has been udpated as well.
2022-10-29 14:22:56 -07:00
#include <isc/log.h>
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
#include <isc/loop.h>
#include <isc/magic.h>
#include <isc/mem.h>
#include <isc/mutex.h>
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
#include <isc/qsbr.h>
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
#include <isc/refcount.h>
#include <isc/result.h>
#include <isc/signal.h>
#include <isc/strerr.h>
#include <isc/thread.h>
#include <isc/tid.h>
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
#include <isc/time.h>
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
#include <isc/util.h>
#include <isc/uv.h>
#include <isc/work.h>
#include "job_p.h"
#include "loop_p.h"
/**
* Private
*/
static void
ignore_signal(int sig, void (*handler)(int)) {
De-duplicate some calls to strerror_r() Specifically, when reporting an unexpected or fatal error.
2022-10-14 17:18:07 +01:00
struct sigaction sa = { .sa_handler = handler };
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
if (sigfillset(&sa.sa_mask) != 0 || sigaction(sig, &sa, NULL) < 0) {
De-duplicate some calls to strerror_r() Specifically, when reporting an unexpected or fatal error.
2022-10-14 17:18:07 +01:00
FATAL_SYSERROR(errno, "ignore_signal(%d)", sig);
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
}
}
void
isc_loopmgr_shutdown(isc_loopmgr_t *loopmgr) {
if (!atomic_compare_exchange_strong(&loopmgr->shuttingdown,
&(bool){ false }, true))
{
return;
}
for (size_t i = 0; i < loopmgr->nloops; i++) {
isc_loop_t *loop = &loopmgr->loops[i];
int r;
r = uv_async_send(&loop->shutdown_trigger);
UV_RUNTIME_CHECK(uv_async_send, r);
}
}
static void
isc__loopmgr_signal(void *arg, int signum) {
isc_loopmgr_t *loopmgr = (isc_loopmgr_t *)arg;
switch (signum) {
case SIGINT:
case SIGTERM:
isc_loopmgr_shutdown(loopmgr);
break;
default:
UNREACHABLE();
}
}
static void
pause_loop(isc_loop_t *loop) {
isc_loopmgr_t *loopmgr = loop->loopmgr;
loop->paused = true;
(void)isc_barrier_wait(&loopmgr->pausing);
}
static void
resume_loop(isc_loop_t *loop) {
isc_loopmgr_t *loopmgr = loop->loopmgr;
(void)isc_barrier_wait(&loopmgr->resuming);
loop->paused = false;
}
static void
pauseresume_cb(uv_async_t *handle) {
isc_loop_t *loop = uv_handle_get_data(handle);
pause_loop(loop);
resume_loop(loop);
}
#define XX(uc, lc) \
case UV_##uc: \
fprintf(stderr, "%s, %s: dangling %p: %p.type = %s\n", \
__func__, (char *)arg, handle->loop, handle, #lc); \
break;
static void
loop_walk_cb(uv_handle_t *handle, void *arg) {
if (uv_is_closing(handle)) {
return;
}
switch (handle->type) {
UV_HANDLE_TYPE_MAP(XX)
default:
fprintf(stderr, "%s, %s: dangling %p: %p.type = %s\n", __func__,
(char *)arg, &handle->loop, handle, "unknown");
}
}
static void
shutdown_trigger_close_cb(uv_handle_t *handle) {
isc_loop_t *loop = uv_handle_get_data(handle);
isc_loop_detach(&loop);
}
static void
destroy_cb(uv_async_t *handle) {
isc_loop_t *loop = uv_handle_get_data(handle);
uv_close(&loop->destroy_trigger, NULL);
uv_close(&loop->queue_trigger, NULL);
uv_close(&loop->pause_trigger, NULL);
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
uv_close(&loop->wakeup_trigger, NULL);
uv_close(&loop->quiescent, NULL);
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
uv_walk(&loop->loop, loop_walk_cb, (char *)"destroy_cb");
}
static void
shutdown_cb(uv_async_t *handle) {
isc_job_t *job = NULL;
isc_loop_t *loop = uv_handle_get_data(handle);
isc_loopmgr_t *loopmgr = loop->loopmgr;
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
loop->shuttingdown = true;
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
/* Make sure, we can't be called again */
uv_close(&loop->shutdown_trigger, shutdown_trigger_close_cb);
if (DEFAULT_LOOP(loopmgr) == CURRENT_LOOP(loopmgr)) {
/* Stop the signal handlers */
isc_signal_stop(loopmgr->sigterm);
isc_signal_stop(loopmgr->sigint);
/* Free the signal handlers */
isc_signal_destroy(&loopmgr->sigterm);
isc_signal_destroy(&loopmgr->sigint);
}
job = ISC_LIST_TAIL(loop->teardown_jobs);
while (job != NULL) {
isc_job_t *prev = ISC_LIST_PREV(job, link);
ISC_LIST_UNLINK(loop->teardown_jobs, job, link);
isc__job_run(job);
job = prev;
}
}
static void
queue_cb(uv_async_t *handle) {
isc_loop_t *loop = uv_handle_get_data(handle);
REQUIRE(VALID_LOOP(loop));
Use atomic stack for async job queue Previously, the async job queue would use a locked-list (ISC_LIST). With introduction of atomic stack (that has to be drained at once), we could use it to remove some contention between the threads and simplify the async queue. Fortunately, the reverse order still works for us - instead of append and tail/prev operation on the list, we are now using prepend and head/next operation on the atomic stack.
2023-02-20 16:16:07 +01:00
ISC_STACK(isc_job_t) drain = ISC_ASTACK_TO_STACK(loop->queue_jobs);
isc_job_t *job = ISC_STACK_POP(drain, link);
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
while (job != NULL) {
isc__job_init(loop, job);
isc__job_run(job);
Use atomic stack for async job queue Previously, the async job queue would use a locked-list (ISC_LIST). With introduction of atomic stack (that has to be drained at once), we could use it to remove some contention between the threads and simplify the async queue. Fortunately, the reverse order still works for us - instead of append and tail/prev operation on the list, we are now using prepend and head/next operation on the atomic stack.
2023-02-20 16:16:07 +01:00
job = ISC_STACK_POP(drain, link);
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
}
}
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
static void
wakeup_cb(uv_async_t *handle) {
/* we only woke up to make the loop take a spin */
UNUSED(handle);
}
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
static void
Properly name the loop->mctx The per loop memory context were unnamed, properly name them as 'loop<tid>'.
2022-11-04 15:03:22 +01:00
loop_init(isc_loop_t *loop, isc_loopmgr_t *loopmgr, uint32_t tid) {
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
*loop = (isc_loop_t){
.tid = tid,
.loopmgr = loopmgr,
Use atomic stack for async job queue Previously, the async job queue would use a locked-list (ISC_LIST). With introduction of atomic stack (that has to be drained at once), we could use it to remove some contention between the threads and simplify the async queue. Fortunately, the reverse order still works for us - instead of append and tail/prev operation on the list, we are now using prepend and head/next operation on the atomic stack.
2023-02-20 16:16:07 +01:00
.queue_jobs = ISC_ASTACK_INITIALIZER,
.setup_jobs = ISC_LIST_INITIALIZER,
.teardown_jobs = ISC_LIST_INITIALIZER,
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
};
int r = uv_loop_init(&loop->loop);
UV_RUNTIME_CHECK(uv_loop_init, r);
r = uv_async_init(&loop->loop, &loop->pause_trigger, pauseresume_cb);
UV_RUNTIME_CHECK(uv_async_init, r);
uv_handle_set_data(&loop->pause_trigger, loop);
r = uv_async_init(&loop->loop, &loop->shutdown_trigger, shutdown_cb);
UV_RUNTIME_CHECK(uv_async_init, r);
uv_handle_set_data(&loop->shutdown_trigger, loop);
r = uv_async_init(&loop->loop, &loop->queue_trigger, queue_cb);
UV_RUNTIME_CHECK(uv_async_init, r);
uv_handle_set_data(&loop->queue_trigger, loop);
r = uv_async_init(&loop->loop, &loop->destroy_trigger, destroy_cb);
UV_RUNTIME_CHECK(uv_async_init, r);
uv_handle_set_data(&loop->destroy_trigger, loop);
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
r = uv_async_init(&loop->loop, &loop->wakeup_trigger, wakeup_cb);
UV_RUNTIME_CHECK(uv_async_init, r);
r = uv_prepare_init(&loop->loop, &loop->quiescent);
UV_RUNTIME_CHECK(uv_prepare_init, r);
uv_handle_set_data(&loop->quiescent, loop);
Properly name the loop->mctx The per loop memory context were unnamed, properly name them as 'loop<tid>'.
2022-11-04 15:03:22 +01:00
char name[16];
snprintf(name, sizeof(name), "loop-%08" PRIx32, tid);
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
isc_mem_create(&loop->mctx);
Properly name the loop->mctx The per loop memory context were unnamed, properly name them as 'loop<tid>'.
2022-11-04 15:03:22 +01:00
isc_mem_setname(loop->mctx, name);
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
isc_refcount_init(&loop->references, 1);
loop->magic = LOOP_MAGIC;
}
static void
loop_run(isc_loop_t *loop) {
int r;
isc_job_t *job;
job = ISC_LIST_HEAD(loop->setup_jobs);
while (job != NULL) {
isc_job_t *next = ISC_LIST_NEXT(job, link);
ISC_LIST_UNLINK(loop->setup_jobs, job, link);
isc__job_run(job);
job = next;
}
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
r = uv_prepare_start(&loop->quiescent, isc__qsbr_quiescent_cb);
UV_RUNTIME_CHECK(uv_prepare_start, r);
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
isc_barrier_wait(&loop->loopmgr->starting);
r = uv_run(&loop->loop, UV_RUN_DEFAULT);
UV_RUNTIME_CHECK(uv_run, r);
isc_barrier_wait(&loop->loopmgr->stopping);
}
static void
loop_close(isc_loop_t *loop) {
int r = uv_loop_close(&loop->loop);
UV_RUNTIME_CHECK(uv_loop_close, r);
Use atomic stack for async job queue Previously, the async job queue would use a locked-list (ISC_LIST). With introduction of atomic stack (that has to be drained at once), we could use it to remove some contention between the threads and simplify the async queue. Fortunately, the reverse order still works for us - instead of append and tail/prev operation on the list, we are now using prepend and head/next operation on the atomic stack.
2023-02-20 16:16:07 +01:00
INSIST(ISC_ASTACK_EMPTY(loop->queue_jobs));
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
loop->magic = 0;
isc_mem_detach(&loop->mctx);
}
static isc_threadresult_t
loop_thread(isc_threadarg_t arg) {
isc_loop_t *loop = (isc_loop_t *)arg;
/* Initialize the thread_local variable */
isc__tid_init(loop->tid);
loop_run(loop);
return ((isc_threadresult_t)0);
}
void
isc_loop_nosetup(isc_loop_t *loop, isc_job_t *job) {
ISC_LIST_DEQUEUE(loop->setup_jobs, job, link);
}
void
isc_loop_noteardown(isc_loop_t *loop, isc_job_t *job) {
ISC_LIST_DEQUEUE(loop->teardown_jobs, job, link);
}
/**
* Public
*/
static void
threadpool_initialize(uint32_t workers) {
char buf[11];
int r = uv_os_getenv("UV_THREADPOOL_SIZE", buf,
&(size_t){ sizeof(buf) });
if (r == UV_ENOENT) {
snprintf(buf, sizeof(buf), "%" PRIu32, workers);
uv_os_setenv("UV_THREADPOOL_SIZE", buf);
}
}
static void
loop_destroy(isc_loop_t *loop) {
int r = uv_async_send(&loop->destroy_trigger);
UV_RUNTIME_CHECK(uv_async_send, r);
}
Enhance the isc_loop unit to allow reference count tracking Use ISC_REFCOUNT_TRACE_{IMPL,DECL} to allow better isc_loop reference tracking - use `#define ISC_LOOP_TRACE 1` in <isc/loop.h> to enable.
2023-01-02 13:26:41 +01:00
#if ISC_LOOP_TRACE
ISC_REFCOUNT_TRACE_IMPL(isc_loop, loop_destroy)
#else
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
ISC_REFCOUNT_IMPL(isc_loop, loop_destroy);
Enhance the isc_loop unit to allow reference count tracking Use ISC_REFCOUNT_TRACE_{IMPL,DECL} to allow better isc_loop reference tracking - use `#define ISC_LOOP_TRACE 1` in <isc/loop.h> to enable.
2023-01-02 13:26:41 +01:00
#endif
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
void
isc_loopmgr_create(isc_mem_t *mctx, uint32_t nloops, isc_loopmgr_t **loopmgrp) {
isc_loopmgr_t *loopmgr = NULL;
REQUIRE(loopmgrp != NULL && *loopmgrp == NULL);
REQUIRE(nloops > 0);
threadpool_initialize(nloops);
loopmgr = isc_mem_get(mctx, sizeof(*loopmgr));
*loopmgr = (isc_loopmgr_t){
.nloops = nloops,
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
.qsbr = ISC_QSBR_INITIALIZER(nloops),
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
};
isc_mem_attach(mctx, &loopmgr->mctx);
isc_barrier_init(&loopmgr->pausing, loopmgr->nloops);
isc_barrier_init(&loopmgr->resuming, loopmgr->nloops);
isc_barrier_init(&loopmgr->starting, loopmgr->nloops);
isc_barrier_init(&loopmgr->stopping, loopmgr->nloops);
loopmgr->loops = isc_mem_get(
loopmgr->mctx, loopmgr->nloops * sizeof(loopmgr->loops[0]));
for (size_t i = 0; i < loopmgr->nloops; i++) {
isc_loop_t *loop = &loopmgr->loops[i];
loop_init(loop, loopmgr, i);
}
loopmgr->sigint = isc_signal_new(loopmgr, isc__loopmgr_signal, loopmgr,
SIGINT);
loopmgr->sigterm = isc_signal_new(loopmgr, isc__loopmgr_signal, loopmgr,
SIGTERM);
isc_signal_start(loopmgr->sigint);
isc_signal_start(loopmgr->sigterm);
loopmgr->magic = LOOPMGR_MAGIC;
*loopmgrp = loopmgr;
}
isc_job_t *
isc_loop_setup(isc_loop_t *loop, isc_job_cb cb, void *cbarg) {
isc_loopmgr_t *loopmgr = NULL;
isc_job_t *job = NULL;
REQUIRE(VALID_LOOP(loop));
REQUIRE(cb != NULL);
loopmgr = loop->loopmgr;
REQUIRE(loop->tid == isc_tid() || !atomic_load(&loopmgr->running) ||
atomic_load(&loopmgr->paused));
job = isc__job_new(loop, cb, cbarg);
isc__job_init(loop, job);
/*
* The ISC_LIST_PREPEND is counterintuitive here, but actually, the
* uv_idle_start() puts the item on the HEAD of the internal list, so we
* want to store items here in reverse order, so on the uv_loop, they
* are scheduled in the correct order
*/
ISC_LIST_PREPEND(loop->setup_jobs, job, link);
return (job);
}
isc_job_t *
isc_loop_teardown(isc_loop_t *loop, isc_job_cb cb, void *cbarg) {
isc_loopmgr_t *loopmgr = NULL;
isc_job_t *job = NULL;
REQUIRE(VALID_LOOP(loop));
loopmgr = loop->loopmgr;
REQUIRE(loop->tid == isc_tid() || !atomic_load(&loopmgr->running) ||
atomic_load(&loopmgr->paused));
job = isc__job_new(loop, cb, cbarg);
isc__job_init(loop, job);
/*
* The ISC_LIST_PREPEND is counterintuitive here, but actually, the
* uv_idle_start() puts the item on the HEAD of the internal list, so we
* want to store items here in reverse order, so on the uv_loop, they
* are scheduled in the correct order
*/
ISC_LIST_PREPEND(loop->teardown_jobs, job, link);
return (job);
}
void
isc_loopmgr_setup(isc_loopmgr_t *loopmgr, isc_job_cb cb, void *cbarg) {
REQUIRE(VALID_LOOPMGR(loopmgr));
REQUIRE(!atomic_load(&loopmgr->running) ||
atomic_load(&loopmgr->paused));
for (size_t i = 0; i < loopmgr->nloops; i++) {
isc_loop_t *loop = &loopmgr->loops[i];
(void)isc_loop_setup(loop, cb, cbarg);
}
}
void
isc_loopmgr_teardown(isc_loopmgr_t *loopmgr, isc_job_cb cb, void *cbarg) {
REQUIRE(VALID_LOOPMGR(loopmgr));
REQUIRE(!atomic_load(&loopmgr->running) ||
atomic_load(&loopmgr->paused));
for (size_t i = 0; i < loopmgr->nloops; i++) {
isc_loop_t *loop = &loopmgr->loops[i];
(void)isc_loop_teardown(loop, cb, cbarg);
}
}
void
isc_loopmgr_run(isc_loopmgr_t *loopmgr) {
REQUIRE(VALID_LOOPMGR(loopmgr));
RUNTIME_CHECK(atomic_compare_exchange_strong(&loopmgr->running,
&(bool){ false }, true));
/*
* Always ignore SIGPIPE.
*/
ignore_signal(SIGPIPE, SIG_IGN);
/*
* The thread 0 is this one.
*/
for (size_t i = 1; i < loopmgr->nloops; i++) {
char name[32];
isc_loop_t *loop = &loopmgr->loops[i];
isc_thread_create(loop_thread, loop, &loop->thread);
snprintf(name, sizeof(name), "isc-loop-%04zu", i);
isc_thread_setname(loop->thread, name);
}
loop_thread(&loopmgr->loops[0]);
}
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
void
isc_loopmgr_wakeup(isc_loopmgr_t *loopmgr) {
REQUIRE(VALID_LOOPMGR(loopmgr));
for (size_t i = 0; i < loopmgr->nloops; i++) {
isc_loop_t *loop = &loopmgr->loops[i];
/* Skip current loop */
if (i == isc_tid()) {
continue;
}
uv_async_send(&loop->wakeup_trigger);
}
}
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
void
isc_loopmgr_pause(isc_loopmgr_t *loopmgr) {
REQUIRE(VALID_LOOPMGR(loopmgr));
remove isc_task completely as there is no further use of isc_task in BIND, this commit removes it, along with isc_taskmgr, isc_event, and all other related types. functions that accepted taskmgr as a parameter have been cleaned up. as a result of this change, some functions can no longer fail, so they've been changed to type void, and their callers have been updated accordingly. the tasks table has been removed from the statistics channel and the stats version has been updated. dns_dyndbctx has been changed to reference the loopmgr instead of taskmgr, and DNS_DYNDB_VERSION has been udpated as well.
2022-10-29 14:22:56 -07:00
if (isc_log_wouldlog(isc_lctx, ISC_LOG_DEBUG(1))) {
isc_log_write(isc_lctx, ISC_LOGCATEGORY_GENERAL,
ISC_LOGMODULE_OTHER, ISC_LOG_DEBUG(1),
"loop exclusive mode: starting");
}
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
for (size_t i = 0; i < loopmgr->nloops; i++) {
isc_loop_t *loop = &loopmgr->loops[i];
/* Skip current loop */
if (i == isc_tid()) {
continue;
}
uv_async_send(&loop->pause_trigger);
}
RUNTIME_CHECK(atomic_compare_exchange_strong(&loopmgr->paused,
&(bool){ false }, true));
pause_loop(CURRENT_LOOP(loopmgr));
remove isc_task completely as there is no further use of isc_task in BIND, this commit removes it, along with isc_taskmgr, isc_event, and all other related types. functions that accepted taskmgr as a parameter have been cleaned up. as a result of this change, some functions can no longer fail, so they've been changed to type void, and their callers have been updated accordingly. the tasks table has been removed from the statistics channel and the stats version has been updated. dns_dyndbctx has been changed to reference the loopmgr instead of taskmgr, and DNS_DYNDB_VERSION has been udpated as well.
2022-10-29 14:22:56 -07:00
if (isc_log_wouldlog(isc_lctx, ISC_LOG_DEBUG(1))) {
isc_log_write(isc_lctx, ISC_LOGCATEGORY_GENERAL,
ISC_LOGMODULE_OTHER, ISC_LOG_DEBUG(1),
"loop exclusive mode: started");
}
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
}
void
isc_loopmgr_resume(isc_loopmgr_t *loopmgr) {
REQUIRE(VALID_LOOPMGR(loopmgr));
remove isc_task completely as there is no further use of isc_task in BIND, this commit removes it, along with isc_taskmgr, isc_event, and all other related types. functions that accepted taskmgr as a parameter have been cleaned up. as a result of this change, some functions can no longer fail, so they've been changed to type void, and their callers have been updated accordingly. the tasks table has been removed from the statistics channel and the stats version has been updated. dns_dyndbctx has been changed to reference the loopmgr instead of taskmgr, and DNS_DYNDB_VERSION has been udpated as well.
2022-10-29 14:22:56 -07:00
if (isc_log_wouldlog(isc_lctx, ISC_LOG_DEBUG(1))) {
isc_log_write(isc_lctx, ISC_LOGCATEGORY_GENERAL,
ISC_LOGMODULE_OTHER, ISC_LOG_DEBUG(1),
"loop exclusive mode: ending");
}
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
RUNTIME_CHECK(atomic_compare_exchange_strong(&loopmgr->paused,
&(bool){ true }, false));
resume_loop(CURRENT_LOOP(loopmgr));
remove isc_task completely as there is no further use of isc_task in BIND, this commit removes it, along with isc_taskmgr, isc_event, and all other related types. functions that accepted taskmgr as a parameter have been cleaned up. as a result of this change, some functions can no longer fail, so they've been changed to type void, and their callers have been updated accordingly. the tasks table has been removed from the statistics channel and the stats version has been updated. dns_dyndbctx has been changed to reference the loopmgr instead of taskmgr, and DNS_DYNDB_VERSION has been udpated as well.
2022-10-29 14:22:56 -07:00
if (isc_log_wouldlog(isc_lctx, ISC_LOG_DEBUG(1))) {
isc_log_write(isc_lctx, ISC_LOGCATEGORY_GENERAL,
ISC_LOGMODULE_OTHER, ISC_LOG_DEBUG(1),
"loop exclusive mode: ended");
}
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
}
void
isc_loopmgr_destroy(isc_loopmgr_t **loopmgrp) {
isc_loopmgr_t *loopmgr = NULL;
REQUIRE(loopmgrp != NULL);
REQUIRE(VALID_LOOPMGR(*loopmgrp));
loopmgr = *loopmgrp;
*loopmgrp = NULL;
RUNTIME_CHECK(atomic_compare_exchange_strong(&loopmgr->running,
&(bool){ true }, false));
/* First wait for all loops to finish */
for (size_t i = 1; i < loopmgr->nloops; i++) {
isc_loop_t *loop = &loopmgr->loops[i];
isc_thread_join(loop->thread, NULL);
}
loopmgr->magic = 0;
for (size_t i = 0; i < loopmgr->nloops; i++) {
isc_loop_t *loop = &loopmgr->loops[i];
loop_close(loop);
}
isc_mem_put(loopmgr->mctx, loopmgr->loops,
loopmgr->nloops * sizeof(loopmgr->loops[0]));
isc_barrier_destroy(&loopmgr->starting);
isc_barrier_destroy(&loopmgr->stopping);
isc_barrier_destroy(&loopmgr->resuming);
isc_barrier_destroy(&loopmgr->pausing);
isc_mem_putanddetach(&loopmgr->mctx, loopmgr, sizeof(*loopmgr));
}
uint32_t
isc_loopmgr_nloops(isc_loopmgr_t *loopmgr) {
REQUIRE(VALID_LOOPMGR(loopmgr));
return (loopmgr->nloops);
}
isc_mem_t *
isc_loop_getmctx(isc_loop_t *loop) {
REQUIRE(VALID_LOOP(loop));
return (loop->mctx);
}
isc_loop_t *
isc_loop_main(isc_loopmgr_t *loopmgr) {
REQUIRE(VALID_LOOPMGR(loopmgr));
return (DEFAULT_LOOP(loopmgr));
}
isc_loop_t *
isc_loop_current(isc_loopmgr_t *loopmgr) {
REQUIRE(VALID_LOOPMGR(loopmgr));
return (CURRENT_LOOP(loopmgr));
}
isc_loop_t *
isc_loop_get(isc_loopmgr_t *loopmgr, uint32_t tid) {
REQUIRE(VALID_LOOPMGR(loopmgr));
REQUIRE(tid < loopmgr->nloops);
return (LOOP(loopmgr, tid));
}
void
isc_loopmgr_blocking(isc_loopmgr_t *loopmgr) {
REQUIRE(VALID_LOOPMGR(loopmgr));
isc_signal_stop(loopmgr->sigterm);
isc_signal_stop(loopmgr->sigint);
}
void
isc_loopmgr_nonblocking(isc_loopmgr_t *loopmgr) {
REQUIRE(VALID_LOOPMGR(loopmgr));
isc_signal_start(loopmgr->sigint);
isc_signal_start(loopmgr->sigterm);
}
switch to using isc_loopmgr_pause() instead of task exclusive change functions using isc_taskmgr_beginexclusive() to use isc_loopmgr_pause() instead. also, removed an unnecessary use of exclusive mode in named_server_tcptimeouts(). most functions that were implemented as task events because they needed to be running in a task to use exclusive mode have now been changed into loop callbacks instead. (the exception is catz, which is being changed in a separate commit because it's a particularly complex change.)
2022-10-29 03:33:05 -07:00
isc_loopmgr_t *
isc_loop_getloopmgr(isc_loop_t *loop) {
REQUIRE(VALID_LOOP(loop));
return (loop->loopmgr);
}
Reference in New Issue Copy Permalink