mir/bind
2
0
Fork 0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-24 02:58:38 +00:00
Code Issues Releases Wiki Activity
bind/lib/isc/loop_p.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

157 lines
3.0 KiB
C
Raw Normal View History

New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
/*
* Copyright (C) Internet Systems Consortium, Inc. ("ISC")
*
* SPDX-License-Identifier: MPL-2.0
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, you can obtain one at https://mozilla.org/MPL/2.0/.
*
* See the COPYRIGHT file distributed with this work for additional
* information regarding copyright ownership.
*/
#pragma once
#include <inttypes.h>
#include <isc/barrier.h>
#include <isc/lang.h>
#include <isc/loop.h>
#include <isc/magic.h>
#include <isc/mem.h>
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
#include <isc/qsbr.h>
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
#include <isc/refcount.h>
#include <isc/result.h>
#include <isc/signal.h>
Use atomic stack for async job queue Previously, the async job queue would use a locked-list (ISC_LIST). With introduction of atomic stack (that has to be drained at once), we could use it to remove some contention between the threads and simplify the async queue. Fortunately, the reverse order still works for us - instead of append and tail/prev operation on the list, we are now using prepend and head/next operation on the atomic stack.
2023-02-20 16:16:07 +01:00
#include <isc/stack.h>
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
#include <isc/thread.h>
#include <isc/types.h>
#include <isc/uv.h>
#include <isc/work.h>
/*
* Per-thread loop
*/
#define LOOP_MAGIC ISC_MAGIC('L', 'O', 'O', 'P')
#define VALID_LOOP(t) ISC_MAGIC_VALID(t, LOOP_MAGIC)
typedef ISC_LIST(isc_job_t) isc_joblist_t;
Use atomic stack for async job queue Previously, the async job queue would use a locked-list (ISC_LIST). With introduction of atomic stack (that has to be drained at once), we could use it to remove some contention between the threads and simplify the async queue. Fortunately, the reverse order still works for us - instead of append and tail/prev operation on the list, we are now using prepend and head/next operation on the atomic stack.
2023-02-20 16:16:07 +01:00
typedef ISC_ASTACK(isc_job_t) isc_jobstack_t;
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
struct isc_loop {
int magic;
isc_refcount_t references;
isc_thread_t thread;
isc_loopmgr_t *loopmgr;
uv_loop_t loop;
uint32_t tid;
isc_mem_t *mctx;
/* states */
bool paused;
bool shuttingdown;
/* Async queue */
uv_async_t queue_trigger;
Use atomic stack for async job queue Previously, the async job queue would use a locked-list (ISC_LIST). With introduction of atomic stack (that has to be drained at once), we could use it to remove some contention between the threads and simplify the async queue. Fortunately, the reverse order still works for us - instead of append and tail/prev operation on the list, we are now using prepend and head/next operation on the atomic stack.
2023-02-20 16:16:07 +01:00
isc_jobstack_t queue_jobs;
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
/* Pause */
uv_async_t pause_trigger;
/* Shutdown */
uv_async_t shutdown_trigger;
isc_joblist_t setup_jobs;
isc_joblist_t teardown_jobs;
/* Destroy */
uv_async_t destroy_trigger;
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
/* safe memory reclamation */
uv_async_t wakeup_trigger;
uv_prepare_t quiescent;
isc_qsbr_phase_t qsbr_phase;
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
};
/*
* Loop Manager
*/
#define LOOPMGR_MAGIC ISC_MAGIC('L', 'o', 'o', 'M')
#define VALID_LOOPMGR(t) ISC_MAGIC_VALID(t, LOOPMGR_MAGIC)
struct isc_loopmgr {
int magic;
isc_mem_t *mctx;
uint_fast32_t nloops;
atomic_bool shuttingdown;
atomic_bool running;
atomic_bool paused;
/* signal handling */
isc_signal_t *sigint;
isc_signal_t *sigterm;
/* pause/resume */
isc_barrier_t pausing;
isc_barrier_t resuming;
/* start/stop */
isc_barrier_t starting;
/* stopping */
isc_barrier_t stopping;
/* per-thread objects */
isc_loop_t *loops;
QSBR: safe memory reclamation for lock-free data structures This "quiescent state based reclamation" module provides support for the qp-trie module in dns/qp. It is a replacement for liburcu, written without reference to the urcu source code, and in fact it works in a significantly different way. A few specifics of BIND make this variant of QSBR somewhat simpler: * We can require that wait-free access to a qp-trie only happens in an isc_loop callback. The loop provides a natural quiescent state, after the callbacks are done, when no qp-trie access occurs. * We can dispense with any API like rcu_synchronize(). In practice, it takes far too long to wait for a grace period to elapse for each write to a data structure. * We use the idea of "phases" (aka epochs or eras) from EBR to reduce the amount of bookkeeping needed to track memory that is no longer needed, knowing that the qp-trie does most of that work already. I considered hazard pointers for safe memory reclamation. They have more read-side overhead (updating the hazard pointers) and it wasn't clear to me how to nicely schedule the cleanup work. Another alternative, epoch-based reclamation, is designed for fine-grained lock-free updates, so it needs some rethinking to work well with the heavily read-biased design of the qp-trie. QSBR has the fastest read side of the basic SMR algorithms (with no barriers), and fits well into a libuv loop. More recent hybrid SMR algorithms do not appear to have enough benefits to justify the extra complexity.
2022-12-29 19:18:00 +00:00
/* safe memory reclamation */
isc_qsbr_t qsbr;
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
};
/*
* Signal Handler
*/
struct isc_signal {
uv_signal_t signal;
isc_loop_t *loop;
isc_signal_cb cb;
void *cbarg;
int signum;
};
/*
* Job to be scheduled in an event loop
*/
#define JOB_MAGIC ISC_MAGIC('J', 'O', 'B', ' ')
#define VALID_JOB(t) ISC_MAGIC_VALID(t, JOB_MAGIC)
struct isc_job {
int magic;
uv_idle_t idle;
isc_loop_t *loop;
isc_job_cb cb;
void *cbarg;
Use atomic stack for async job queue Previously, the async job queue would use a locked-list (ISC_LIST). With introduction of atomic stack (that has to be drained at once), we could use it to remove some contention between the threads and simplify the async queue. Fortunately, the reverse order still works for us - instead of append and tail/prev operation on the list, we are now using prepend and head/next operation on the atomic stack.
2023-02-20 16:16:07 +01:00
ISC_LINK(isc_job_t) link;
New event loop handling API This commit introduces new APIs for applications and signal handling, intended to replace isc_app for applications built on top of libisc. * isc_app will be replaced with isc_loopmgr, which handles the starting and stopping of applications. In isc_loopmgr, the main thread is not blocked, but is part of the working thread set. The loop manager will start a number of threads, each with a uv_loop event loop running. Setup and teardown functions can be assigned which will run when the loop starts and stops, and jobs can be scheduled to run in the meantime. When isc_loopmgr_shutdown() is run from any the loops, all loops will shut down and the application can terminate. * signal handling will now be handled with a separate isc_signal unit. isc_loopmgr only handles SIGTERM and SIGINT for application termination, but the application may install additional signal handlers, such as SIGHUP as a signal to reload configuration. * new job running primitives, isc_job and isc_async, have been added. Both units schedule callbacks (specifying a callback function and argument) on an event loop. The difference is that isc_job unit is unlocked and not thread-safe, so it can be used to efficiently run jobs in the same thread, while isc_async is thread-safe and uses locking, so it can be used to pass jobs from one thread to another. * isc_tid will be used to track the thread ID in isc_loop worker threads. * unit tests have been added for the new APIs.
2022-07-26 13:03:22 +02:00
};
/*
* Work to be offloaded to an external thread.
*/
struct isc_work {
uv_work_t work;
isc_loop_t *loop;
isc_work_cb work_cb;
isc_after_work_cb after_work_cb;
void *cbarg;
};
#define DEFAULT_LOOP(loopmgr) (&(loopmgr)->loops[0])
#define CURRENT_LOOP(loopmgr) (&(loopmgr)->loops[isc_tid()])
#define LOOP(loopmgr, tid) (&(loopmgr)->loops[tid])
#define ON_LOOP(loop) ((loop) == CURRENT_LOOP((loop)->loopmgr))
Reference in New Issue Copy Permalink