mir/bind
2
0
Fork 0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-23 10:39:16 +00:00
Code Issues Releases Wiki Activity
bind/lib/isc/netmgr/netmgr.c

3753 lines
88 KiB
C
Raw Normal View History

netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
* Copyright (C) Internet Systems Consortium, Inc. ("ISC")
*
Update the copyright information in all files in the repository This commit converts the license handling to adhere to the REUSE specification. It specifically: 1. Adds used licnses to LICENSES/ directory 2. Add "isc" template for adding the copyright boilerplate 3. Changes all source files to include copyright and SPDX license header, this includes all the C sources, documentation, zone files, configuration files. There are notes in the doc/dev/copyrights file on how to add correct headers to the new files. 4. Handle the rest that can't be modified via .reuse/dep5 file. The binary (or otherwise unmodifiable) files could have license places next to them in <foo>.license file, but this would lead to cluttered repository and most of the files handled in the .reuse/dep5 file are system test files.
2021-06-03 08:37:05 +02:00
* SPDX-License-Identifier: MPL-2.0
*
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
update all copyright headers to eliminate the typo
2020-09-14 16:20:40 -07:00
* file, you can obtain one at https://mozilla.org/MPL/2.0/.
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*
* See the COPYRIGHT file distributed with this work for additional
* information regarding copyright ownership.
*/
#include <inttypes.h>
#include <unistd.h>
#include <uv.h>
#include <isc/atomic.h>
Add support for generating backtraces on Windows This commit adds support for generating backtraces on Windows and refactors the isc_backtrace API to match the Linux/BSD API (without the isc_ prefix) * isc_backtrace_gettrace() was renamed to isc_backtrace(), the third argument was removed and the return type was changed to int * isc_backtrace_symbols() was added * isc_backtrace_symbols_fd() was added and used as appropriate
2021-04-27 16:20:03 +02:00
#include <isc/backtrace.h>
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
#include <isc/barrier.h>
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
#include <isc/buffer.h>
#include <isc/condition.h>
netmgr: Don't crash if socket() returns an error in udpconnect socket() call can return an error - e.g. EMFILE, so we need to handle this nicely and not crash. Additionally wrap the socket() call inside a platform independent helper function as the Socket data type on Windows is unsigned integer: > This means, for example, that checking for errors when the socket and > accept functions return should not be done by comparing the return > value with –1, or seeing if the value is negative (both common and > legal approaches in UNIX). Instead, an application should use the > manifest constant INVALID_SOCKET as defined in the Winsock2.h header > file.
2020-11-07 20:48:37 +01:00
#include <isc/errno.h>
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
#include <isc/list.h>
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
#include <isc/log.h>
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
#include <isc/magic.h>
#include <isc/mem.h>
#include <isc/netmgr.h>
#include <isc/print.h>
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
#include <isc/quota.h>
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
#include <isc/random.h>
#include <isc/refcount.h>
#include <isc/region.h>
#include <isc/result.h>
#include <isc/sockaddr.h>
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
#include <isc/stats.h>
netmgr: Don't crash if socket() returns an error in udpconnect socket() call can return an error - e.g. EMFILE, so we need to handle this nicely and not crash. Additionally wrap the socket() call inside a platform independent helper function as the Socket data type on Windows is unsigned integer: > This means, for example, that checking for errors when the socket and > accept functions return should not be done by comparing the return > value with –1, or seeing if the value is negative (both common and > legal approaches in UNIX). Instead, an application should use the > manifest constant INVALID_SOCKET as defined in the Winsock2.h header > file.
2020-11-07 20:48:37 +01:00
#include <isc/strerr.h>
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
#include <isc/task.h>
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
#include <isc/thread.h>
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
#include <isc/tls.h>
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
#include <isc/util.h>
#include "netmgr-int.h"
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
#include "netmgr_p.h"
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
#include "openssl_shim.h"
Add asynchronous work API to the network manager The libuv has a support for running long running tasks in the dedicated threadpools, so it doesn't affect networking IO. This commit adds isc_nm_work_enqueue() wrapper that would wraps around the libuv API and runs it on top of associated worker loop. The only limitation is that the function must be called from inside network manager thread, so the call to the function should be wrapped inside a (bound) task.
2021-05-27 09:45:07 +02:00
#include "trampoline_p.h"
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
#include "uv-compat.h"
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Increase inactivehandles and inactivereqs size for better reuse.
2020-01-29 15:16:02 +01:00
/*%
* How many isc_nmhandles and isc_nm_uvreqs will we be
* caching for reuse in a socket.
*/
#define ISC_NM_HANDLES_STACK_SIZE 600
#define ISC_NM_REQS_STACK_SIZE 600
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
/*%
* Shortcut index arrays to get access to statistics counters.
*/
static const isc_statscounter_t udp4statsindex[] = {
isc_sockstatscounter_udp4open,
isc_sockstatscounter_udp4openfail,
isc_sockstatscounter_udp4close,
isc_sockstatscounter_udp4bindfail,
isc_sockstatscounter_udp4connectfail,
isc_sockstatscounter_udp4connect,
-1,
-1,
isc_sockstatscounter_udp4sendfail,
isc_sockstatscounter_udp4recvfail,
isc_sockstatscounter_udp4active
};
static const isc_statscounter_t udp6statsindex[] = {
isc_sockstatscounter_udp6open,
isc_sockstatscounter_udp6openfail,
isc_sockstatscounter_udp6close,
isc_sockstatscounter_udp6bindfail,
isc_sockstatscounter_udp6connectfail,
isc_sockstatscounter_udp6connect,
-1,
-1,
isc_sockstatscounter_udp6sendfail,
isc_sockstatscounter_udp6recvfail,
isc_sockstatscounter_udp6active
};
static const isc_statscounter_t tcp4statsindex[] = {
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
isc_sockstatscounter_tcp4open, isc_sockstatscounter_tcp4openfail,
isc_sockstatscounter_tcp4close, isc_sockstatscounter_tcp4bindfail,
isc_sockstatscounter_tcp4connectfail, isc_sockstatscounter_tcp4connect,
isc_sockstatscounter_tcp4acceptfail, isc_sockstatscounter_tcp4accept,
isc_sockstatscounter_tcp4sendfail, isc_sockstatscounter_tcp4recvfail,
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
isc_sockstatscounter_tcp4active
};
static const isc_statscounter_t tcp6statsindex[] = {
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
isc_sockstatscounter_tcp6open, isc_sockstatscounter_tcp6openfail,
isc_sockstatscounter_tcp6close, isc_sockstatscounter_tcp6bindfail,
isc_sockstatscounter_tcp6connectfail, isc_sockstatscounter_tcp6connect,
isc_sockstatscounter_tcp6acceptfail, isc_sockstatscounter_tcp6accept,
isc_sockstatscounter_tcp6sendfail, isc_sockstatscounter_tcp6recvfail,
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
isc_sockstatscounter_tcp6active
};
#if 0
/* XXX: not currently used */
static const isc_statscounter_t unixstatsindex[] = {
isc_sockstatscounter_unixopen,
isc_sockstatscounter_unixopenfail,
isc_sockstatscounter_unixclose,
isc_sockstatscounter_unixbindfail,
isc_sockstatscounter_unixconnectfail,
isc_sockstatscounter_unixconnect,
isc_sockstatscounter_unixacceptfail,
isc_sockstatscounter_unixaccept,
isc_sockstatscounter_unixsendfail,
isc_sockstatscounter_unixrecvfail,
isc_sockstatscounter_unixactive
};
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
#endif /* if 0 */
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
* libuv is not thread safe, but has mechanisms to pass messages
* between threads. Each socket is owned by a thread. For UDP
* sockets we have a set of sockets for each interface and we can
* choose a sibling and send the message directly. For TCP, or if
* we're calling from a non-networking thread, we need to pass the
* request using async_cb.
*/
Complete rewrite the BIND 9 build system The rewrite of BIND 9 build system is a large work and cannot be reasonable split into separate merge requests. Addition of the automake has a positive effect on the readability and maintainability of the build system as it is more declarative, it allows conditional and we are able to drop all of the custom make code that BIND 9 developed over the years to overcome the deficiencies of autoconf + custom Makefile.in files. This squashed commit contains following changes: - conversion (or rather fresh rewrite) of all Makefile.in files to Makefile.am by using automake - the libtool is now properly integrated with automake (the way we used it was rather hackish as the only official way how to use libtool is via automake - the dynamic module loading was rewritten from a custom patchwork to libtool's libltdl (which includes the patchwork to support module loading on different systems internally) - conversion of the unit test executor from kyua to automake parallel driver - conversion of the system test executor from custom make/shell to automake parallel driver - The GSSAPI has been refactored, the custom SPNEGO on the basis that all major KRB5/GSSAPI (mit-krb5, heimdal and Windows) implementations support SPNEGO mechanism. - The various defunct tests from bin/tests have been removed: bin/tests/optional and bin/tests/pkcs11 - The text files generated from the MD files have been removed, the MarkDown has been designed to be readable by both humans and computers - The xsl header is now generated by a simple sed command instead of perl helper - The <irs/platform.h> header has been removed - cleanups of configure.ac script to make it more simpler, addition of multiple macros (there's still work to be done though) - the tarball can now be prepared with `make dist` - the system tests are partially able to run in oot build Here's a list of unfinished work that needs to be completed in subsequent merge requests: - `make distcheck` doesn't yet work (because of system tests oot run is not yet finished) - documentation is not yet built, there's a different merge request with docbook to sphinx-build rst conversion that needs to be rebased and adapted on top of the automake - msvc build is non functional yet and we need to decide whether we will just cross-compile bind9 using mingw-w64 or fix the msvc build - contributed dlz modules are not included neither in the autoconf nor automake
2018-08-07 16:46:53 +02:00
static thread_local int isc__nm_tid_v = ISC_NETMGR_TID_UNKNOWN;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
remove all references to isc_socket and related types Removed socket.c, socket.h, and all references to isc_socket_t, isc_socketmgr_t, isc_sockevent_t, etc.
2021-10-03 00:27:52 -07:00
/*
* Set by the -T dscp option on the command line. If set to a value
* other than -1, we check to make sure DSCP values match it, and
* assert if not. (Not currently in use.)
*/
int isc_dscp_check_value = -1;
Reformat using the new rules
2020-02-14 08:14:03 +01:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_maybe_destroy(isc_nmsocket_t *sock FLARG);
Reformat using the new rules
2020-02-14 08:14:03 +01:00
static void
nmhandle_free(isc_nmsocket_t *sock, isc_nmhandle_t *handle);
static isc_threadresult_t
nm_thread(isc_threadarg_t worker0);
static void
async_cb(uv_async_t *handle);
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static bool
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
process_netievent(isc__networker_t *worker, isc__netievent_t *ievent);
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
static isc_result_t
process_queue(isc__networker_t *worker, netievent_type_t type);
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
static void
wait_for_priority_queue(isc__networker_t *worker);
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
static void
drain_queue(isc__networker_t *worker, netievent_type_t type);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_async_stop(isc__networker_t *worker, isc__netievent_t *ev0);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_async_pause(isc__networker_t *worker, isc__netievent_t *ev0);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_async_resume(isc__networker_t *worker, isc__netievent_t *ev0);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
static void
isc__nm_async_detach(isc__networker_t *worker, isc__netievent_t *ev0);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
static void
isc__nm_async_close(isc__networker_t *worker, isc__netievent_t *ev0);
Add asynchronous work API to the network manager The libuv has a support for running long running tasks in the dedicated threadpools, so it doesn't affect networking IO. This commit adds isc_nm_work_enqueue() wrapper that would wraps around the libuv API and runs it on top of associated worker loop. The only limitation is that the function must be called from inside network manager thread, so the call to the function should be wrapped inside a (bound) task.
2021-05-27 09:45:07 +02:00
static void
isc__nm_threadpool_initialize(uint32_t workers);
static void
isc__nm_work_cb(uv_work_t *req);
static void
isc__nm_after_work_cb(uv_work_t *req, int status);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
/*%<
* Issue a 'handle closed' callback on the socket.
*/
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmhandle_detach_cb(isc_nmhandle_t **handlep FLARG);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
int
Complete rewrite the BIND 9 build system The rewrite of BIND 9 build system is a large work and cannot be reasonable split into separate merge requests. Addition of the automake has a positive effect on the readability and maintainability of the build system as it is more declarative, it allows conditional and we are able to drop all of the custom make code that BIND 9 developed over the years to overcome the deficiencies of autoconf + custom Makefile.in files. This squashed commit contains following changes: - conversion (or rather fresh rewrite) of all Makefile.in files to Makefile.am by using automake - the libtool is now properly integrated with automake (the way we used it was rather hackish as the only official way how to use libtool is via automake - the dynamic module loading was rewritten from a custom patchwork to libtool's libltdl (which includes the patchwork to support module loading on different systems internally) - conversion of the unit test executor from kyua to automake parallel driver - conversion of the system test executor from custom make/shell to automake parallel driver - The GSSAPI has been refactored, the custom SPNEGO on the basis that all major KRB5/GSSAPI (mit-krb5, heimdal and Windows) implementations support SPNEGO mechanism. - The various defunct tests from bin/tests have been removed: bin/tests/optional and bin/tests/pkcs11 - The text files generated from the MD files have been removed, the MarkDown has been designed to be readable by both humans and computers - The xsl header is now generated by a simple sed command instead of perl helper - The <irs/platform.h> header has been removed - cleanups of configure.ac script to make it more simpler, addition of multiple macros (there's still work to be done though) - the tarball can now be prepared with `make dist` - the system tests are partially able to run in oot build Here's a list of unfinished work that needs to be completed in subsequent merge requests: - `make distcheck` doesn't yet work (because of system tests oot run is not yet finished) - documentation is not yet built, there's a different merge request with docbook to sphinx-build rst conversion that needs to be rebased and adapted on top of the automake - msvc build is non functional yet and we need to decide whether we will just cross-compile bind9 using mingw-w64 or fix the msvc build - contributed dlz modules are not included neither in the autoconf nor automake
2018-08-07 16:46:53 +02:00
isc_nm_tid(void) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
return (isc__nm_tid_v);
}
bool
Complete rewrite the BIND 9 build system The rewrite of BIND 9 build system is a large work and cannot be reasonable split into separate merge requests. Addition of the automake has a positive effect on the readability and maintainability of the build system as it is more declarative, it allows conditional and we are able to drop all of the custom make code that BIND 9 developed over the years to overcome the deficiencies of autoconf + custom Makefile.in files. This squashed commit contains following changes: - conversion (or rather fresh rewrite) of all Makefile.in files to Makefile.am by using automake - the libtool is now properly integrated with automake (the way we used it was rather hackish as the only official way how to use libtool is via automake - the dynamic module loading was rewritten from a custom patchwork to libtool's libltdl (which includes the patchwork to support module loading on different systems internally) - conversion of the unit test executor from kyua to automake parallel driver - conversion of the system test executor from custom make/shell to automake parallel driver - The GSSAPI has been refactored, the custom SPNEGO on the basis that all major KRB5/GSSAPI (mit-krb5, heimdal and Windows) implementations support SPNEGO mechanism. - The various defunct tests from bin/tests have been removed: bin/tests/optional and bin/tests/pkcs11 - The text files generated from the MD files have been removed, the MarkDown has been designed to be readable by both humans and computers - The xsl header is now generated by a simple sed command instead of perl helper - The <irs/platform.h> header has been removed - cleanups of configure.ac script to make it more simpler, addition of multiple macros (there's still work to be done though) - the tarball can now be prepared with `make dist` - the system tests are partially able to run in oot build Here's a list of unfinished work that needs to be completed in subsequent merge requests: - `make distcheck` doesn't yet work (because of system tests oot run is not yet finished) - documentation is not yet built, there's a different merge request with docbook to sphinx-build rst conversion that needs to be rebased and adapted on top of the automake - msvc build is non functional yet and we need to decide whether we will just cross-compile bind9 using mingw-w64 or fix the msvc build - contributed dlz modules are not included neither in the autoconf nor automake
2018-08-07 16:46:53 +02:00
isc__nm_in_netthread(void) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
return (isc__nm_tid_v >= 0);
}
Reduce the number of clientmgr objects created Previously, as a way of reducing the contention between threads a clientmgr object would be created for each interface/IP address. We tasks being more strictly bound to netmgr workers, this is no longer needed and we can just create clientmgr object per worker queue (ncpus). Each clientmgr object than would have a single task and single memory context.
2021-05-22 18:12:11 +02:00
void
isc__nm_force_tid(int tid) {
isc__nm_tid_v = tid;
}
Add asynchronous work API to the network manager The libuv has a support for running long running tasks in the dedicated threadpools, so it doesn't affect networking IO. This commit adds isc_nm_work_enqueue() wrapper that would wraps around the libuv API and runs it on top of associated worker loop. The only limitation is that the function must be called from inside network manager thread, so the call to the function should be wrapped inside a (bound) task.
2021-05-27 09:45:07 +02:00
static void
isc__nm_threadpool_initialize(uint32_t workers) {
char buf[11];
int r = uv_os_getenv("UV_THREADPOOL_SIZE", buf,
&(size_t){ sizeof(buf) });
if (r == UV_ENOENT) {
snprintf(buf, sizeof(buf), "%" PRIu32, workers);
uv_os_setenv("UV_THREADPOOL_SIZE", buf);
}
}
Abort when libuv at runtime mismatches libuv at compile time When we compile with libuv that has some capabilities via flags passed to f.e. uv_udp_listen() or uv_udp_bind(), the call with such flags would fail with invalid arguments when older libuv version is linked at the runtime that doesn't understand the flag that was available at the compile time. Enforce minimal libuv version when flags have been available at the compile time, but are not available at the runtime. This check is less strict than enforcing the runtime libuv version to be same or higher than compile time libuv version.
2022-04-25 14:43:14 +02:00
#if HAVE_DECL_UV_UDP_LINUX_RECVERR
#define MINIMAL_UV_VERSION UV_VERSION(1, 42, 0)
#elif HAVE_DECL_UV_UDP_MMSG_FREE
#define MINIMAL_UV_VERSION UV_VERSION(1, 40, 0)
#elif HAVE_DECL_UV_UDP_RECVMMSG
#define MINIMAL_UV_VERSION UV_VERSION(1, 37, 0)
#elif HAVE_DECL_UV_UDP_MMSG_CHUNK
#define MINIMAL_UV_VERSION UV_VERSION(1, 35, 0)
#else
#define MINIMAL_UV_VERSION UV_VERSION(1, 0, 0)
#endif
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
void
isc__netmgr_create(isc_mem_t *mctx, uint32_t workers, isc_nm_t **netmgrp) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nm_t *mgr = NULL;
apply the modified style
2020-02-13 14:44:37 -08:00
char name[32];
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
REQUIRE(workers > 0);
Abort when libuv at runtime mismatches libuv at compile time When we compile with libuv that has some capabilities via flags passed to f.e. uv_udp_listen() or uv_udp_bind(), the call with such flags would fail with invalid arguments when older libuv version is linked at the runtime that doesn't understand the flag that was available at the compile time. Enforce minimal libuv version when flags have been available at the compile time, but are not available at the runtime. This check is less strict than enforcing the runtime libuv version to be same or higher than compile time libuv version.
2022-04-25 14:43:14 +02:00
if (uv_version() < MINIMAL_UV_VERSION) {
isc_error_fatal(__FILE__, __LINE__,
"libuv version too old: running with libuv %s "
"when compiled with libuv %s will lead to "
"libuv failures because of unknown flags",
uv_version_string(), UV_VERSION_STRING);
}
Add asynchronous work API to the network manager The libuv has a support for running long running tasks in the dedicated threadpools, so it doesn't affect networking IO. This commit adds isc_nm_work_enqueue() wrapper that would wraps around the libuv API and runs it on top of associated worker loop. The only limitation is that the function must be called from inside network manager thread, so the call to the function should be wrapped inside a (bound) task.
2021-05-27 09:45:07 +02:00
isc__nm_threadpool_initialize(workers);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
mgr = isc_mem_get(mctx, sizeof(*mgr));
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*mgr = (isc_nm_t){ .nworkers = workers };
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_mem_attach(mctx, &mgr->mctx);
isc_mutex_init(&mgr->lock);
isc_condition_init(&mgr->wkstatecond);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
isc_condition_init(&mgr->wkpausecond);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_refcount_init(&mgr->references, 1);
atomic_init(&mgr->maxudp, 0);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
atomic_init(&mgr->interlocked, ISC_NETMGR_NON_INTERLOCKED);
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
atomic_init(&mgr->workers_paused, 0);
Add missing initialisations configuring with --enable-mutex-atomics flagged these incorrectly initialised variables on systems where pthread_mutex_init doesn't just zero out the structure.
2021-05-26 17:55:43 +10:00
atomic_init(&mgr->paused, false);
atomic_init(&mgr->closing, false);
atomic_init(&mgr->recv_tcp_buffer_size, 0);
atomic_init(&mgr->send_tcp_buffer_size, 0);
atomic_init(&mgr->recv_udp_buffer_size, 0);
atomic_init(&mgr->send_udp_buffer_size, 0);
Add option to configure load balance sockets Previously, the option to enable kernel load balancing of the sockets was always enabled when supported by the operating system (SO_REUSEPORT on Linux and SO_REUSEPORT_LB on FreeBSD). It was reported that in scenarios where the networking threads are also responsible for processing long-running tasks (like RPZ processing, CATZ processing or large zone transfers), this could lead to intermitten brownouts for some clients, because the thread assigned by the operating system might be busy. In such scenarious, the overall performance would be better served by threads competing over the sockets because the idle threads can pick up the incoming traffic. Add new configuration option (`load-balance-sockets`) to allow enabling or disabling the load balancing of the sockets.
2022-04-01 14:43:14 +02:00
#if HAVE_SO_REUSEPORT_LB
mgr->load_balance_sockets = true;
#else
mgr->load_balance_sockets = false;
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
ISC_LIST_INIT(mgr->active_sockets);
#endif
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
/*
* Default TCP timeout values.
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
* May be updated by isc_nm_tcptimeouts().
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
*/
Fix the data race in accessing the isc_nm_t timers The following TSAN report about accessing the mgr timers (mgr->init, mgr->idle, mgr->keepalive and mgr->advertised) has been fixed in this commit: ================== WARNING: ThreadSanitizer: data race (pid=2746) Read of size 4 at 0x7b440008a948 by thread T18: #0 isc__nm_tcpdns_read /home/ondrej/Projects/bind9/lib/isc/netmgr/tcpdns.c:849:25 (libisc.so.1706+0x2ba0f) #1 isc_nm_read /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1679:3 (libisc.so.1706+0x22258) #2 tcpdns_connect_connect_cb /home/ondrej/Projects/bind9/lib/isc/tests/tcpdns_test.c:363:2 (tcpdns_test+0x4bc5fb) #3 isc__nm_async_connectcb /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1816:2 (libisc.so.1706+0x228c9) #4 isc__nm_connectcb /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1791:3 (libisc.so.1706+0x22713) #5 tcpdns_connect_cb /home/ondrej/Projects/bind9/lib/isc/netmgr/tcpdns.c:343:2 (libisc.so.1706+0x2d89d) #6 uv__stream_connect /home/ondrej/Projects/tsan/libuv/src/unix/stream.c:1381:5 (libuv.so.1+0x27c18) #7 uv__stream_io /home/ondrej/Projects/tsan/libuv/src/unix/stream.c:1298:5 (libuv.so.1+0x25977) #8 uv__io_poll /home/ondrej/Projects/tsan/libuv/src/unix/linux-core.c:462:11 (libuv.so.1+0x2e795) #9 uv_run /home/ondrej/Projects/tsan/libuv/src/unix/core.c:385:5 (libuv.so.1+0x158ec) #10 nm_thread /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:530:11 (libisc.so.1706+0x1c94a) Previous write of size 4 at 0x7b440008a948 by main thread: #0 isc_nm_settimeouts /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:490:12 (libisc.so.1706+0x1dda5) #1 tcpdns_recv_two /home/ondrej/Projects/bind9/lib/isc/tests/tcpdns_test.c:601:2 (tcpdns_test+0x4bad0e) #2 cmocka_run_one_test_or_fixture <null> (libcmocka.so.0+0x70be) #3 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16 (libc.so.6+0x2409a) Location is heap block of size 281 at 0x7b440008a840 allocated by main thread: #0 malloc <null> (tcpdns_test+0x42864b) #1 default_memalloc /home/ondrej/Projects/bind9/lib/isc/mem.c:713:8 (libisc.so.1706+0x6d261) #2 mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:622:8 (libisc.so.1706+0x69b9c) #3 isc___mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:1044:9 (libisc.so.1706+0x6d379) #4 isc__mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:2432:10 (libisc.so.1706+0x6889e) #5 isc_nm_start /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:203:8 (libisc.so.1706+0x1c219) #6 nm_setup /home/ondrej/Projects/bind9/lib/isc/tests/tcpdns_test.c:244:11 (tcpdns_test+0x4baaa4) #7 cmocka_run_one_test_or_fixture <null> (libcmocka.so.0+0x70fd) #8 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16 (libc.so.6+0x2409a) Thread T18 'isc-net-0000' (tid=3513, running) created by main thread at: #0 pthread_create <null> (tcpdns_test+0x429e7b) #1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/pthreads/thread.c:73:8 (libisc.so.1706+0x8476a) #2 isc_nm_start /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:271:3 (libisc.so.1706+0x1c66a) #3 nm_setup /home/ondrej/Projects/bind9/lib/isc/tests/tcpdns_test.c:244:11 (tcpdns_test+0x4baaa4) #4 cmocka_run_one_test_or_fixture <null> (libcmocka.so.0+0x70fd) #5 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16 (libc.so.6+0x2409a) SUMMARY: ThreadSanitizer: data race /home/ondrej/Projects/bind9/lib/isc/netmgr/tcpdns.c:849:25 in isc__nm_tcpdns_read ================== ThreadSanitizer: reported 1 warnings
2020-12-02 09:52:39 +01:00
atomic_init(&mgr->init, 30000);
atomic_init(&mgr->idle, 30000);
atomic_init(&mgr->keepalive, 30000);
atomic_init(&mgr->advertised, 30000);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
isc_barrier_init(&mgr->pausing, workers);
isc_barrier_init(&mgr->resuming, workers);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
mgr->workers = isc_mem_get(mctx, workers * sizeof(isc__networker_t));
for (size_t i = 0; i < workers; i++) {
isc__networker_t *worker = &mgr->workers[i];
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
int r;
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*worker = (isc__networker_t){
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
.mgr = mgr,
.id = i,
};
r = uv_loop_init(&worker->loop);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_loop_init, r);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
worker->loop.data = &mgr->workers[i];
r = uv_async_init(&worker->loop, &worker->async, async_cb);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_async_init, r);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
for (size_t type = 0; type < NETIEVENT_MAX; type++) {
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
isc_mutex_init(&worker->ievents[type].lock);
isc_condition_init(&worker->ievents[type].cond);
ISC_LIST_INIT(worker->ievents[type].list);
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
}
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
Make nm->recvbuf larger and heap allocated, to allow uv_recvmmsg usage. Upcoming version of libuv will suport uv_recvmmsg and uv_sendmmsg. To use uv_recvmmsg we need to provide a larger buffer and be able to properly free it.
2020-01-29 13:16:04 +01:00
worker->recvbuf = isc_mem_get(mctx, ISC_NETMGR_RECVBUF_SIZE);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
worker->sendbuf = isc_mem_get(mctx, ISC_NETMGR_SENDBUF_SIZE);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
* We need to do this here and not in nm_thread to avoid a
* race - we could exit isc_nm_start, launch nm_destroy,
* and nm_thread would still not be up.
*/
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
mgr->workers_running++;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_thread_create(nm_thread, &mgr->workers[i], &worker->thread);
snprintf(name, sizeof(name), "isc-net-%04zu", i);
isc_thread_setname(worker->thread, name);
}
mgr->magic = NM_MAGIC;
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
*netmgrp = mgr;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
/*
* Free the resources of the network manager.
*/
static void
apply the modified style
2020-02-13 14:44:37 -08:00
nm_destroy(isc_nm_t **mgr0) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NM(*mgr0));
REQUIRE(!isc__nm_in_netthread());
isc_nm_t *mgr = *mgr0;
Clear the pointer to destroyed object early using the semantic patch Also disable the semantic patch as the code needs tweaks here and there because some destroy functions might not destroy the object and return early if the object is still in use.
2020-02-08 04:37:54 -08:00
*mgr0 = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Add isc_refcount_destroy() call to nm_destroy()
2019-12-10 10:47:08 +01:00
isc_refcount_destroy(&mgr->references);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
mgr->magic = 0;
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
for (int i = 0; i < mgr->nworkers; i++) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc__networker_t *worker = &mgr->workers[i];
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__netievent_t *event = isc__nm_get_netievent_stop(mgr);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc__nm_enqueue_ievent(worker, event);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
address lock order inversion
2019-12-12 17:43:03 +11:00
LOCK(&mgr->lock);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
while (mgr->workers_running > 0) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
WAIT(&mgr->wkstatecond, &mgr->lock);
}
UNLOCK(&mgr->lock);
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
for (int i = 0; i < mgr->nworkers; i++) {
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
isc__networker_t *worker = &mgr->workers[i];
apply the modified style
2020-02-13 14:44:37 -08:00
int r;
style, comments
2019-12-03 00:07:59 -08:00
r = uv_loop_close(&worker->loop);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_loop_close, r);
style, comments
2019-12-03 00:07:59 -08:00
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
for (size_t type = 0; type < NETIEVENT_MAX; type++) {
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
INSIST(ISC_LIST_EMPTY(worker->ievents[type].list));
isc_condition_destroy(&worker->ievents[type].cond);
isc_mutex_destroy(&worker->ievents[type].lock);
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
}
Add missing isc_mutex_destroy and isc_conditional_destroy calls. While harmless on Linux, missing isc_{mutex,conditional}_destroy causes a memory leak on *BSD. Missing calls were added.
2020-05-28 12:34:37 +02:00
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
isc_mem_put(mgr->mctx, worker->sendbuf,
ISC_NETMGR_SENDBUF_SIZE);
Make nm->recvbuf larger and heap allocated, to allow uv_recvmmsg usage. Upcoming version of libuv will suport uv_recvmmsg and uv_sendmmsg. To use uv_recvmmsg we need to provide a larger buffer and be able to properly free it.
2020-01-29 13:16:04 +01:00
isc_mem_put(mgr->mctx, worker->recvbuf,
ISC_NETMGR_RECVBUF_SIZE);
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
isc_thread_join(worker->thread, NULL);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
if (mgr->stats != NULL) {
isc_stats_detach(&mgr->stats);
}
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
isc_barrier_destroy(&mgr->resuming);
isc_barrier_destroy(&mgr->pausing);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_condition_destroy(&mgr->wkstatecond);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
isc_condition_destroy(&mgr->wkpausecond);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_mutex_destroy(&mgr->lock);
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_mem_put(mgr->mctx, mgr->workers,
mgr->nworkers * sizeof(isc__networker_t));
isc_mem_putanddetach(&mgr->mctx, mgr, sizeof(*mgr));
}
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
static void
enqueue_pause(isc__networker_t *worker) {
isc__netievent_pause_t *event =
isc__nm_get_netievent_pause(worker->mgr);
isc__nm_enqueue_ievent(worker, (isc__netievent_t *)event);
}
static void
isc__nm_async_pause(isc__networker_t *worker, isc__netievent_t *ev0) {
UNUSED(ev0);
REQUIRE(worker->paused == false);
worker->paused = true;
uv_stop(&worker->loop);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_pause(isc_nm_t *mgr) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NM(mgr));
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
REQUIRE(!atomic_load(&mgr->paused));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_acquire_interlocked_force(mgr);
ensure interlocked netmgr events run on worker[0] Network manager events that require interlock (pause, resume, listen) are now always executed in the same worker thread, mgr->workers[0], to prevent races. "stoplistening" events no longer require interlock.
2021-05-06 16:11:43 +02:00
if (isc__nm_in_netthread()) {
REQUIRE(isc_nm_tid() == 0);
}
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
for (int i = 0; i < mgr->nworkers; i++) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc__networker_t *worker = &mgr->workers[i];
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
if (i == isc_nm_tid()) {
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
isc__nm_async_pause(worker, NULL);
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
} else {
enqueue_pause(worker);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
if (isc__nm_in_netthread()) {
ensure interlocked netmgr events run on worker[0] Network manager events that require interlock (pause, resume, listen) are now always executed in the same worker thread, mgr->workers[0], to prevent races. "stoplistening" events no longer require interlock.
2021-05-06 16:11:43 +02:00
atomic_fetch_add(&mgr->workers_paused, 1);
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
isc_barrier_wait(&mgr->pausing);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
LOCK(&mgr->lock);
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
while (atomic_load(&mgr->workers_paused) != mgr->workers_running) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
WAIT(&mgr->wkstatecond, &mgr->lock);
}
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
UNLOCK(&mgr->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
REQUIRE(atomic_compare_exchange_strong(&mgr->paused, &(bool){ false },
true));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
static void
enqueue_resume(isc__networker_t *worker) {
isc__netievent_resume_t *event =
isc__nm_get_netievent_resume(worker->mgr);
isc__nm_enqueue_ievent(worker, (isc__netievent_t *)event);
}
static void
isc__nm_async_resume(isc__networker_t *worker, isc__netievent_t *ev0) {
UNUSED(ev0);
REQUIRE(worker->paused == true);
worker->paused = false;
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_resume(isc_nm_t *mgr) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NM(mgr));
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
REQUIRE(atomic_load(&mgr->paused));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
ensure interlocked netmgr events run on worker[0] Network manager events that require interlock (pause, resume, listen) are now always executed in the same worker thread, mgr->workers[0], to prevent races. "stoplistening" events no longer require interlock.
2021-05-06 16:11:43 +02:00
if (isc__nm_in_netthread()) {
REQUIRE(isc_nm_tid() == 0);
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
drain_queue(&mgr->workers[isc_nm_tid()], NETIEVENT_PRIORITY);
ensure interlocked netmgr events run on worker[0] Network manager events that require interlock (pause, resume, listen) are now always executed in the same worker thread, mgr->workers[0], to prevent races. "stoplistening" events no longer require interlock.
2021-05-06 16:11:43 +02:00
}
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
for (int i = 0; i < mgr->nworkers; i++) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc__networker_t *worker = &mgr->workers[i];
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
if (i == isc_nm_tid()) {
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
isc__nm_async_resume(worker, NULL);
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
} else {
enqueue_resume(worker);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
if (isc__nm_in_netthread()) {
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
atomic_fetch_sub(&mgr->workers_paused, 1);
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
isc_barrier_wait(&mgr->resuming);
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
LOCK(&mgr->lock);
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
while (atomic_load(&mgr->workers_paused) != 0) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
WAIT(&mgr->wkstatecond, &mgr->lock);
}
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
UNLOCK(&mgr->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
REQUIRE(atomic_compare_exchange_strong(&mgr->paused, &(bool){ true },
false));
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc__nm_drop_interlocked(mgr);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_attach(isc_nm_t *mgr, isc_nm_t **dst) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NM(mgr));
REQUIRE(dst != NULL && *dst == NULL);
Use isc_refcount_increment0() when reusing handle or socket; remove extra DbC checks
2020-01-14 09:43:37 +01:00
isc_refcount_increment(&mgr->references);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*dst = mgr;
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_detach(isc_nm_t **mgr0) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nm_t *mgr = NULL;
REQUIRE(mgr0 != NULL);
REQUIRE(VALID_NM(*mgr0));
mgr = *mgr0;
*mgr0 = NULL;
Use isc_refcount_increment0() when reusing handle or socket; remove extra DbC checks
2020-01-14 09:43:37 +01:00
if (isc_refcount_decrement(&mgr->references) == 1) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
nm_destroy(&mgr);
}
}
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
void
Destroy netmgr before destroying taskmgr With taskmgr running on top of netmgr, the ordering of how the tasks and netmgr shutdown interacts was wrong as previously isc_taskmgr_destroy() was waiting until all tasks were properly shutdown and detached. This responsibility was moved to netmgr, so we now need to do the following: 1. shutdown all the tasks - this schedules all shutdown events onto the netmgr queue 2. shutdown the netmgr - this also makes sure all the tasks and events are properly executed 3. Shutdown the taskmgr - this now waits for all the tasks to finish running before returning 4. Shutdown the netmgr - this call waits for all the netmgr netievents to finish before returning This solves the race when the taskmgr object would be destroyed before all the tasks were finished running in the netmgr loops.
2021-04-27 10:28:40 +02:00
isc__netmgr_shutdown(isc_nm_t *mgr) {
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
REQUIRE(VALID_NM(mgr));
atomic_store(&mgr->closing, true);
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
for (int i = 0; i < mgr->nworkers; i++) {
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
isc__netievent_t *event = NULL;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
event = isc__nm_get_netievent_shutdown(mgr);
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
isc__nm_enqueue_ievent(&mgr->workers[i], event);
}
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
isc__netmgr_destroy(isc_nm_t **netmgrp) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nm_t *mgr = NULL;
limit the time we wait for netmgr to be destroyed if more than 10 seconds pass while we wait for netmgr events to finish running on shutdown, something is almost certainly wrong and we should assert and crash.
2020-07-15 17:57:58 -07:00
int counter = 0;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
REQUIRE(VALID_NM(*netmgrp));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
mgr = *netmgrp;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
/*
* Close active connections.
*/
Destroy netmgr before destroying taskmgr With taskmgr running on top of netmgr, the ordering of how the tasks and netmgr shutdown interacts was wrong as previously isc_taskmgr_destroy() was waiting until all tasks were properly shutdown and detached. This responsibility was moved to netmgr, so we now need to do the following: 1. shutdown all the tasks - this schedules all shutdown events onto the netmgr queue 2. shutdown the netmgr - this also makes sure all the tasks and events are properly executed 3. Shutdown the taskmgr - this now waits for all the tasks to finish running before returning 4. Shutdown the netmgr - this call waits for all the netmgr netievents to finish before returning This solves the race when the taskmgr object would be destroyed before all the tasks were finished running in the netmgr loops.
2021-04-27 10:28:40 +02:00
isc__netmgr_shutdown(mgr);
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
* Wait for the manager to be dereferenced elsewhere.
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*/
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
while (isc_refcount_current(&mgr->references) > 1 && counter++ < 1000) {
Use uv_sleep in the netmgr code libuv added uv_sleep(unsigned int msec) to the API since 1.34.0. Use that in the netmgr code and define usleep based shim for libuv << 1.34.0.
2021-05-03 12:10:03 +02:00
uv_sleep(10);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
#ifdef NETMGR_TRACE
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
if (isc_refcount_current(&mgr->references) > 1) {
isc__nm_dump_active(mgr);
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
}
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
#endif
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
/*
* Now just patiently wait
*/
while (isc_refcount_current(&mgr->references) > 1) {
Use uv_sleep in the netmgr code libuv added uv_sleep(unsigned int msec) to the API since 1.34.0. Use that in the netmgr code and define usleep based shim for libuv << 1.34.0.
2021-05-03 12:10:03 +02:00
uv_sleep(10);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
}
limit the time we wait for netmgr to be destroyed if more than 10 seconds pass while we wait for netmgr events to finish running on shutdown, something is almost certainly wrong and we should assert and crash.
2020-07-15 17:57:58 -07:00
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
/*
* Detach final reference.
*/
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
isc_nm_detach(netmgrp);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_maxudp(isc_nm_t *mgr, uint32_t maxudp) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NM(mgr));
atomic_store(&mgr->maxudp, maxudp);
}
Add isc_nmhandle_setwritetimeout() function In some situations (unit test and forthcoming XFR timeouts MR), we need to modify the write timeout independently of the read timeout. Add a isc_nmhandle_setwritetimeout() function that could be called before isc_nm_send() to specify a custom write timeout interval.
2022-02-09 19:48:13 +01:00
void
isc_nmhandle_setwritetimeout(isc_nmhandle_t *handle, uint64_t write_timeout) {
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
handle->sock->write_timeout = write_timeout;
}
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_nm_settimeouts(isc_nm_t *mgr, uint32_t init, uint32_t idle,
uint32_t keepalive, uint32_t advertised) {
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
REQUIRE(VALID_NM(mgr));
Change the isc_nm_(get|set)timeouts() to work with milliseconds The RFC7828 specifies the keepalive interval to be 16-bit, specified in units of 100 milliseconds and the configuration options tcp-*-timeouts are following the suit. The units of 100 milliseconds are very unintuitive and while we can't change the configuration and presentation format, we should not follow this weird unit in the API. This commit changes the isc_nm_(get|set)timeouts() functions to work with milliseconds and convert the values to milliseconds before passing them to the function, not just internally.
2021-03-18 11:16:45 +01:00
atomic_store(&mgr->init, init);
atomic_store(&mgr->idle, idle);
atomic_store(&mgr->keepalive, keepalive);
atomic_store(&mgr->advertised, advertised);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
}
Add configuration option to set send/recv buffers on the nm sockets This commit adds a new configuration option to set the receive and send buffer sizes on the TCP and UDP netmgr sockets. The default is `0` which doesn't set any value and just uses the value set by the operating system. There's no magic value here - set it too small and the performance will drop, set it too large, the buffers can fill-up with queries that have already timeouted on the client side and nobody is interested for the answer and this would just make the server clog up even more by making it produce useless work. The `netstat -su` can be used on POSIX systems to monitor the receive and send buffer errors.
2020-12-02 20:51:38 +01:00
void
isc_nm_setnetbuffers(isc_nm_t *mgr, int32_t recv_tcp, int32_t send_tcp,
int32_t recv_udp, int32_t send_udp) {
REQUIRE(VALID_NM(mgr));
atomic_store(&mgr->recv_tcp_buffer_size, recv_tcp);
atomic_store(&mgr->send_tcp_buffer_size, send_tcp);
atomic_store(&mgr->recv_udp_buffer_size, recv_udp);
atomic_store(&mgr->send_udp_buffer_size, send_udp);
}
Enable the load-balance-sockets configuration Previously, HAVE_SO_REUSEPORT_LB has been defined only in the private netmgr-int.h header file, making the configuration of load balanced sockets inoperable. Move the missing HAVE_SO_REUSEPORT_LB define the isc/netmgr.h and add missing isc_nm_getloadbalancesockets() implementation.
2022-04-05 01:20:13 +02:00
bool
isc_nm_getloadbalancesockets(isc_nm_t *mgr) {
REQUIRE(VALID_NM(mgr));
return (mgr->load_balance_sockets);
}
Add option to configure load balance sockets Previously, the option to enable kernel load balancing of the sockets was always enabled when supported by the operating system (SO_REUSEPORT on Linux and SO_REUSEPORT_LB on FreeBSD). It was reported that in scenarios where the networking threads are also responsible for processing long-running tasks (like RPZ processing, CATZ processing or large zone transfers), this could lead to intermitten brownouts for some clients, because the thread assigned by the operating system might be busy. In such scenarious, the overall performance would be better served by threads competing over the sockets because the idle threads can pick up the incoming traffic. Add new configuration option (`load-balance-sockets`) to allow enabling or disabling the load balancing of the sockets.
2022-04-01 14:43:14 +02:00
void
isc_nm_setloadbalancesockets(isc_nm_t *mgr, bool enabled) {
REQUIRE(VALID_NM(mgr));
#if HAVE_SO_REUSEPORT_LB
mgr->load_balance_sockets = enabled;
#else
UNUSED(enabled);
#endif
}
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_nm_gettimeouts(isc_nm_t *mgr, uint32_t *initial, uint32_t *idle,
uint32_t *keepalive, uint32_t *advertised) {
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
REQUIRE(VALID_NM(mgr));
if (initial != NULL) {
Change the isc_nm_(get|set)timeouts() to work with milliseconds The RFC7828 specifies the keepalive interval to be 16-bit, specified in units of 100 milliseconds and the configuration options tcp-*-timeouts are following the suit. The units of 100 milliseconds are very unintuitive and while we can't change the configuration and presentation format, we should not follow this weird unit in the API. This commit changes the isc_nm_(get|set)timeouts() functions to work with milliseconds and convert the values to milliseconds before passing them to the function, not just internally.
2021-03-18 11:16:45 +01:00
*initial = atomic_load(&mgr->init);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
}
if (idle != NULL) {
Change the isc_nm_(get|set)timeouts() to work with milliseconds The RFC7828 specifies the keepalive interval to be 16-bit, specified in units of 100 milliseconds and the configuration options tcp-*-timeouts are following the suit. The units of 100 milliseconds are very unintuitive and while we can't change the configuration and presentation format, we should not follow this weird unit in the API. This commit changes the isc_nm_(get|set)timeouts() functions to work with milliseconds and convert the values to milliseconds before passing them to the function, not just internally.
2021-03-18 11:16:45 +01:00
*idle = atomic_load(&mgr->idle);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
}
if (keepalive != NULL) {
Change the isc_nm_(get|set)timeouts() to work with milliseconds The RFC7828 specifies the keepalive interval to be 16-bit, specified in units of 100 milliseconds and the configuration options tcp-*-timeouts are following the suit. The units of 100 milliseconds are very unintuitive and while we can't change the configuration and presentation format, we should not follow this weird unit in the API. This commit changes the isc_nm_(get|set)timeouts() functions to work with milliseconds and convert the values to milliseconds before passing them to the function, not just internally.
2021-03-18 11:16:45 +01:00
*keepalive = atomic_load(&mgr->keepalive);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
}
if (advertised != NULL) {
Change the isc_nm_(get|set)timeouts() to work with milliseconds The RFC7828 specifies the keepalive interval to be 16-bit, specified in units of 100 milliseconds and the configuration options tcp-*-timeouts are following the suit. The units of 100 milliseconds are very unintuitive and while we can't change the configuration and presentation format, we should not follow this weird unit in the API. This commit changes the isc_nm_(get|set)timeouts() functions to work with milliseconds and convert the values to milliseconds before passing them to the function, not just internally.
2021-03-18 11:16:45 +01:00
*advertised = atomic_load(&mgr->advertised);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
}
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
* nm_thread is a single worker thread, that runs uv_run event loop
* until asked to stop.
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
*
* There are four queues for asynchronous events:
*
* 1. priority queue - netievents on the priority queue are run even when
* the taskmgr enters exclusive mode and the netmgr is paused. This
* is needed to properly start listening on the interfaces, free
* resources on shutdown, or resume from a pause.
*
Remove task privileged mode Previously, the task privileged mode has been used only when the named was starting up and loading the zones from the disk as the "first" thing to do. The privileged task was setup with quantum == 2, which made the taskmgr/netmgr spin around the privileged queue processing two events at the time. The same effect can be achieved by setting the quantum to UINT_MAX (e.g. practically unlimited) for the loadzone task, hence the privileged task mode was removed in favor of just processing all the events on the loadzone task in a single task_run().
2022-03-31 09:08:47 +02:00
* 2. task queue - only (traditional) tasks are scheduled here, and this queue
* is processed when the netmgr workers are finishing. This is needed to
* process the task shutdown events.
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
*
Remove task privileged mode Previously, the task privileged mode has been used only when the named was starting up and loading the zones from the disk as the "first" thing to do. The privileged task was setup with quantum == 2, which made the taskmgr/netmgr spin around the privileged queue processing two events at the time. The same effect can be achieved by setting the quantum to UINT_MAX (e.g. practically unlimited) for the loadzone task, hence the privileged task mode was removed in favor of just processing all the events on the loadzone task in a single task_run().
2022-03-31 09:08:47 +02:00
* 3. normal queue - this is the queue with netmgr events, e.g. reading,
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
* sending, callbacks, etc.
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*/
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
Use isc_threadresult_t instead of pthread specific void * return type The ISC thread API already defines isc_threadresult_t type, but we are using a pthread specific return type (void *).
2019-12-04 10:41:40 +01:00
static isc_threadresult_t
apply the modified style
2020-02-13 14:44:37 -08:00
nm_thread(isc_threadarg_t worker0) {
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
isc__networker_t *worker = (isc__networker_t *)worker0;
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc_nm_t *mgr = worker->mgr;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_tid_v = worker->id;
while (true) {
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
/*
* uv_run() runs async_cb() in a loop, which processes
* all four event queues until a "pause" or "stop" event
Remove task privileged mode Previously, the task privileged mode has been used only when the named was starting up and loading the zones from the disk as the "first" thing to do. The privileged task was setup with quantum == 2, which made the taskmgr/netmgr spin around the privileged queue processing two events at the time. The same effect can be achieved by setting the quantum to UINT_MAX (e.g. practically unlimited) for the loadzone task, hence the privileged task mode was removed in favor of just processing all the events on the loadzone task in a single task_run().
2022-03-31 09:08:47 +02:00
* is encountered. On pause, we process only priority
* events until resuming.
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
*/
apply the modified style
2020-02-13 14:44:37 -08:00
int r = uv_run(&worker->loop, UV_RUN_DEFAULT);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
INSIST(r > 0 || worker->finished);
if (worker->paused) {
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
INSIST(atomic_load(&mgr->interlocked) != isc_nm_tid());
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
atomic_fetch_add(&mgr->workers_paused, 1);
if (isc_barrier_wait(&mgr->pausing) != 0) {
LOCK(&mgr->lock);
SIGNAL(&mgr->wkstatecond);
UNLOCK(&mgr->lock);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
while (worker->paused) {
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
wait_for_priority_queue(worker);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
atomic_fetch_sub(&mgr->workers_paused, 1);
if (isc_barrier_wait(&mgr->resuming) != 0) {
LOCK(&mgr->lock);
SIGNAL(&mgr->wkstatecond);
UNLOCK(&mgr->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
if (r == 0) {
INSIST(worker->finished);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
break;
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
INSIST(!worker->finished);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
/*
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
* We are shutting down. Drain the queues.
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
*/
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
drain_queue(worker, NETIEVENT_TASK);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
for (size_t type = 0; type < NETIEVENT_MAX; type++) {
LOCK(&worker->ievents[type].lock);
INSIST(ISC_LIST_EMPTY(worker->ievents[type].list));
UNLOCK(&worker->ievents[type].lock);
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
LOCK(&mgr->lock);
mgr->workers_running--;
SIGNAL(&mgr->wkstatecond);
UNLOCK(&mgr->lock);
Use isc_threadresult_t instead of pthread specific void * return type The ISC thread API already defines isc_threadresult_t type, but we are using a pthread specific return type (void *).
2019-12-04 10:41:40 +01:00
return ((isc_threadresult_t)0);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Make the netmgr queue processing quantized There was a theoretical possibility of clogging up the queue processing with an endless loop where currently processing netievent would schedule new netievent that would get processed immediately. This wasn't such a problem when only netmgr netievents were processed, but with the addition of the tasks, there are at least two situation where this could happen: 1. In lib/dns/zone.c:setnsec3param() the task would get re-enqueued when the zone was not yet fully loaded. 2. Tasks have internal quantum for maximum number of isc_events to be processed, when the task quantum is reached, the task would get rescheduled and then immediately processed by the netmgr queue processing. As the isc_queue doesn't have a mechanism to atomically move the queue, this commit adds a mechanism to quantize the queue, so enqueueing new netievents will never stop processing other uv_loop_t events. The default quantum size is 128. Since the queue used in the network manager allows items to be enqueued more than once, tasks are now reference-counted around task_ready() and task_run(). task_ready() now has a public API wrapper, isc_task_ready(), that the netmgr can use to reschedule processing of a task if the quantum has been reached. Incidental changes: Cleaned up some unused fields left in isc_task_t and isc_taskmgr_t after the last refactoring, and changed atomic flags to atomic_bools for easier manipulation.
2021-04-27 12:03:34 +02:00
static bool
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
process_all_queues(isc__networker_t *worker) {
bool reschedule = false;
Make the netmgr queue processing quantized There was a theoretical possibility of clogging up the queue processing with an endless loop where currently processing netievent would schedule new netievent that would get processed immediately. This wasn't such a problem when only netmgr netievents were processed, but with the addition of the tasks, there are at least two situation where this could happen: 1. In lib/dns/zone.c:setnsec3param() the task would get re-enqueued when the zone was not yet fully loaded. 2. Tasks have internal quantum for maximum number of isc_events to be processed, when the task quantum is reached, the task would get rescheduled and then immediately processed by the netmgr queue processing. As the isc_queue doesn't have a mechanism to atomically move the queue, this commit adds a mechanism to quantize the queue, so enqueueing new netievents will never stop processing other uv_loop_t events. The default quantum size is 128. Since the queue used in the network manager allows items to be enqueued more than once, tasks are now reference-counted around task_ready() and task_run(). task_ready() now has a public API wrapper, isc_task_ready(), that the netmgr can use to reschedule processing of a task if the quantum has been reached. Incidental changes: Cleaned up some unused fields left in isc_task_t and isc_taskmgr_t after the last refactoring, and changed atomic flags to atomic_bools for easier manipulation.
2021-04-27 12:03:34 +02:00
/*
* The queue processing functions will return false when the
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
* system is pausing or stopping and we don't want to process
* the other queues in such case, but we need the async event
* to be rescheduled in the next uv_run().
Make the netmgr queue processing quantized There was a theoretical possibility of clogging up the queue processing with an endless loop where currently processing netievent would schedule new netievent that would get processed immediately. This wasn't such a problem when only netmgr netievents were processed, but with the addition of the tasks, there are at least two situation where this could happen: 1. In lib/dns/zone.c:setnsec3param() the task would get re-enqueued when the zone was not yet fully loaded. 2. Tasks have internal quantum for maximum number of isc_events to be processed, when the task quantum is reached, the task would get rescheduled and then immediately processed by the netmgr queue processing. As the isc_queue doesn't have a mechanism to atomically move the queue, this commit adds a mechanism to quantize the queue, so enqueueing new netievents will never stop processing other uv_loop_t events. The default quantum size is 128. Since the queue used in the network manager allows items to be enqueued more than once, tasks are now reference-counted around task_ready() and task_run(). task_ready() now has a public API wrapper, isc_task_ready(), that the netmgr can use to reschedule processing of a task if the quantum has been reached. Incidental changes: Cleaned up some unused fields left in isc_task_t and isc_taskmgr_t after the last refactoring, and changed atomic flags to atomic_bools for easier manipulation.
2021-04-27 12:03:34 +02:00
*/
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
for (size_t type = 0; type < NETIEVENT_MAX; type++) {
isc_result_t result = process_queue(worker, type);
switch (result) {
case ISC_R_SUSPEND:
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
reschedule = true;
break;
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
case ISC_R_EMPTY:
/* empty queue */
break;
case ISC_R_SUCCESS:
reschedule = true;
break;
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
}
}
return (reschedule);
Make the netmgr queue processing quantized There was a theoretical possibility of clogging up the queue processing with an endless loop where currently processing netievent would schedule new netievent that would get processed immediately. This wasn't such a problem when only netmgr netievents were processed, but with the addition of the tasks, there are at least two situation where this could happen: 1. In lib/dns/zone.c:setnsec3param() the task would get re-enqueued when the zone was not yet fully loaded. 2. Tasks have internal quantum for maximum number of isc_events to be processed, when the task quantum is reached, the task would get rescheduled and then immediately processed by the netmgr queue processing. As the isc_queue doesn't have a mechanism to atomically move the queue, this commit adds a mechanism to quantize the queue, so enqueueing new netievents will never stop processing other uv_loop_t events. The default quantum size is 128. Since the queue used in the network manager allows items to be enqueued more than once, tasks are now reference-counted around task_ready() and task_run(). task_ready() now has a public API wrapper, isc_task_ready(), that the netmgr can use to reschedule processing of a task if the quantum has been reached. Incidental changes: Cleaned up some unused fields left in isc_task_t and isc_taskmgr_t after the last refactoring, and changed atomic flags to atomic_bools for easier manipulation.
2021-04-27 12:03:34 +02:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
* async_cb() is a universal callback for 'async' events sent to event loop.
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
* It's the only way to safely pass data to the libuv event loop. We use a
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
* single async event and a set of lockless queues of 'isc__netievent_t'
* structures passed from other threads.
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*/
static void
apply the modified style
2020-02-13 14:44:37 -08:00
async_cb(uv_async_t *handle) {
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
isc__networker_t *worker = (isc__networker_t *)handle->loop->data;
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
if (process_all_queues(worker)) {
Add asynchronous work API to the network manager The libuv has a support for running long running tasks in the dedicated threadpools, so it doesn't affect networking IO. This commit adds isc_nm_work_enqueue() wrapper that would wraps around the libuv API and runs it on top of associated worker loop. The only limitation is that the function must be called from inside network manager thread, so the call to the function should be wrapped inside a (bound) task.
2021-05-27 09:45:07 +02:00
/*
* If we didn't process all the events, we need to enqueue
Make the netmgr queue processing quantized There was a theoretical possibility of clogging up the queue processing with an endless loop where currently processing netievent would schedule new netievent that would get processed immediately. This wasn't such a problem when only netmgr netievents were processed, but with the addition of the tasks, there are at least two situation where this could happen: 1. In lib/dns/zone.c:setnsec3param() the task would get re-enqueued when the zone was not yet fully loaded. 2. Tasks have internal quantum for maximum number of isc_events to be processed, when the task quantum is reached, the task would get rescheduled and then immediately processed by the netmgr queue processing. As the isc_queue doesn't have a mechanism to atomically move the queue, this commit adds a mechanism to quantize the queue, so enqueueing new netievents will never stop processing other uv_loop_t events. The default quantum size is 128. Since the queue used in the network manager allows items to be enqueued more than once, tasks are now reference-counted around task_ready() and task_run(). task_ready() now has a public API wrapper, isc_task_ready(), that the netmgr can use to reschedule processing of a task if the quantum has been reached. Incidental changes: Cleaned up some unused fields left in isc_task_t and isc_taskmgr_t after the last refactoring, and changed atomic flags to atomic_bools for easier manipulation.
2021-04-27 12:03:34 +02:00
* async_cb to be run in the next iteration of the uv_loop
*/
uv_async_send(handle);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
}
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_async_stop(isc__networker_t *worker, isc__netievent_t *ev0) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
UNUSED(ev0);
worker->finished = true;
/* Close the async handler */
uv_close((uv_handle_t *)&worker->async, NULL);
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
}
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
void
Make netmgr the authority on number of threads running Instead of passing the "workers" variable back and forth along with passing the single isc_nm_t instance, add isc_nm_getnworkers() function that returns the number of netmgr threads are running. Change the ns_interfacemgr and ns_taskmgr to utilize the newly acquired knowledge.
2022-03-18 09:50:58 +01:00
isc_nm_task_enqueue(isc_nm_t *nm, isc_task_t *task, int tid) {
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
isc__netievent_t *event = NULL;
isc__networker_t *worker = NULL;
Make netmgr the authority on number of threads running Instead of passing the "workers" variable back and forth along with passing the single isc_nm_t instance, add isc_nm_getnworkers() function that returns the number of netmgr threads are running. Change the ns_interfacemgr and ns_taskmgr to utilize the newly acquired knowledge.
2022-03-18 09:50:58 +01:00
REQUIRE(tid >= 0 && tid < nm->nworkers);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
worker = &nm->workers[tid];
Remove task privileged mode Previously, the task privileged mode has been used only when the named was starting up and loading the zones from the disk as the "first" thing to do. The privileged task was setup with quantum == 2, which made the taskmgr/netmgr spin around the privileged queue processing two events at the time. The same effect can be achieved by setting the quantum to UINT_MAX (e.g. practically unlimited) for the loadzone task, hence the privileged task mode was removed in favor of just processing all the events on the loadzone task in a single task_run().
2022-03-31 09:08:47 +02:00
event = (isc__netievent_t *)isc__nm_get_netievent_task(nm, task);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
isc__nm_enqueue_ievent(worker, event);
}
static void
isc__nm_async_task(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_task_t *ievent = (isc__netievent_task_t *)ev0;
isc_result_t result;
UNUSED(worker);
result = isc_task_run(ievent->task);
switch (result) {
case ISC_R_QUOTA:
Make the netmgr queue processing quantized There was a theoretical possibility of clogging up the queue processing with an endless loop where currently processing netievent would schedule new netievent that would get processed immediately. This wasn't such a problem when only netmgr netievents were processed, but with the addition of the tasks, there are at least two situation where this could happen: 1. In lib/dns/zone.c:setnsec3param() the task would get re-enqueued when the zone was not yet fully loaded. 2. Tasks have internal quantum for maximum number of isc_events to be processed, when the task quantum is reached, the task would get rescheduled and then immediately processed by the netmgr queue processing. As the isc_queue doesn't have a mechanism to atomically move the queue, this commit adds a mechanism to quantize the queue, so enqueueing new netievents will never stop processing other uv_loop_t events. The default quantum size is 128. Since the queue used in the network manager allows items to be enqueued more than once, tasks are now reference-counted around task_ready() and task_run(). task_ready() now has a public API wrapper, isc_task_ready(), that the netmgr can use to reschedule processing of a task if the quantum has been reached. Incidental changes: Cleaned up some unused fields left in isc_task_t and isc_taskmgr_t after the last refactoring, and changed atomic flags to atomic_bools for easier manipulation.
2021-04-27 12:03:34 +02:00
isc_task_ready(ievent->task);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
return;
case ISC_R_SUCCESS:
return;
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
}
}
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
static void
wait_for_priority_queue(isc__networker_t *worker) {
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
isc_condition_t *cond = &worker->ievents[NETIEVENT_PRIORITY].cond;
isc_mutex_t *lock = &worker->ievents[NETIEVENT_PRIORITY].lock;
isc__netievent_list_t *list =
&(worker->ievents[NETIEVENT_PRIORITY].list);
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
LOCK(lock);
while (ISC_LIST_EMPTY(*list)) {
WAIT(cond, lock);
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
}
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
UNLOCK(lock);
drain_queue(worker, NETIEVENT_PRIORITY);
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
}
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
static void
drain_queue(isc__networker_t *worker, netievent_type_t type) {
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
bool empty = false;
while (!empty) {
if (process_queue(worker, type) == ISC_R_EMPTY) {
LOCK(&worker->ievents[type].lock);
empty = ISC_LIST_EMPTY(worker->ievents[type].list);
UNLOCK(&worker->ievents[type].lock);
}
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
/*
* The two macros here generate the individual cases for the process_netievent()
* function. The NETIEVENT_CASE(type) macro is the common case, and
* NETIEVENT_CASE_NOMORE(type) is a macro that causes the loop in the
* process_queue() to stop, e.g. it's only used for the netievent that
* stops/pauses processing the enqueued netievents.
*/
#define NETIEVENT_CASE(type) \
case netievent_##type: { \
isc__nm_async_##type(worker, ievent); \
isc__nm_put_netievent_##type( \
worker->mgr, (isc__netievent_##type##_t *)ievent); \
return (true); \
}
#define NETIEVENT_CASE_NOMORE(type) \
case netievent_##type: { \
isc__nm_async_##type(worker, ievent); \
isc__nm_put_netievent_##type(worker->mgr, ievent); \
return (false); \
}
static bool
process_netievent(isc__networker_t *worker, isc__netievent_t *ievent) {
REQUIRE(worker->id == isc_nm_tid());
switch (ievent->type) {
/* Don't process more ievents when we are stopping */
NETIEVENT_CASE_NOMORE(stop);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
NETIEVENT_CASE(task);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_CASE(udpconnect);
NETIEVENT_CASE(udplisten);
NETIEVENT_CASE(udpstop);
NETIEVENT_CASE(udpsend);
NETIEVENT_CASE(udpread);
NETIEVENT_CASE(udpcancel);
NETIEVENT_CASE(udpclose);
netmgr: add isc_nm_routeconnect() isc_nm_routeconnect() opens a route/netlink socket, then calls a connect callback, much like isc_nm_udpconnect(), with a handle that can then be monitored for network changes. Internally the socket is treated as a UDP socket, since route/netlink sockets follow the datagram contract.
2021-10-02 14:52:46 -07:00
NETIEVENT_CASE(routeconnect);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_CASE(tcpaccept);
NETIEVENT_CASE(tcpconnect);
NETIEVENT_CASE(tcplisten);
NETIEVENT_CASE(tcpstartread);
NETIEVENT_CASE(tcppauseread);
NETIEVENT_CASE(tcpsend);
NETIEVENT_CASE(tcpstop);
NETIEVENT_CASE(tcpcancel);
NETIEVENT_CASE(tcpclose);
NETIEVENT_CASE(tcpdnsaccept);
NETIEVENT_CASE(tcpdnslisten);
NETIEVENT_CASE(tcpdnsconnect);
NETIEVENT_CASE(tcpdnssend);
NETIEVENT_CASE(tcpdnscancel);
NETIEVENT_CASE(tcpdnsclose);
NETIEVENT_CASE(tcpdnsread);
NETIEVENT_CASE(tcpdnsstop);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
NETIEVENT_CASE(tlsdnscycle);
NETIEVENT_CASE(tlsdnsaccept);
NETIEVENT_CASE(tlsdnslisten);
NETIEVENT_CASE(tlsdnsconnect);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_CASE(tlsdnssend);
NETIEVENT_CASE(tlsdnscancel);
NETIEVENT_CASE(tlsdnsclose);
NETIEVENT_CASE(tlsdnsread);
NETIEVENT_CASE(tlsdnsstop);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
NETIEVENT_CASE(tlsdnsshutdown);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
NETIEVENT_CASE(tlsstartread);
NETIEVENT_CASE(tlssend);
NETIEVENT_CASE(tlsclose);
NETIEVENT_CASE(tlsdobio);
NETIEVENT_CASE(tlscancel);
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
NETIEVENT_CASE(httpstop);
NETIEVENT_CASE(httpsend);
NETIEVENT_CASE(httpclose);
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
Add isc_nmsocket_set_tlsctx() This commit adds isc_nmsocket_set_tlsctx() - an asynchronous function that replaces the TLS context within a given TLS-enabled listener socket object. It is based on the newly added reference counting functionality. The intention of adding this function is to add functionality to replace a TLS context without recreating the whole socket object, including the underlying TCP listener socket, as a BIND process might not have enough permissions to re-create it fully on reconfiguration.
2022-03-31 13:47:48 +03:00
NETIEVENT_CASE(settlsctx);
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_CASE(connectcb);
NETIEVENT_CASE(readcb);
NETIEVENT_CASE(sendcb);
NETIEVENT_CASE(close);
NETIEVENT_CASE(detach);
NETIEVENT_CASE(shutdown);
NETIEVENT_CASE(resume);
NETIEVENT_CASE_NOMORE(pause);
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
}
return (true);
}
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
static isc_result_t
process_queue(isc__networker_t *worker, netievent_type_t type) {
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
isc__netievent_t *ievent = NULL;
isc__netievent_list_t list;
ISC_LIST_INIT(list);
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
LOCK(&worker->ievents[type].lock);
ISC_LIST_MOVE(list, worker->ievents[type].list);
UNLOCK(&worker->ievents[type].lock);
ievent = ISC_LIST_HEAD(list);
if (ievent == NULL) {
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
/* There's nothing scheduled */
return (ISC_R_EMPTY);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
while (ievent != NULL) {
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
isc__netievent_t *next = ISC_LIST_NEXT(ievent, link);
ISC_LIST_DEQUEUE(list, ievent, link);
if (!process_netievent(worker, ievent)) {
/* The netievent told us to stop */
if (!ISC_LIST_EMPTY(list)) {
/*
* Reschedule the rest of the unprocessed
* events.
*/
LOCK(&worker->ievents[type].lock);
ISC_LIST_PREPENDLIST(worker->ievents[type].list,
list, link);
UNLOCK(&worker->ievents[type].lock);
}
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
return (ISC_R_SUSPEND);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
}
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
ievent = next;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Make the netmgr queue processing quantized There was a theoretical possibility of clogging up the queue processing with an endless loop where currently processing netievent would schedule new netievent that would get processed immediately. This wasn't such a problem when only netmgr netievents were processed, but with the addition of the tasks, there are at least two situation where this could happen: 1. In lib/dns/zone.c:setnsec3param() the task would get re-enqueued when the zone was not yet fully loaded. 2. Tasks have internal quantum for maximum number of isc_events to be processed, when the task quantum is reached, the task would get rescheduled and then immediately processed by the netmgr queue processing. As the isc_queue doesn't have a mechanism to atomically move the queue, this commit adds a mechanism to quantize the queue, so enqueueing new netievents will never stop processing other uv_loop_t events. The default quantum size is 128. Since the queue used in the network manager allows items to be enqueued more than once, tasks are now reference-counted around task_ready() and task_run(). task_ready() now has a public API wrapper, isc_task_ready(), that the netmgr can use to reschedule processing of a task if the quantum has been reached. Incidental changes: Cleaned up some unused fields left in isc_task_t and isc_taskmgr_t after the last refactoring, and changed atomic flags to atomic_bools for easier manipulation.
2021-04-27 12:03:34 +02:00
Replace netmgr quantum with loop-preventing barrier Instead of using fixed quantum, this commit adds atomic counter for number of items on each queue and uses the number of netievents scheduled to run as the limit of maximum number of netievents for a single process_queue() run. This prevents the endless loops when the netievent would schedule more netievents onto the same loop, but we don't have to pick "magic" number for the quantum.
2021-05-07 12:58:40 +02:00
/* We processed at least one */
return (ISC_R_SUCCESS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void *
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_get_netievent(isc_nm_t *mgr, isc__netievent_type type) {
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
isc__netievent_storage_t *event = isc_mem_get(mgr->mctx,
sizeof(*event));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*event = (isc__netievent_storage_t){ .ni.type = type };
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
ISC_LINK_INIT(&(event->ni), link);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
return (event);
}
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_put_netievent(isc_nm_t *mgr, void *ievent) {
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
isc_mem_put(mgr->mctx, ievent, sizeof(isc__netievent_storage_t));
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_DEF(tcpclose);
NETIEVENT_SOCKET_DEF(tcplisten);
NETIEVENT_SOCKET_DEF(tcppauseread);
NETIEVENT_SOCKET_DEF(tcpstartread);
NETIEVENT_SOCKET_DEF(tcpstop);
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
NETIEVENT_SOCKET_DEF(tlsclose);
NETIEVENT_SOCKET_DEF(tlsconnect);
NETIEVENT_SOCKET_DEF(tlsdobio);
NETIEVENT_SOCKET_DEF(tlsstartread);
NETIEVENT_SOCKET_HANDLE_DEF(tlscancel);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_DEF(udpclose);
NETIEVENT_SOCKET_DEF(udplisten);
NETIEVENT_SOCKET_DEF(udpread);
NETIEVENT_SOCKET_DEF(udpsend);
NETIEVENT_SOCKET_DEF(udpstop);
NETIEVENT_SOCKET_DEF(tcpdnsclose);
NETIEVENT_SOCKET_DEF(tcpdnsread);
NETIEVENT_SOCKET_DEF(tcpdnsstop);
NETIEVENT_SOCKET_DEF(tcpdnslisten);
NETIEVENT_SOCKET_REQ_DEF(tcpdnsconnect);
NETIEVENT_SOCKET_REQ_DEF(tcpdnssend);
NETIEVENT_SOCKET_HANDLE_DEF(tcpdnscancel);
NETIEVENT_SOCKET_QUOTA_DEF(tcpdnsaccept);
NETIEVENT_SOCKET_DEF(tlsdnsclose);
NETIEVENT_SOCKET_DEF(tlsdnsread);
NETIEVENT_SOCKET_DEF(tlsdnsstop);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
NETIEVENT_SOCKET_DEF(tlsdnslisten);
NETIEVENT_SOCKET_REQ_DEF(tlsdnsconnect);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_REQ_DEF(tlsdnssend);
NETIEVENT_SOCKET_HANDLE_DEF(tlsdnscancel);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
NETIEVENT_SOCKET_QUOTA_DEF(tlsdnsaccept);
NETIEVENT_SOCKET_DEF(tlsdnscycle);
NETIEVENT_SOCKET_DEF(tlsdnsshutdown);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
NETIEVENT_SOCKET_DEF(httpstop);
NETIEVENT_SOCKET_REQ_DEF(httpsend);
NETIEVENT_SOCKET_DEF(httpclose);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_REQ_DEF(tcpconnect);
NETIEVENT_SOCKET_REQ_DEF(tcpsend);
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
NETIEVENT_SOCKET_REQ_DEF(tlssend);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_REQ_DEF(udpconnect);
netmgr: add isc_nm_routeconnect() isc_nm_routeconnect() opens a route/netlink socket, then calls a connect callback, much like isc_nm_udpconnect(), with a handle that can then be monitored for network changes. Internally the socket is treated as a UDP socket, since route/netlink sockets follow the datagram contract.
2021-10-02 14:52:46 -07:00
NETIEVENT_SOCKET_REQ_DEF(routeconnect);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_REQ_RESULT_DEF(connectcb);
NETIEVENT_SOCKET_REQ_RESULT_DEF(readcb);
NETIEVENT_SOCKET_REQ_RESULT_DEF(sendcb);
NETIEVENT_SOCKET_DEF(detach);
NETIEVENT_SOCKET_HANDLE_DEF(tcpcancel);
NETIEVENT_SOCKET_HANDLE_DEF(udpcancel);
NETIEVENT_SOCKET_QUOTA_DEF(tcpaccept);
NETIEVENT_SOCKET_DEF(close);
NETIEVENT_DEF(pause);
NETIEVENT_DEF(resume);
NETIEVENT_DEF(shutdown);
NETIEVENT_DEF(stop);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
NETIEVENT_TASK_DEF(task);
Add isc_nmsocket_set_tlsctx() This commit adds isc_nmsocket_set_tlsctx() - an asynchronous function that replaces the TLS context within a given TLS-enabled listener socket object. It is based on the newly added reference counting functionality. The intention of adding this function is to add functionality to replace a TLS context without recreating the whole socket object, including the underlying TCP listener socket, as a BIND process might not have enough permissions to re-create it fully on reconfiguration.
2022-03-31 13:47:48 +03:00
NETIEVENT_SOCKET_TLSCTX_DEF(settlsctx);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
void
isc__nm_maybe_enqueue_ievent(isc__networker_t *worker,
isc__netievent_t *event) {
/*
* If we are already in the matching nmthread, process the ievent
* directly.
*/
if (worker->id == isc_nm_tid()) {
process_netievent(worker, event);
return;
}
isc__nm_enqueue_ievent(worker, event);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_enqueue_ievent(isc__networker_t *worker, isc__netievent_t *event) {
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
netievent_type_t type;
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
if (event->type > netievent_prio) {
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
type = NETIEVENT_PRIORITY;
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
} else {
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
switch (event->type) {
case netievent_prio:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
break;
case netievent_task:
type = NETIEVENT_TASK;
break;
default:
type = NETIEVENT_NORMAL;
break;
}
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
}
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
/*
* We need to make sure this signal will be delivered and
* the queue will be processed.
*/
LOCK(&worker->ievents[type].lock);
ISC_LIST_ENQUEUE(worker->ievents[type].list, event, link);
if (type == NETIEVENT_PRIORITY) {
SIGNAL(&worker->ievents[type].cond);
}
UNLOCK(&worker->ievents[type].lock);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
uv_async_send(&worker->async);
}
netmgr: don't send to an inactive (closing) udp socket We had a race in which n UDP socket could have been already closing by libuv but we still sent data to it. Mark socket as not-active when stopping listening and verify that socket is not active when trying to send data to it.
2020-01-16 11:52:58 +01:00
bool
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nmsocket_active(isc_nmsocket_t *sock) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMSOCK(sock));
if (sock->parent != NULL) {
return (atomic_load(&sock->parent->active));
}
return (atomic_load(&sock->active));
}
Fix the data race when read-writing sock->active by using cmpxchg
2020-10-22 10:07:56 +02:00
bool
isc__nmsocket_deactivate(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
if (sock->parent != NULL) {
return (atomic_compare_exchange_strong(&sock->parent->active,
&(bool){ true }, false));
}
return (atomic_compare_exchange_strong(&sock->active, &(bool){ true },
false));
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_attach(isc_nmsocket_t *sock, isc_nmsocket_t **target FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(target != NULL && *target == NULL);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_nmsocket_t *rsock = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (sock->parent != NULL) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
rsock = sock->parent;
INSIST(rsock->parent == NULL); /* sanity check */
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
} else {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
rsock = sock;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc__nmsocket_attach():%p->references = %" PRIuFAST32
"\n",
rsock, isc_refcount_current(&rsock->references) + 1);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_refcount_increment0(&rsock->references);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*target = sock;
}
/*
* Free all resources inside a socket (including its children if any).
*/
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_cleanup(isc_nmsocket_t *sock, bool dofree FLARG) {
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_t *handle = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_uvreq_t *uvreq = NULL;
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(!isc__nmsocket_active(sock));
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("nmsocket_cleanup():%p->references = %" PRIuFAST32
"\n",
sock, isc_refcount_current(&sock->references));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
isc__nm_decstats(sock, STATID_ACTIVE);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
atomic_store(&sock->destroying, true);
if (sock->parent == NULL && sock->children != NULL) {
/*
* We shouldn't be here unless there are no active handles,
* so we can clean up and free the children.
*/
Use sock->nchildren instead of mgr->nworkers when initializing NM On Windows, we were limiting the number of listening children to just 1, but we were then iterating on mgr->nworkers. That lead to scheduling more async_*listen() than actually allocated and out-of-bound read-write operation on the heap.
2020-12-03 17:58:10 +01:00
for (size_t i = 0; i < sock->nchildren; i++) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (!atomic_load(&sock->children[i].destroying)) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_cleanup(&sock->children[i],
false FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
}
/*
Use barriers for netmgr synchronization The netmgr listening, stoplistening, pausing and resuming functions now use barriers for synchronization, which makes the code much simpler. isc/barrier.h defines isc_barrier macros as a front-end for uv_barrier on platforms where that works, and pthread_barrier where it doesn't (including TSAN builds).
2021-05-05 11:51:39 +02:00
* This was a parent socket: destroy the listening
* barriers that synchronized the children.
*/
isc_barrier_destroy(&sock->startlistening);
isc_barrier_destroy(&sock->stoplistening);
/*
* Now free them.
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*/
isc_mem_put(sock->mgr->mctx, sock->children,
sock->nchildren * sizeof(*sock));
sock->children = NULL;
sock->nchildren = 0;
}
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
sock->statichandle = NULL;
modify reference counting within netmgr - isc__nmhandle_get() now attaches to the sock in the nmhandle object. the caller is responsible for dereferencing the original socket pointer when necessary. - tcpdns listener sockets attach sock->outer to the outer tcp listener socket. tcpdns connected sockets attach sock->outerhandle to the handle for the tcp connected socket. - only listener sockets need to be attached/detached directly. connected sockets should only be accessed and reference-counted via their associated handles.
2020-06-04 23:13:54 -07:00
if (sock->outerhandle != NULL) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nmhandle_detach(&sock->outerhandle FLARG_PASS);
modify reference counting within netmgr - isc__nmhandle_get() now attaches to the sock in the nmhandle object. the caller is responsible for dereferencing the original socket pointer when necessary. - tcpdns listener sockets attach sock->outer to the outer tcp listener socket. tcpdns connected sockets attach sock->outerhandle to the handle for the tcp connected socket. - only listener sockets need to be attached/detached directly. connected sockets should only be accessed and reference-counted via their associated handles.
2020-06-04 23:13:54 -07:00
}
if (sock->outer != NULL) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_detach(&sock->outer FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
while ((handle = isc_astack_pop(sock->inactivehandles)) != NULL) {
nmhandle_free(sock, handle);
}
if (sock->buf != NULL) {
Replace isc_mem_allocate() usage with isc_mem_get() in netmgr.c The isc_mem_allocate() comes with additional cost because of the memory tracking. In this commit, we replace the usage with isc_mem_get() because we track the allocated sizes anyway, so it's possible to also replace isc_mem_free() with isc_mem_put().
2021-05-11 14:37:18 +02:00
isc_mem_put(sock->mgr->mctx, sock->buf, sock->buf_size);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
if (sock->quota != NULL) {
isc_quota_detach(&sock->quota);
}
netmgr: fix TCP backlog and client quota count - add support for TCP backlog, using the value provided by config. - don't attach to TCP client quota for listening sockets, only connected sockets.
2019-11-22 13:19:45 +01:00
sock->pquota = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_astack_destroy(sock->inactivehandles);
while ((uvreq = isc_astack_pop(sock->inactivereqs)) != NULL) {
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
isc_mem_put(sock->mgr->mctx, uvreq, sizeof(*uvreq));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
isc_astack_destroy(sock->inactivereqs);
Clear sock->magic to 0 when destroying a netmgr socket
2020-05-28 14:47:04 +02:00
sock->magic = 0;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_condition_destroy(&sock->scond);
Stop leaking mutex in nmworker and cond in nm socket On FreeBSD, the pthread primitives are not solely allocated on stack, but part of the object lives on the heap. Missing pthread_*_destroy causes the heap memory to grow and in case of fast lived object it's possible to run out-of-memory. Properly destroy the leaking mutex (worker->lock) and the leaking condition (sock->cond).
2021-12-06 11:10:17 +01:00
isc_condition_destroy(&sock->cond);
isc_mutex_destroy(&sock->lock);
Use isc_tlsctx_attach() in TLS DNS code This commit adds proper reference counting for TLS contexts into generic TLS DNS (DoT) code.
2022-03-16 17:02:46 +02:00
isc__nm_tlsdns_cleanup_data(sock);
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
isc__nm_tls_cleanup_data(sock);
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
isc__nm_http_cleanup_data(sock);
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
LOCK(&sock->mgr->lock);
ISC_LIST_UNLINK(sock->mgr->active_sockets, sock, active_link);
UNLOCK(&sock->mgr->lock);
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (dofree) {
isc_nm_t *mgr = sock->mgr;
isc_mem_put(mgr->mctx, sock, sizeof(*sock));
isc_nm_detach(&mgr);
} else {
isc_nm_detach(&sock->mgr);
}
}
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_maybe_destroy(isc_nmsocket_t *sock FLARG) {
apply the modified style
2020-02-13 14:44:37 -08:00
int active_handles;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
bool destroy = false;
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
NETMGR_TRACE_LOG("%s():%p->references = %" PRIuFAST32 "\n", __func__,
sock, isc_refcount_current(&sock->references));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (sock->parent != NULL) {
/*
* This is a child socket and cannot be destroyed except
* as a side effect of destroying the parent, so let's go
* see if the parent is ready to be destroyed.
*/
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_maybe_destroy(sock->parent FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
return;
}
/*
* This is a parent socket (or a standalone). See whether the
* children have active handles before deciding whether to
* accept destruction.
*/
LOCK(&sock->lock);
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
if (atomic_load(&sock->active) || atomic_load(&sock->destroying) ||
apply the modified style
2020-02-13 14:44:37 -08:00
!atomic_load(&sock->closed) || atomic_load(&sock->references) != 0)
{
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
UNLOCK(&sock->lock);
return;
}
active_handles = atomic_load(&sock->ah);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (sock->children != NULL) {
Use sock->nchildren instead of mgr->nworkers when initializing NM On Windows, we were limiting the number of listening children to just 1, but we were then iterating on mgr->nworkers. That lead to scheduling more async_*listen() than actually allocated and out-of-bound read-write operation on the heap.
2020-12-03 17:58:10 +01:00
for (size_t i = 0; i < sock->nchildren; i++) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
LOCK(&sock->children[i].lock);
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
active_handles += atomic_load(&sock->children[i].ah);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
UNLOCK(&sock->children[i].lock);
}
}
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
if (active_handles == 0 || sock->statichandle != NULL) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
destroy = true;
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETMGR_TRACE_LOG("%s:%p->active_handles = %d, .statichandle = %p\n",
__func__, sock, active_handles, sock->statichandle);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (destroy) {
Fix a shutdown race in netmgr udp We need to mark the socket as inactive early (and synchronously) in the stoplistening process; otherwise we might destroy the callback argument before we actually stop listening, and call the callback on bad memory.
2020-06-22 15:46:11 -07:00
atomic_store(&sock->destroying, true);
UNLOCK(&sock->lock);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_cleanup(sock, true FLARG_PASS);
Fix a shutdown race in netmgr udp We need to mark the socket as inactive early (and synchronously) in the stoplistening process; otherwise we might destroy the callback argument before we actually stop listening, and call the callback on bad memory.
2020-06-22 15:46:11 -07:00
} else {
UNLOCK(&sock->lock);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
}
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_prep_destroy(isc_nmsocket_t *sock FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(sock->parent == NULL);
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc___nmsocket_prep_destroy():%p->references = "
"%" PRIuFAST32 "\n",
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
sock, isc_refcount_current(&sock->references));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
* The final external reference to the socket is gone. We can try
* destroying the socket, but we have to wait for all the inflight
* handles to finish first.
*/
atomic_store(&sock->active, false);
/*
* If the socket has children, they'll need to be marked inactive
* so they can be cleaned up too.
*/
if (sock->children != NULL) {
Use sock->nchildren instead of mgr->nworkers when initializing NM On Windows, we were limiting the number of listening children to just 1, but we were then iterating on mgr->nworkers. That lead to scheduling more async_*listen() than actually allocated and out-of-bound read-write operation on the heap.
2020-12-03 17:58:10 +01:00
for (size_t i = 0; i < sock->nchildren; i++) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
atomic_store(&sock->children[i].active, false);
}
}
/*
* If we're here then we already stopped listening; otherwise
* we'd have a hanging reference from the listening process.
*
* If it's a regular socket we may need to close it.
*/
if (!atomic_load(&sock->closed)) {
switch (sock->type) {
add netmgr functions to support outgoing DNS queries - isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection. - isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a timeout argument to ensure connections time out and are correctly cleaned up on failure. - isc_nm_read() now supports UDP; it reads a single datagram and then stops until the next time it's called. - isc_nm_cancelread() now runs asynchronously to prevent assertion failure if reading is interrupted by a non-network thread (e.g. a timeout). - isc_nm_cancelread() can now apply to UDP sockets. - added shim code to support UDP connection in versions of libuv prior to 1.27, when uv_udp_connect() was added all these functions will be used to support outgoing queries in dig, xfrin, dispatch, etc.
2020-09-05 11:07:40 -07:00
case isc_nm_udpsocket:
isc__nm_udp_close(sock);
return;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
case isc_nm_tcpsocket:
isc__nm_tcp_close(sock);
Don't destroy a non-closed socket, wait for all the callbacks. We erroneously tried to destroy a socket after issuing isc__nm_tcp{,dns}_close. Under some (race) circumstances we could get nm_socket_cleanup to be called twice for the same socket, causing an access to a dead memory.
2020-07-01 00:49:12 -07:00
return;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
case isc_nm_tcpdnssocket:
isc__nm_tcpdns_close(sock);
Don't destroy a non-closed socket, wait for all the callbacks. We erroneously tried to destroy a socket after issuing isc__nm_tcp{,dns}_close. Under some (race) circumstances we could get nm_socket_cleanup to be called twice for the same socket, causing an access to a dead memory.
2020-07-01 00:49:12 -07:00
return;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnssocket:
isc__nm_tlsdns_close(sock);
return;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
case isc_nm_tlssocket:
isc__nm_tls_close(sock);
break;
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
case isc_nm_httpsocket:
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
isc__nm_http_close(sock);
return;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
default:
break;
}
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_maybe_destroy(sock FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_detach(isc_nmsocket_t **sockp FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(sockp != NULL && *sockp != NULL);
REQUIRE(VALID_NMSOCK(*sockp));
isc_nmsocket_t *sock = *sockp, *rsock = NULL;
*sockp = NULL;
/*
* If the socket is a part of a set (a child socket) we are
* counting references for the whole set at the parent.
*/
if (sock->parent != NULL) {
rsock = sock->parent;
INSIST(rsock->parent == NULL); /* Sanity check */
} else {
rsock = sock;
}
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc__nmsocket_detach():%p->references = %" PRIuFAST32
"\n",
rsock, isc_refcount_current(&rsock->references) - 1);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
Use isc_refcount_increment0() when reusing handle or socket; remove extra DbC checks
2020-01-14 09:43:37 +01:00
if (isc_refcount_decrement(&rsock->references) == 1) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_prep_destroy(rsock FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
}
make isc_nmsocket_{attach,detach}{} functions private there is no need for a caller to reference-count socket objects. they need tto be able tto close listener sockets (i.e., those returned by isc_nm_listen{udp,tcp,tcpdns}), and an isc_nmsocket_close() function has been added for that. other sockets are only accessed via handles.
2020-06-04 14:54:36 -07:00
void
isc_nmsocket_close(isc_nmsocket_t **sockp) {
REQUIRE(sockp != NULL);
REQUIRE(VALID_NMSOCK(*sockp));
REQUIRE((*sockp)->type == isc_nm_udplistener ||
(*sockp)->type == isc_nm_tcplistener ||
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
(*sockp)->type == isc_nm_tcpdnslistener ||
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
(*sockp)->type == isc_nm_tlsdnslistener ||
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
(*sockp)->type == isc_nm_tlslistener ||
(*sockp)->type == isc_nm_httplistener);
make isc_nmsocket_{attach,detach}{} functions private there is no need for a caller to reference-count socket objects. they need tto be able tto close listener sockets (i.e., those returned by isc_nm_listen{udp,tcp,tcpdns}), and an isc_nmsocket_close() function has been added for that. other sockets are only accessed via handles.
2020-06-04 14:54:36 -07:00
isc__nmsocket_detach(sockp);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_init(isc_nmsocket_t *sock, isc_nm_t *mgr, isc_nmsocket_type type,
Refactor the interface handling in the netmgr The isc_nmiface_t type was holding just a single isc_sockaddr_t, so we got rid of the datatype and use plain isc_sockaddr_t in place where isc_nmiface_t was used before. This means less type-casting and shorter path to access isc_sockaddr_t members. At the same time, instead of keeping the reference to the isc_sockaddr_t that was passed to us when we start listening, we will keep a local copy. This prevents the data race on destruction of the ns_interface_t objects where pending nmsockets could reference the sockaddr of already destroyed ns_interface_t object.
2021-05-26 08:15:34 +02:00
isc_sockaddr_t *iface FLARG) {
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
uint16_t family;
REQUIRE(sock != NULL);
REQUIRE(mgr != NULL);
Increase inactivehandles and inactivereqs size for better reuse.
2020-01-29 15:16:02 +01:00
*sock = (isc_nmsocket_t){ .type = type,
.fd = -1,
.inactivehandles = isc_astack_new(
mgr->mctx, ISC_NM_HANDLES_STACK_SIZE),
.inactivereqs = isc_astack_new(
mgr->mctx, ISC_NM_REQS_STACK_SIZE) };
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
netmgr: add isc_nm_routeconnect() isc_nm_routeconnect() opens a route/netlink socket, then calls a connect callback, much like isc_nm_udpconnect(), with a handle that can then be monitored for network changes. Internally the socket is treated as a UDP socket, since route/netlink sockets follow the datagram contract.
2021-10-02 14:52:46 -07:00
if (iface != NULL) {
family = iface->type.sa.sa_family;
sock->iface = *iface;
} else {
family = AF_UNSPEC;
}
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
#if NETMGR_TRACE
Add support for generating backtraces on Windows This commit adds support for generating backtraces on Windows and refactors the isc_backtrace API to match the Linux/BSD API (without the isc_ prefix) * isc_backtrace_gettrace() was renamed to isc_backtrace(), the third argument was removed and the return type was changed to int * isc_backtrace_symbols() was added * isc_backtrace_symbols_fd() was added and used as appropriate
2021-04-27 16:20:03 +02:00
sock->backtrace_size = isc_backtrace(sock->backtrace, TRACE_SIZE);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
ISC_LINK_INIT(sock, active_link);
ISC_LIST_INIT(sock->active_handles);
LOCK(&mgr->lock);
ISC_LIST_APPEND(mgr->active_sockets, sock, active_link);
UNLOCK(&mgr->lock);
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nm_attach(mgr, &sock->mgr);
sock->uv_handle.handle.data = sock;
Redesigned TCP accepting: one listen/accept loop, passing the connected socket. Instead of using bind() and passing the listening socket to the children threads using uv_export/uv_import use one thread that does the accepting, and then passes the connected socket using uv_export/uv_import to a random worker. The previous solution had thundering herd problems (all workers waking up on one connection and trying to accept()), this one avoids this and is simpler. The tcp clients quota is simplified with isc_quota_attach_cb - a callback is issued when the quota is available.
2020-03-24 13:38:51 +01:00
ISC_LINK_INIT(&sock->quotacb, link);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
switch (type) {
case isc_nm_udpsocket:
case isc_nm_udplistener:
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
switch (family) {
case AF_INET:
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
sock->statsindex = udp4statsindex;
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
break;
case AF_INET6:
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
sock->statsindex = udp6statsindex;
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
break;
netmgr: add isc_nm_routeconnect() isc_nm_routeconnect() opens a route/netlink socket, then calls a connect callback, much like isc_nm_udpconnect(), with a handle that can then be monitored for network changes. Internally the socket is treated as a UDP socket, since route/netlink sockets follow the datagram contract.
2021-10-02 14:52:46 -07:00
case AF_UNSPEC:
/*
* Route sockets are AF_UNSPEC, and don't
* have stats counters.
*/
break;
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
}
break;
case isc_nm_tcpsocket:
case isc_nm_tcplistener:
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tcpdnssocket:
case isc_nm_tcpdnslistener:
case isc_nm_tlsdnssocket:
case isc_nm_tlsdnslistener:
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
case isc_nm_httpsocket:
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
case isc_nm_httplistener:
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
switch (family) {
case AF_INET:
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
sock->statsindex = tcp4statsindex;
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
break;
case AF_INET6:
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
sock->statsindex = tcp6statsindex;
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
break;
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
}
break was on wrong line. 959 break; CID 1457872 (#1 of 1): Structurally dead code (UNREACHABLE) unreachable: This code cannot be reached: isc__nm_incstats(sock->mgr,.... 960 isc__nm_incstats(sock->mgr, sock->statsindex[STATID_ACTIVE]); 961 default:
2020-02-05 15:50:29 +11:00
break;
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
default:
break;
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_mutex_init(&sock->lock);
initalise sock->cond
2021-05-11 14:01:37 +10:00
isc_condition_init(&sock->cond);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_condition_init(&sock->scond);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_refcount_init(&sock->references, 1);
place a limit on pipelined queries that can be processed simultaneously when the TCPDNS_CLIENTS_PER_CONN limit has been exceeded for a TCP DNS connection, switch to sequential mode to ensure that memory cannot be exhausted by too many simultaneous queries.
2019-11-08 10:52:49 -08:00
Get rid of some HTTP/2 related types when NGHTTP2 is not available This commit removes definitions of some DoH-related types when libnghttp2 is not available.
2021-07-16 14:57:08 +03:00
#if HAVE_LIBNGHTTP2
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
memset(&sock->tlsstream, 0, sizeof(sock->tlsstream));
Get rid of some HTTP/2 related types when NGHTTP2 is not available This commit removes definitions of some DoH-related types when libnghttp2 is not available.
2021-07-16 14:57:08 +03:00
#endif /* HAVE_LIBNGHTTP2 */
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc__nmsocket_init():%p->references = %" PRIuFAST32
"\n",
sock, isc_refcount_current(&sock->references));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
atomic_init(&sock->active, true);
place a limit on pipelined queries that can be processed simultaneously when the TCPDNS_CLIENTS_PER_CONN limit has been exceeded for a TCP DNS connection, switch to sequential mode to ensure that memory cannot be exhausted by too many simultaneous queries.
2019-11-08 10:52:49 -08:00
atomic_init(&sock->readpaused, false);
Fix more races between connect and shutdown There were more races that could happen while connecting to a socket while closing or shutting down the same socket. This commit introduces a .closing flag to guard the socket from being closed twice.
2020-10-29 12:04:00 +01:00
atomic_init(&sock->closing, false);
Add missing initialisations configuring with --enable-mutex-atomics flagged these incorrectly initialised variables on systems where pthread_mutex_init doesn't just zero out the structure.
2021-05-26 17:55:43 +10:00
atomic_init(&sock->listening, 0);
atomic_init(&sock->closed, 0);
atomic_init(&sock->destroying, 0);
atomic_init(&sock->ah, 0);
atomic_init(&sock->client, 0);
atomic_init(&sock->connecting, false);
atomic_init(&sock->keepalive, false);
atomic_init(&sock->connected, false);
Delay isc__nm_uvreq_t deallocation to connection callback When the TCP, TCPDNS or TLSDNS connection times out, the isc__nm_uvreq_t would be pushed into sock->inactivereqs before the uv_tcp_connect() callback finishes. Because the isc__nmsocket_t keeps the list of inactive isc__nm_uvreq_t, this would cause use-after-free only when the sock->inactivereqs is full (which could never happen because the failure happens in connection timeout callback) or when the sock->inactivereqs mechanism is completely removed (f.e. when running under Address or Thread Sanitizer). Delay isc__nm_uvreq_t deallocation to the connection callback and only signal the connection callback should be called by shutting down the libuv socket from the connection timeout callback.
2022-02-22 18:12:18 +01:00
atomic_init(&sock->timedout, false);
Add missing initialisations configuring with --enable-mutex-atomics flagged these incorrectly initialised variables on systems where pthread_mutex_init doesn't just zero out the structure.
2021-05-26 17:55:43 +10:00
atomic_init(&sock->active_child_connections, 0);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
isc__nm_http_initsocket(sock);
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
sock->magic = NMSOCK_MAGIC;
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
isc__nm_incstats(sock, STATID_ACTIVE);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
clean up outerhandle when a tcpdns socket is disconnected this prevents a crash when some non-netmgr thread, such as a recursive lookup, times out after the TCP socket is already disconnected.
2020-06-20 15:03:05 -07:00
void
isc__nmsocket_clearcb(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
Clone the csock in accept_connection(), not in callback If we clone the csock (children socket) in TCP accept_connection() instead of passing the ssock (server socket) to the call back and cloning it there we unbreak the assumption that every socket is handled inside it's own worker thread and therefore we can get rid of (at least) callback locking.
2020-09-18 12:27:40 +02:00
REQUIRE(!isc__nm_in_netthread() || sock->tid == isc_nm_tid());
clean up outerhandle when a tcpdns socket is disconnected this prevents a crash when some non-netmgr thread, such as a recursive lookup, times out after the TCP socket is already disconnected.
2020-06-20 15:03:05 -07:00
properly lock the setting/unsetting of callbacks in isc_nmsocket_t changes to socket callback functions were not thread safe.
2020-09-11 10:53:31 +02:00
sock->recv_cb = NULL;
sock->recv_cbarg = NULL;
sock->accept_cb = NULL;
clean up outerhandle when a tcpdns socket is disconnected this prevents a crash when some non-netmgr thread, such as a recursive lookup, times out after the TCP socket is already disconnected.
2020-06-20 15:03:05 -07:00
sock->accept_cbarg = NULL;
Fix more races between connect and shutdown There were more races that could happen while connecting to a socket while closing or shutting down the same socket. This commit introduces a .closing flag to guard the socket from being closed twice.
2020-10-29 12:04:00 +01:00
sock->connect_cb = NULL;
sock->connect_cbarg = NULL;
clean up outerhandle when a tcpdns socket is disconnected this prevents a crash when some non-netmgr thread, such as a recursive lookup, times out after the TCP socket is already disconnected.
2020-06-20 15:03:05 -07:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_free_uvbuf(isc_nmsocket_t *sock, const uv_buf_t *buf) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__networker_t *worker = NULL;
REQUIRE(VALID_NMSOCK(sock));
Fix the UDP recvmmsg support Previously, the netmgr/udp.c tried to detect the recvmmsg detection in libuv with #ifdef UV_UDP_<foo> preprocessor macros. However, because the UV_UDP_<foo> are not preprocessor macros, but enum members, the detection didn't work. Because the detection didn't work, the code didn't have access to the information when we received the final chunk of the recvmmsg and tried to free the uvbuf every time. Fortunately, the isc__nm_free_uvbuf() had a kludge that detected attempt to free in the middle of the receive buffer, so the code worked. However, libuv 1.37.0 changed the way the recvmmsg was enabled from implicit to explicit, and we checked for yet another enum member presence with preprocessor macro, so in fact libuv recvmmsg support was never enabled with libuv >= 1.37.0. This commit changes to the preprocessor macros to autoconf checks for declaration, so the detection now works again. On top of that, it's now possible to cleanup the alloc_cb and free_uvbuf functions because now, the information whether we can or cannot free the buffer is available to us.
2022-01-11 12:14:23 +01:00
worker = &sock->mgr->workers[sock->tid];
clean up comments
2019-11-15 13:22:13 -08:00
REQUIRE(buf->base == worker->recvbuf);
Fix the UDP recvmmsg support Previously, the netmgr/udp.c tried to detect the recvmmsg detection in libuv with #ifdef UV_UDP_<foo> preprocessor macros. However, because the UV_UDP_<foo> are not preprocessor macros, but enum members, the detection didn't work. Because the detection didn't work, the code didn't have access to the information when we received the final chunk of the recvmmsg and tried to free the uvbuf every time. Fortunately, the isc__nm_free_uvbuf() had a kludge that detected attempt to free in the middle of the receive buffer, so the code worked. However, libuv 1.37.0 changed the way the recvmmsg was enabled from implicit to explicit, and we checked for yet another enum member presence with preprocessor macro, so in fact libuv recvmmsg support was never enabled with libuv >= 1.37.0. This commit changes to the preprocessor macros to autoconf checks for declaration, so the detection now works again. On top of that, it's now possible to cleanup the alloc_cb and free_uvbuf functions because now, the information whether we can or cannot free the buffer is available to us.
2022-01-11 12:14:23 +01:00
clean up comments
2019-11-15 13:22:13 -08:00
worker->recvbuf_inuse = false;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
static isc_nmhandle_t *
apply the modified style
2020-02-13 14:44:37 -08:00
alloc_handle(isc_nmsocket_t *sock) {
Remove extrahandle size from netmgr Previously, it was possible to assign a bit of memory space in the nmhandle to store the client data. This was complicated and prevents further refactoring of isc_nmhandle_t caching (future work). Instead of caching the data in the nmhandle, allocate the hot-path ns_client_t objects from per-thread clientmgr memory context and just assign it to the isc_nmhandle_t via isc_nmhandle_set().
2022-03-23 13:57:15 +01:00
isc_nmhandle_t *handle = isc_mem_get(sock->mgr->mctx,
sizeof(isc_nmhandle_t));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*handle = (isc_nmhandle_t){ .magic = NMHANDLE_MAGIC };
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
ISC_LINK_INIT(handle, active_link);
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_refcount_init(&handle->references, 1);
return (handle);
}
isc_nmhandle_t *
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmhandle_get(isc_nmsocket_t *sock, isc_sockaddr_t *peer,
isc_sockaddr_t *local FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nmhandle_t *handle = NULL;
REQUIRE(VALID_NMSOCK(sock));
handle = isc_astack_pop(sock->inactivehandles);
if (handle == NULL) {
handle = alloc_handle(sock);
} else {
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
isc_refcount_init(&handle->references, 1);
Don't destroy a non-closed socket, wait for all the callbacks. We erroneously tried to destroy a socket after issuing isc__nm_tcp{,dns}_close. Under some (race) circumstances we could get nm_socket_cleanup to be called twice for the same socket, causing an access to a dead memory.
2020-07-01 00:49:12 -07:00
INSIST(VALID_NMHANDLE(handle));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG(
"isc__nmhandle_get():handle %p->references = %" PRIuFAST32 "\n",
handle, isc_refcount_current(&handle->references));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_attach(sock, &handle->sock FLARG_PASS);
modify reference counting within netmgr - isc__nmhandle_get() now attaches to the sock in the nmhandle object. the caller is responsible for dereferencing the original socket pointer when necessary. - tcpdns listener sockets attach sock->outer to the outer tcp listener socket. tcpdns connected sockets attach sock->outerhandle to the handle for the tcp connected socket. - only listener sockets need to be attached/detached directly. connected sockets should only be accessed and reference-counted via their associated handles.
2020-06-04 23:13:54 -07:00
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
#if NETMGR_TRACE
Add support for generating backtraces on Windows This commit adds support for generating backtraces on Windows and refactors the isc_backtrace API to match the Linux/BSD API (without the isc_ prefix) * isc_backtrace_gettrace() was renamed to isc_backtrace(), the third argument was removed and the return type was changed to int * isc_backtrace_symbols() was added * isc_backtrace_symbols_fd() was added and used as appropriate
2021-04-27 16:20:03 +02:00
handle->backtrace_size = isc_backtrace(handle->backtrace, TRACE_SIZE);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (peer != NULL) {
Refactor the interface handling in the netmgr The isc_nmiface_t type was holding just a single isc_sockaddr_t, so we got rid of the datatype and use plain isc_sockaddr_t in place where isc_nmiface_t was used before. This means less type-casting and shorter path to access isc_sockaddr_t members. At the same time, instead of keeping the reference to the isc_sockaddr_t that was passed to us when we start listening, we will keep a local copy. This prevents the data race on destruction of the ns_interface_t objects where pending nmsockets could reference the sockaddr of already destroyed ns_interface_t object.
2021-05-26 08:15:34 +02:00
handle->peer = *peer;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
} else {
Refactor the interface handling in the netmgr The isc_nmiface_t type was holding just a single isc_sockaddr_t, so we got rid of the datatype and use plain isc_sockaddr_t in place where isc_nmiface_t was used before. This means less type-casting and shorter path to access isc_sockaddr_t members. At the same time, instead of keeping the reference to the isc_sockaddr_t that was passed to us when we start listening, we will keep a local copy. This prevents the data race on destruction of the ns_interface_t objects where pending nmsockets could reference the sockaddr of already destroyed ns_interface_t object.
2021-05-26 08:15:34 +02:00
handle->peer = sock->peer;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
if (local != NULL) {
Refactor the interface handling in the netmgr The isc_nmiface_t type was holding just a single isc_sockaddr_t, so we got rid of the datatype and use plain isc_sockaddr_t in place where isc_nmiface_t was used before. This means less type-casting and shorter path to access isc_sockaddr_t members. At the same time, instead of keeping the reference to the isc_sockaddr_t that was passed to us when we start listening, we will keep a local copy. This prevents the data race on destruction of the ns_interface_t objects where pending nmsockets could reference the sockaddr of already destroyed ns_interface_t object.
2021-05-26 08:15:34 +02:00
handle->local = *local;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
} else {
Refactor the interface handling in the netmgr The isc_nmiface_t type was holding just a single isc_sockaddr_t, so we got rid of the datatype and use plain isc_sockaddr_t in place where isc_nmiface_t was used before. This means less type-casting and shorter path to access isc_sockaddr_t members. At the same time, instead of keeping the reference to the isc_sockaddr_t that was passed to us when we start listening, we will keep a local copy. This prevents the data race on destruction of the ns_interface_t objects where pending nmsockets could reference the sockaddr of already destroyed ns_interface_t object.
2021-05-26 08:15:34 +02:00
handle->local = sock->iface;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Remove active handles tracking from isc__nmsocket_t The isc__nmsocket_t has locked array of isc_nmhandle_t that's not used for anything. The isc__nmhandle_get() adds the isc_nmhandle_t to the locked array (and resized if necessary) and removed when isc_nmhandle_put() finally destroys the handle. That's all it does, so it serves no useful purpose. Remove the .ah_handles, .ah_size, and .ah_frees members of the isc__nmsocket_t and .ah_pos member of the isc_nmhandle_t struct.
2022-02-23 08:54:49 +01:00
(void)atomic_fetch_add(&sock->ah, 1);
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
Remove active handles tracking from isc__nmsocket_t The isc__nmsocket_t has locked array of isc_nmhandle_t that's not used for anything. The isc__nmhandle_get() adds the isc_nmhandle_t to the locked array (and resized if necessary) and removed when isc_nmhandle_put() finally destroys the handle. That's all it does, so it serves no useful purpose. Remove the .ah_handles, .ah_size, and .ah_frees members of the isc__nmsocket_t and .ah_pos member of the isc_nmhandle_t struct.
2022-02-23 08:54:49 +01:00
LOCK(&sock->lock);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
ISC_LIST_APPEND(sock->active_handles, handle, active_link);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
UNLOCK(&sock->lock);
Remove active handles tracking from isc__nmsocket_t The isc__nmsocket_t has locked array of isc_nmhandle_t that's not used for anything. The isc__nmhandle_get() adds the isc_nmhandle_t to the locked array (and resized if necessary) and removed when isc_nmhandle_put() finally destroys the handle. That's all it does, so it serves no useful purpose. Remove the .ah_handles, .ah_size, and .ah_frees members of the isc__nmsocket_t and .ah_pos member of the isc_nmhandle_t struct.
2022-02-23 08:54:49 +01:00
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
switch (sock->type) {
case isc_nm_udpsocket:
case isc_nm_tcpdnssocket:
case isc_nm_tlsdnssocket:
if (!atomic_load(&sock->client)) {
break;
}
Add FALLTHROUGH macro for __attribute__((fallthrough)) Gcc 7+ and Clang 10+ have implemented __attribute__((fallthrough)) which is explicit version of the /* FALLTHROUGH */ comment we are currently using. Add and apply FALLTHROUGH macro that uses the attribute if available, but does nothing on older compilers. In one case (lib/dns/zone.c), using the macro revealed that we were using the /* FALLTHROUGH */ comment in wrong place, remove that comment.
2021-10-11 12:09:16 +02:00
FALLTHROUGH;
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
case isc_nm_tcpsocket:
case isc_nm_tlssocket:
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
INSIST(sock->statichandle == NULL);
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
/*
* statichandle must be assigned, not attached;
* otherwise, if a handle was detached elsewhere
* it could never reach 0 references, and the
* handle and socket would never be freed.
*/
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
sock->statichandle = handle;
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
break;
default:
break;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
if (sock->type == isc_nm_httpsocket && sock->h2.session) {
isc__nm_httpsession_attach(sock->h2.session,
&handle->httpsession);
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
}
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
return (handle);
}
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nmhandle_attach(isc_nmhandle_t *handle, isc_nmhandle_t **handlep FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
REQUIRE(handlep != NULL && *handlep == NULL);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc__nmhandle_attach():handle %p->references = "
"%" PRIuFAST32 "\n",
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
handle, isc_refcount_current(&handle->references) + 1);
Use isc_refcount_increment0() when reusing handle or socket; remove extra DbC checks
2020-01-14 09:43:37 +01:00
isc_refcount_increment(&handle->references);
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
*handlep = handle;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
bool
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_is_stream(isc_nmhandle_t *handle) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
return (handle->sock->type == isc_nm_tcpsocket ||
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
handle->sock->type == isc_nm_tcpdnssocket ||
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
handle->sock->type == isc_nm_tlssocket ||
Fix BIND to serve large HTTP responses This commit makes NM code to report HTTP as a stream protocol. This makes it possible to handle large responses properly. Like: dig +https @127.0.0.1 A cmts1-dhcp.longlines.com
2021-06-01 15:16:19 +03:00
handle->sock->type == isc_nm_tlsdnssocket ||
handle->sock->type == isc_nm_httpsocket);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
static void
apply the modified style
2020-02-13 14:44:37 -08:00
nmhandle_free(isc_nmsocket_t *sock, isc_nmhandle_t *handle) {
Add isc_refcount_destroy() call to nm_handle_free()
2019-12-10 11:09:56 +01:00
isc_refcount_destroy(&handle->references);
place a limit on pipelined queries that can be processed simultaneously when the TCPDNS_CLIENTS_PER_CONN limit has been exceeded for a TCP DNS connection, switch to sequential mode to ensure that memory cannot be exhausted by too many simultaneous queries.
2019-11-08 10:52:49 -08:00
if (handle->dofree != NULL) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
handle->dofree(handle->opaque);
}
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*handle = (isc_nmhandle_t){ .magic = 0 };
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
Remove extrahandle size from netmgr Previously, it was possible to assign a bit of memory space in the nmhandle to store the client data. This was complicated and prevents further refactoring of isc_nmhandle_t caching (future work). Instead of caching the data in the nmhandle, allocate the hot-path ns_client_t objects from per-thread clientmgr memory context and just assign it to the isc_nmhandle_t via isc_nmhandle_set().
2022-03-23 13:57:15 +01:00
isc_mem_put(sock->mgr->mctx, handle, sizeof(isc_nmhandle_t));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
static void
apply the modified style
2020-02-13 14:44:37 -08:00
nmhandle_deactivate(isc_nmsocket_t *sock, isc_nmhandle_t *handle) {
implement isc_nm_tcpconnect() the isc_nm_tcpconnect() function establishes a client connection via TCP. once the connection is esablished, a callback function will be called with a newly created network manager handle.
2019-12-16 18:24:55 -08:00
bool reuse = false;
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
/*
* We do all of this under lock to avoid races with socket
* destruction. We have to do this now, because at this point the
* socket is either unused or still attached to event->sock.
*/
LOCK(&sock->lock);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
ISC_LIST_UNLINK(sock->active_handles, handle, active_link);
#endif
Remove active handles tracking from isc__nmsocket_t The isc__nmsocket_t has locked array of isc_nmhandle_t that's not used for anything. The isc__nmhandle_get() adds the isc_nmhandle_t to the locked array (and resized if necessary) and removed when isc_nmhandle_put() finally destroys the handle. That's all it does, so it serves no useful purpose. Remove the .ah_handles, .ah_size, and .ah_frees members of the isc__nmsocket_t and .ah_pos member of the isc_nmhandle_t struct.
2022-02-23 08:54:49 +01:00
INSIST(atomic_fetch_sub(&sock->ah, 1) > 0);
Fix typo __SANITIZE_ADDRESS -> __SANITIZE_ADDRESS__ When checking for Address Sanitizer to disable the inactivehandles caching, there was a typo in the macro.
2022-02-24 00:11:45 +01:00
#if !__SANITIZE_ADDRESS__ && !__SANITIZE_THREAD__
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
if (atomic_load(&sock->active)) {
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
reuse = isc_astack_trypush(sock->inactivehandles, handle);
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
}
Fix typo __SANITIZE_ADDRESS -> __SANITIZE_ADDRESS__ When checking for Address Sanitizer to disable the inactivehandles caching, there was a typo in the macro.
2022-02-24 00:11:45 +01:00
#endif /* !__SANITIZE_ADDRESS__ && !__SANITIZE_THREAD__ */
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
if (!reuse) {
nmhandle_free(sock, handle);
}
UNLOCK(&sock->lock);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nmhandle_detach(isc_nmhandle_t **handlep FLARG) {
modify reference counting within netmgr - isc__nmhandle_get() now attaches to the sock in the nmhandle object. the caller is responsible for dereferencing the original socket pointer when necessary. - tcpdns listener sockets attach sock->outer to the outer tcp listener socket. tcpdns connected sockets attach sock->outerhandle to the handle for the tcp connected socket. - only listener sockets need to be attached/detached directly. connected sockets should only be accessed and reference-counted via their associated handles.
2020-06-04 23:13:54 -07:00
isc_nmsocket_t *sock = NULL;
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
isc_nmhandle_t *handle = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
REQUIRE(handlep != NULL);
REQUIRE(VALID_NMHANDLE(*handlep));
handle = *handlep;
*handlep = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Run .closehandle_cb asynchrounosly in nmhandle_detach_cb() When sock->closehandle_cb is set, we need to run nmhandle_detach_cb() asynchronously to ensure correct order of multiple packets processing in the isc__nm_process_sock_buffer(). When not run asynchronously, it would cause: a) out-of-order processing of the return codes from processbuffer(); b) stack growth because the next TCP DNS message read callback will be called from within the current TCP DNS message read callback. The sock->closehandle_cb is set to isc__nm_resume_processing() for TCP sockets which calls isc__nm_process_sock_buffer(). If the read callback (called from isc__nm_process_sock_buffer()->processbuffer()) doesn't attach to the nmhandle (f.e. because it wants to drop the processing or we send the response directly via uv_try_write()), the isc__nm_resume_processing() (via .closehandle_cb) would call isc__nm_process_sock_buffer() recursively. The below shortened code path shows how the stack can grow: 1: ns__client_request(handle, ...); 2: isc_nm_tcpdns_sequential(handle); 3: ns_query_start(client, handle); 4: query_lookup(qctx); 5: query_send(qctcx->client); 6: isc__nmhandle_detach(&client->reqhandle); 7: nmhandle_detach_cb(&handle); 8: sock->closehandle_cb(sock); // isc__nm_resume_processing 9: isc__nm_process_sock_buffer(sock); 10: processbuffer(sock); // isc__nm_tcpdns_processbuffer 11: isc_nmhandle_attach(req->handle, &handle); 12: isc__nm_readcb(sock, req, ISC_R_SUCCESS); 13: isc__nm_async_readcb(NULL, ...); 14: uvreq->cb.recv(...); // ns__client_request Instead, if 'sock->closehandle_cb' is set, we need to run detach the handle asynchroniously in 'isc__nmhandle_detach', so that on line 8 in the code flow above does not start this recursion. This ensures the correct order when processing multiple packets in the function 'isc__nm_process_sock_buffer()' and prevents the stack growth. When not run asynchronously, the out-of-order processing leaves the first TCP socket open until all requests on the stream have been processed. If the pipelining is disabled on the TCP via `keep-response-order` configuration option, named would keep the first socket in lingering CLOSE_WAIT state when the client sends an incomplete packet and then closes the connection from the client side.
2022-02-08 12:42:34 +01:00
/*
* If the closehandle_cb is set, it needs to run asynchronously to
* ensure correct ordering of the isc__nm_process_sock_buffer().
*/
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
sock = handle->sock;
Run .closehandle_cb asynchrounosly in nmhandle_detach_cb() When sock->closehandle_cb is set, we need to run nmhandle_detach_cb() asynchronously to ensure correct order of multiple packets processing in the isc__nm_process_sock_buffer(). When not run asynchronously, it would cause: a) out-of-order processing of the return codes from processbuffer(); b) stack growth because the next TCP DNS message read callback will be called from within the current TCP DNS message read callback. The sock->closehandle_cb is set to isc__nm_resume_processing() for TCP sockets which calls isc__nm_process_sock_buffer(). If the read callback (called from isc__nm_process_sock_buffer()->processbuffer()) doesn't attach to the nmhandle (f.e. because it wants to drop the processing or we send the response directly via uv_try_write()), the isc__nm_resume_processing() (via .closehandle_cb) would call isc__nm_process_sock_buffer() recursively. The below shortened code path shows how the stack can grow: 1: ns__client_request(handle, ...); 2: isc_nm_tcpdns_sequential(handle); 3: ns_query_start(client, handle); 4: query_lookup(qctx); 5: query_send(qctcx->client); 6: isc__nmhandle_detach(&client->reqhandle); 7: nmhandle_detach_cb(&handle); 8: sock->closehandle_cb(sock); // isc__nm_resume_processing 9: isc__nm_process_sock_buffer(sock); 10: processbuffer(sock); // isc__nm_tcpdns_processbuffer 11: isc_nmhandle_attach(req->handle, &handle); 12: isc__nm_readcb(sock, req, ISC_R_SUCCESS); 13: isc__nm_async_readcb(NULL, ...); 14: uvreq->cb.recv(...); // ns__client_request Instead, if 'sock->closehandle_cb' is set, we need to run detach the handle asynchroniously in 'isc__nmhandle_detach', so that on line 8 in the code flow above does not start this recursion. This ensures the correct order when processing multiple packets in the function 'isc__nm_process_sock_buffer()' and prevents the stack growth. When not run asynchronously, the out-of-order processing leaves the first TCP socket open until all requests on the stream have been processed. If the pipelining is disabled on the TCP via `keep-response-order` configuration option, named would keep the first socket in lingering CLOSE_WAIT state when the client sends an incomplete packet and then closes the connection from the client side.
2022-02-08 12:42:34 +01:00
if (sock->tid == isc_nm_tid() && sock->closehandle_cb == NULL) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmhandle_detach_cb(&handle FLARG_PASS);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
} else {
isc__netievent_detach_t *event =
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_get_netievent_detach(sock->mgr, sock);
/*
* we are using implicit "attach" as the last reference
* need to be destroyed explicitly in the async callback
*/
event->handle = handle;
FLARG_IEVENT_PASS(event);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
isc__nm_enqueue_ievent(&sock->mgr->workers[sock->tid],
(isc__netievent_t *)event);
}
}
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmhandle_detach_cb(isc_nmhandle_t **handlep FLARG) {
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
isc_nmsocket_t *sock = NULL;
isc_nmhandle_t *handle = NULL;
REQUIRE(handlep != NULL);
REQUIRE(VALID_NMHANDLE(*handlep));
handle = *handlep;
*handlep = NULL;
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc__nmhandle_detach():%p->references = %" PRIuFAST32
"\n",
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
handle, isc_refcount_current(&handle->references) - 1);
Use isc_refcount_increment0() when reusing handle or socket; remove extra DbC checks
2020-01-14 09:43:37 +01:00
if (isc_refcount_decrement(&handle->references) > 1) {
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
return;
}
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
Add missing acquire memory barrier in isc_nmhandle_unref The ThreadSanitizer uses system synchronization primitives to check for data race. The netmgr handle->references was missing acquire memory barrier before resetting and reusing the memory occupied by isc_nmhandle_t.
2020-06-11 13:01:26 +02:00
/* We need an acquire memory barrier here */
(void)isc_refcount_current(&handle->references);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
sock = handle->sock;
handle->sock = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
if (handle->doreset != NULL) {
handle->doreset(handle->opaque);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
if (sock->type == isc_nm_httpsocket && handle->httpsession != NULL) {
isc__nm_httpsession_detach(&handle->httpsession);
}
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
Deactivate the handle before sending the async close callback. We could have a race between handle closing and processing async callback. Deactivate the handle before issuing the callback - we have the socket referenced anyway so it's not a problem.
2020-03-26 14:25:06 +01:00
nmhandle_deactivate(sock, handle);
/*
* The handle is gone now. If the socket has a callback configured
netmgr: fix a race in socket destruction, happening if we close the socket externally and, at the same time, a timeout timer callback was called.
2019-12-06 20:45:00 +01:00
* for that (e.g., to perform cleanup after request processing),
Fix a race in tcpdns close with uv_close on timer stop timers before closing netmgr: tcpdns_close needs to be asynchronous, it manipulates sock->timer
2019-12-06 22:25:52 +01:00
* call it now, or schedule it to run asynchronously.
netmgr: fix a race in socket destruction, happening if we close the socket externally and, at the same time, a timeout timer callback was called.
2019-12-06 20:45:00 +01:00
*/
if (sock->closehandle_cb != NULL) {
if (sock->tid == isc_nm_tid()) {
sock->closehandle_cb(sock);
} else {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__netievent_close_t *event =
isc__nm_get_netievent_close(sock->mgr, sock);
netmgr: fix a race in socket destruction, happening if we close the socket externally and, at the same time, a timeout timer callback was called.
2019-12-06 20:45:00 +01:00
isc__nm_enqueue_ievent(&sock->mgr->workers[sock->tid],
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
(isc__netievent_t *)event);
netmgr: fix a race in socket destruction, happening if we close the socket externally and, at the same time, a timeout timer callback was called.
2019-12-06 20:45:00 +01:00
}
}
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
if (handle == sock->statichandle) {
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
/* statichandle is assigned, not attached. */
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
sock->statichandle = NULL;
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_detach(&sock FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void *
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_getdata(isc_nmhandle_t *handle) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
return (handle->opaque);
}
void
isc_nmhandle_setdata(isc_nmhandle_t *handle, void *arg,
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_opaquecb_t doreset, isc_nm_opaquecb_t dofree) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
handle->opaque = arg;
handle->doreset = doreset;
handle->dofree = dofree;
}
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
void
isc__nm_alloc_dnsbuf(isc_nmsocket_t *sock, size_t len) {
REQUIRE(len <= NM_BIG_BUF);
if (sock->buf == NULL) {
/* We don't have the buffer at all */
size_t alloc_len = len < NM_REG_BUF ? NM_REG_BUF : NM_BIG_BUF;
Replace isc_mem_allocate() usage with isc_mem_get() in netmgr.c The isc_mem_allocate() comes with additional cost because of the memory tracking. In this commit, we replace the usage with isc_mem_get() because we track the allocated sizes anyway, so it's possible to also replace isc_mem_free() with isc_mem_put().
2021-05-11 14:37:18 +02:00
sock->buf = isc_mem_get(sock->mgr->mctx, alloc_len);
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
sock->buf_size = alloc_len;
} else {
/* We have the buffer but it's too small */
Preserve the contents of socket buffer on realloc On TCPDNS/TLSDNS read callback, the socket buffer could be reallocated if the received contents would be larger than the buffer. The existing code would not preserve the contents of the existing buffer which lead to the loss of the already received data. This commit changes the isc_mem_put()+isc_mem_get() with isc_mem_reget() to preserve the existing contents of the socket buffer.
2021-09-22 18:55:02 +02:00
sock->buf = isc_mem_reget(sock->mgr->mctx, sock->buf,
sock->buf_size, NM_BIG_BUF);
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
sock->buf_size = NM_BIG_BUF;
}
}
void
isc__nm_failed_send_cb(isc_nmsocket_t *sock, isc__nm_uvreq_t *req,
isc_result_t eresult) {
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(req));
if (req->cb.send != NULL) {
isc__nm_sendcb(sock, req, eresult, true);
} else {
isc__nm_uvreq_put(&req, sock);
}
}
void
isc__nm_failed_accept_cb(isc_nmsocket_t *sock, isc_result_t eresult) {
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
REQUIRE(atomic_load(&sock->accepting));
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
REQUIRE(sock->server);
/*
* Detach the quota early to make room for other connections;
* otherwise it'd be detached later asynchronously, and clog
* the quota unnecessarily.
*/
if (sock->quota != NULL) {
isc_quota_detach(&sock->quota);
}
isc__nmsocket_detach(&sock->server);
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
atomic_store(&sock->accepting, false);
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
switch (eresult) {
case ISC_R_NOTCONNECTED:
/* IGNORE: The client disconnected before we could accept */
break;
default:
isc_log_write(isc_lctx, ISC_LOGCATEGORY_GENERAL,
ISC_LOGMODULE_NETMGR, ISC_LOG_ERROR,
"Accepting TCP connection failed: %s",
isc_result_totext(eresult));
}
}
void
isc__nm_failed_connect_cb(isc_nmsocket_t *sock, isc__nm_uvreq_t *req,
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc_result_t eresult, bool async) {
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(req));
REQUIRE(sock->tid == isc_nm_tid());
REQUIRE(req->cb.connect != NULL);
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
isc__nmsocket_timer_stop(sock);
Rename sock->timer to sock->read_timer Before adding the write timer, we have to remove the generic sock->timer to sock->read_timer. We don't touch the function names to limit the impact of the refactoring.
2022-02-09 10:59:08 +01:00
uv_handle_set_data((uv_handle_t *)&sock->read_timer, sock);
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
INSIST(atomic_compare_exchange_strong(&sock->connecting,
&(bool){ true }, false));
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
isc__nmsocket_clearcb(sock);
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc__nm_connectcb(sock, req, eresult, async);
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
isc__nmsocket_prep_destroy(sock);
}
void
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc__nm_failed_read_cb(isc_nmsocket_t *sock, isc_result_t result, bool async) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
REQUIRE(VALID_NMSOCK(sock));
switch (sock->type) {
case isc_nm_udpsocket:
isc__nm_udp_failed_read_cb(sock, result);
return;
case isc_nm_tcpsocket:
isc__nm_tcp_failed_read_cb(sock, result);
return;
case isc_nm_tcpdnssocket:
isc__nm_tcpdns_failed_read_cb(sock, result);
return;
case isc_nm_tlsdnssocket:
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc__nm_tlsdns_failed_read_cb(sock, result, async);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return;
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
}
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
void
isc__nmsocket_connecttimeout_cb(uv_timer_t *timer) {
uv_connect_t *uvreq = uv_handle_get_data((uv_handle_t *)timer);
isc_nmsocket_t *sock = uv_handle_get_data((uv_handle_t *)uvreq->handle);
isc__nm_uvreq_t *req = uv_handle_get_data((uv_handle_t *)uvreq);
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(sock->tid == isc_nm_tid());
REQUIRE(atomic_load(&sock->connecting));
REQUIRE(VALID_UVREQ(req));
REQUIRE(VALID_NMHANDLE(req->handle));
isc__nmsocket_timer_stop(sock);
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
if (sock->tls.pending_req != NULL) {
REQUIRE(req == sock->tls.pending_req);
sock->tls.pending_req = NULL;
}
Delay isc__nm_uvreq_t deallocation to connection callback When the TCP, TCPDNS or TLSDNS connection times out, the isc__nm_uvreq_t would be pushed into sock->inactivereqs before the uv_tcp_connect() callback finishes. Because the isc__nmsocket_t keeps the list of inactive isc__nm_uvreq_t, this would cause use-after-free only when the sock->inactivereqs is full (which could never happen because the failure happens in connection timeout callback) or when the sock->inactivereqs mechanism is completely removed (f.e. when running under Address or Thread Sanitizer). Delay isc__nm_uvreq_t deallocation to the connection callback and only signal the connection callback should be called by shutting down the libuv socket from the connection timeout callback.
2022-02-22 18:12:18 +01:00
/*
* Mark the connection as timed out and shutdown the socket.
*/
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
Delay isc__nm_uvreq_t deallocation to connection callback When the TCP, TCPDNS or TLSDNS connection times out, the isc__nm_uvreq_t would be pushed into sock->inactivereqs before the uv_tcp_connect() callback finishes. Because the isc__nmsocket_t keeps the list of inactive isc__nm_uvreq_t, this would cause use-after-free only when the sock->inactivereqs is full (which could never happen because the failure happens in connection timeout callback) or when the sock->inactivereqs mechanism is completely removed (f.e. when running under Address or Thread Sanitizer). Delay isc__nm_uvreq_t deallocation to the connection callback and only signal the connection callback should be called by shutting down the libuv socket from the connection timeout callback.
2022-02-22 18:12:18 +01:00
INSIST(atomic_compare_exchange_strong(&sock->timedout, &(bool){ false },
true));
isc__nmsocket_clearcb(sock);
isc__nmsocket_shutdown(sock);
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
}
Improve the logging on failed TCP accept Previously, when TCP accept failed, we have logged a message with ISC_LOG_ERROR level. One common case, how this could happen is that the client hits TCP client quota and is put on hold and when resumed, the client has already given up and closed the TCP connection. In such case, the named would log: TCP connection failed: socket is not connected This message was quite confusing because it actually doesn't say that it's related to the accepting the TCP connection and also it logs everything on the ISC_LOG_ERROR level. Change the log message to "Accepting TCP connection failed" and for specific error states lower the severity of the log message to ISC_LOG_INFO.
2021-12-01 17:41:20 +01:00
void
isc__nm_accept_connection_log(isc_result_t result, bool can_log_quota) {
int level;
switch (result) {
case ISC_R_SUCCESS:
case ISC_R_NOCONN:
return;
case ISC_R_QUOTA:
case ISC_R_SOFTQUOTA:
if (!can_log_quota) {
return;
}
level = ISC_LOG_INFO;
break;
case ISC_R_NOTCONNECTED:
level = ISC_LOG_INFO;
break;
default:
level = ISC_LOG_ERROR;
}
isc_log_write(isc_lctx, ISC_LOGCATEGORY_GENERAL, ISC_LOGMODULE_NETMGR,
level, "Accepting TCP connection failed: %s",
isc_result_totext(result));
}
Add TCP, TCPDNS and TLSDNS write timer When the outgoing TCP write buffers are full because the other party is not reading the data, the uv_write() could wait indefinitely on the uv_loop and never calling the callback. Add a new write timer that uses the `tcp-idle-timeout` value to interrupt the TCP connection when we are not able to send data for defined period of time.
2022-02-09 11:21:04 +01:00
void
Change single write timer to per-send timers Previously, there was a single per-socket write timer that would get restarted for every new write. This turned out to be insufficient because the other side could keep reseting the timer, and never reading back the responses. Change the single write timer to per-send timer which would in turn reset the TCP connection on the first send timeout.
2022-03-10 13:51:08 +01:00
isc__nmsocket_writetimeout_cb(void *data, isc_result_t eresult) {
isc__nm_uvreq_t *req = data;
isc_nmsocket_t *sock = NULL;
Add TCP, TCPDNS and TLSDNS write timer When the outgoing TCP write buffers are full because the other party is not reading the data, the uv_write() could wait indefinitely on the uv_loop and never calling the callback. Add a new write timer that uses the `tcp-idle-timeout` value to interrupt the TCP connection when we are not able to send data for defined period of time.
2022-02-09 11:21:04 +01:00
Change single write timer to per-send timers Previously, there was a single per-socket write timer that would get restarted for every new write. This turned out to be insufficient because the other side could keep reseting the timer, and never reading back the responses. Change the single write timer to per-send timer which would in turn reset the TCP connection on the first send timeout.
2022-03-10 13:51:08 +01:00
REQUIRE(eresult == ISC_R_TIMEDOUT);
REQUIRE(VALID_UVREQ(req));
REQUIRE(VALID_NMSOCK(req->sock));
sock = req->sock;
Add TCP, TCPDNS and TLSDNS write timer When the outgoing TCP write buffers are full because the other party is not reading the data, the uv_write() could wait indefinitely on the uv_loop and never calling the callback. Add a new write timer that uses the `tcp-idle-timeout` value to interrupt the TCP connection when we are not able to send data for defined period of time.
2022-02-09 11:21:04 +01:00
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
isc__nmsocket_reset(sock);
Add TCP, TCPDNS and TLSDNS write timer When the outgoing TCP write buffers are full because the other party is not reading the data, the uv_write() could wait indefinitely on the uv_loop and never calling the callback. Add a new write timer that uses the `tcp-idle-timeout` value to interrupt the TCP connection when we are not able to send data for defined period of time.
2022-02-09 11:21:04 +01:00
}
void
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc__nmsocket_readtimeout_cb(uv_timer_t *timer) {
isc_nmsocket_t *sock = uv_handle_get_data((uv_handle_t *)timer);
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(sock->tid == isc_nm_tid());
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
REQUIRE(atomic_load(&sock->reading));
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
if (atomic_load(&sock->client)) {
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
uv_timer_stop(timer);
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
sock->recv_read = false;
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
if (sock->recv_cb != NULL) {
isc__nm_uvreq_t *req = isc__nm_get_read_req(sock, NULL);
isc__nm_readcb(sock, req, ISC_R_TIMEDOUT);
}
if (!isc__nmsocket_timer_running(sock)) {
isc__nmsocket_clearcb(sock);
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc__nm_failed_read_cb(sock, ISC_R_CANCELED, false);
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
}
} else {
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc__nm_failed_read_cb(sock, ISC_R_TIMEDOUT, false);
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
}
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
void
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc__nmsocket_timer_restart(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
if (atomic_load(&sock->connecting)) {
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
int r;
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
if (sock->connect_timeout == 0) {
return;
}
Rename sock->timer to sock->read_timer Before adding the write timer, we have to remove the generic sock->timer to sock->read_timer. We don't touch the function names to limit the impact of the refactoring.
2022-02-09 10:59:08 +01:00
r = uv_timer_start(&sock->read_timer,
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
isc__nmsocket_connecttimeout_cb,
sock->connect_timeout + 10, 0);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_timer_start, r);
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
} else {
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
int r;
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
if (sock->read_timeout == 0) {
return;
}
Rename sock->timer to sock->read_timer Before adding the write timer, we have to remove the generic sock->timer to sock->read_timer. We don't touch the function names to limit the impact of the refactoring.
2022-02-09 10:59:08 +01:00
r = uv_timer_start(&sock->read_timer,
isc__nmsocket_readtimeout_cb,
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
sock->read_timeout, 0);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_timer_start, r);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
}
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
bool
isc__nmsocket_timer_running(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
Rename sock->timer to sock->read_timer Before adding the write timer, we have to remove the generic sock->timer to sock->read_timer. We don't touch the function names to limit the impact of the refactoring.
2022-02-09 10:59:08 +01:00
return (uv_is_active((uv_handle_t *)&sock->read_timer));
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
}
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
void
isc__nmsocket_timer_start(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
if (isc__nmsocket_timer_running(sock)) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return;
}
isc__nmsocket_timer_restart(sock);
}
void
isc__nmsocket_timer_stop(isc_nmsocket_t *sock) {
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
int r;
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
REQUIRE(VALID_NMSOCK(sock));
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
/* uv_timer_stop() is idempotent, no need to check if running */
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
Rename sock->timer to sock->read_timer Before adding the write timer, we have to remove the generic sock->timer to sock->read_timer. We don't touch the function names to limit the impact of the refactoring.
2022-02-09 10:59:08 +01:00
r = uv_timer_stop(&sock->read_timer);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_timer_stop, r);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
isc__nm_uvreq_t *
isc__nm_get_read_req(isc_nmsocket_t *sock, isc_sockaddr_t *sockaddr) {
isc__nm_uvreq_t *req = NULL;
req = isc__nm_uvreq_get(sock->mgr, sock);
req->cb.recv = sock->recv_cb;
req->cbarg = sock->recv_cbarg;
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
switch (sock->type) {
case isc_nm_tcpsocket:
case isc_nm_tlssocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc_nmhandle_attach(sock->statichandle, &req->handle);
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
break;
default:
check statichandle before attaching it is possible for udp_recv_cb() to fire after the socket is already shutting down and statichandle is NULL; we need to create a temporary handle in this case.
2021-10-17 13:30:47 -07:00
if (atomic_load(&sock->client) && sock->statichandle != NULL) {
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
isc_nmhandle_attach(sock->statichandle, &req->handle);
} else {
req->handle = isc__nmhandle_get(sock, sockaddr, NULL);
}
break;
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
return (req);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
/*%<
Fix the UDP recvmmsg support Previously, the netmgr/udp.c tried to detect the recvmmsg detection in libuv with #ifdef UV_UDP_<foo> preprocessor macros. However, because the UV_UDP_<foo> are not preprocessor macros, but enum members, the detection didn't work. Because the detection didn't work, the code didn't have access to the information when we received the final chunk of the recvmmsg and tried to free the uvbuf every time. Fortunately, the isc__nm_free_uvbuf() had a kludge that detected attempt to free in the middle of the receive buffer, so the code worked. However, libuv 1.37.0 changed the way the recvmmsg was enabled from implicit to explicit, and we checked for yet another enum member presence with preprocessor macro, so in fact libuv recvmmsg support was never enabled with libuv >= 1.37.0. This commit changes to the preprocessor macros to autoconf checks for declaration, so the detection now works again. On top of that, it's now possible to cleanup the alloc_cb and free_uvbuf functions because now, the information whether we can or cannot free the buffer is available to us.
2022-01-11 12:14:23 +01:00
* Allocator callback for read operations.
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
*
* Note this doesn't actually allocate anything, it just assigns the
* worker's receive buffer to a socket, and marks it as "in use".
*/
void
isc__nm_alloc_cb(uv_handle_t *handle, size_t size, uv_buf_t *buf) {
isc_nmsocket_t *sock = uv_handle_get_data(handle);
isc__networker_t *worker = NULL;
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(isc__nm_in_netthread());
Fix the UDP recvmmsg support Previously, the netmgr/udp.c tried to detect the recvmmsg detection in libuv with #ifdef UV_UDP_<foo> preprocessor macros. However, because the UV_UDP_<foo> are not preprocessor macros, but enum members, the detection didn't work. Because the detection didn't work, the code didn't have access to the information when we received the final chunk of the recvmmsg and tried to free the uvbuf every time. Fortunately, the isc__nm_free_uvbuf() had a kludge that detected attempt to free in the middle of the receive buffer, so the code worked. However, libuv 1.37.0 changed the way the recvmmsg was enabled from implicit to explicit, and we checked for yet another enum member presence with preprocessor macro, so in fact libuv recvmmsg support was never enabled with libuv >= 1.37.0. This commit changes to the preprocessor macros to autoconf checks for declaration, so the detection now works again. On top of that, it's now possible to cleanup the alloc_cb and free_uvbuf functions because now, the information whether we can or cannot free the buffer is available to us.
2022-01-11 12:14:23 +01:00
/*
* The size provided by libuv is only suggested size, and it always
* defaults to 64 * 1024 in the current versions of libuv (see
* src/unix/udp.c and src/unix/stream.c).
*/
UNUSED(size);
worker = &sock->mgr->workers[sock->tid];
INSIST(!worker->recvbuf_inuse);
INSIST(worker->recvbuf != NULL);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
switch (sock->type) {
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
case isc_nm_udpsocket:
Fix the UDP recvmmsg support Previously, the netmgr/udp.c tried to detect the recvmmsg detection in libuv with #ifdef UV_UDP_<foo> preprocessor macros. However, because the UV_UDP_<foo> are not preprocessor macros, but enum members, the detection didn't work. Because the detection didn't work, the code didn't have access to the information when we received the final chunk of the recvmmsg and tried to free the uvbuf every time. Fortunately, the isc__nm_free_uvbuf() had a kludge that detected attempt to free in the middle of the receive buffer, so the code worked. However, libuv 1.37.0 changed the way the recvmmsg was enabled from implicit to explicit, and we checked for yet another enum member presence with preprocessor macro, so in fact libuv recvmmsg support was never enabled with libuv >= 1.37.0. This commit changes to the preprocessor macros to autoconf checks for declaration, so the detection now works again. On top of that, it's now possible to cleanup the alloc_cb and free_uvbuf functions because now, the information whether we can or cannot free the buffer is available to us.
2022-01-11 12:14:23 +01:00
buf->len = ISC_NETMGR_UDP_RECVBUF_SIZE;
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
break;
Use generic functions for reading and timers in TCP The TCP module has been updated to use the generic functions from netmgr.c instead of its own local copies. This brings the module mostly up to par with the TCPDNS and TLSDNS modules.
2021-03-31 12:14:54 +02:00
case isc_nm_tcpsocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
case isc_nm_tcpdnssocket:
case isc_nm_tlsdnssocket:
Fix the UDP recvmmsg support Previously, the netmgr/udp.c tried to detect the recvmmsg detection in libuv with #ifdef UV_UDP_<foo> preprocessor macros. However, because the UV_UDP_<foo> are not preprocessor macros, but enum members, the detection didn't work. Because the detection didn't work, the code didn't have access to the information when we received the final chunk of the recvmmsg and tried to free the uvbuf every time. Fortunately, the isc__nm_free_uvbuf() had a kludge that detected attempt to free in the middle of the receive buffer, so the code worked. However, libuv 1.37.0 changed the way the recvmmsg was enabled from implicit to explicit, and we checked for yet another enum member presence with preprocessor macro, so in fact libuv recvmmsg support was never enabled with libuv >= 1.37.0. This commit changes to the preprocessor macros to autoconf checks for declaration, so the detection now works again. On top of that, it's now possible to cleanup the alloc_cb and free_uvbuf functions because now, the information whether we can or cannot free the buffer is available to us.
2022-01-11 12:14:23 +01:00
buf->len = ISC_NETMGR_TCP_RECVBUF_SIZE;
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
break;
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
Fix the UDP recvmmsg support Previously, the netmgr/udp.c tried to detect the recvmmsg detection in libuv with #ifdef UV_UDP_<foo> preprocessor macros. However, because the UV_UDP_<foo> are not preprocessor macros, but enum members, the detection didn't work. Because the detection didn't work, the code didn't have access to the information when we received the final chunk of the recvmmsg and tried to free the uvbuf every time. Fortunately, the isc__nm_free_uvbuf() had a kludge that detected attempt to free in the middle of the receive buffer, so the code worked. However, libuv 1.37.0 changed the way the recvmmsg was enabled from implicit to explicit, and we checked for yet another enum member presence with preprocessor macro, so in fact libuv recvmmsg support was never enabled with libuv >= 1.37.0. This commit changes to the preprocessor macros to autoconf checks for declaration, so the detection now works again. On top of that, it's now possible to cleanup the alloc_cb and free_uvbuf functions because now, the information whether we can or cannot free the buffer is available to us.
2022-01-11 12:14:23 +01:00
REQUIRE(buf->len <= ISC_NETMGR_RECVBUF_SIZE);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
buf->base = worker->recvbuf;
Fix the UDP recvmmsg support Previously, the netmgr/udp.c tried to detect the recvmmsg detection in libuv with #ifdef UV_UDP_<foo> preprocessor macros. However, because the UV_UDP_<foo> are not preprocessor macros, but enum members, the detection didn't work. Because the detection didn't work, the code didn't have access to the information when we received the final chunk of the recvmmsg and tried to free the uvbuf every time. Fortunately, the isc__nm_free_uvbuf() had a kludge that detected attempt to free in the middle of the receive buffer, so the code worked. However, libuv 1.37.0 changed the way the recvmmsg was enabled from implicit to explicit, and we checked for yet another enum member presence with preprocessor macro, so in fact libuv recvmmsg support was never enabled with libuv >= 1.37.0. This commit changes to the preprocessor macros to autoconf checks for declaration, so the detection now works again. On top of that, it's now possible to cleanup the alloc_cb and free_uvbuf functions because now, the information whether we can or cannot free the buffer is available to us.
2022-01-11 12:14:23 +01:00
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
worker->recvbuf_inuse = true;
}
void
isc__nm_start_reading(isc_nmsocket_t *sock) {
int r;
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
if (atomic_load(&sock->reading)) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return;
}
switch (sock->type) {
case isc_nm_udpsocket:
r = uv_udp_recv_start(&sock->uv_handle.udp, isc__nm_alloc_cb,
isc__nm_udp_read_cb);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_udp_recv_start, r);
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
break;
Use generic functions for reading and timers in TCP The TCP module has been updated to use the generic functions from netmgr.c instead of its own local copies. This brings the module mostly up to par with the TCPDNS and TLSDNS modules.
2021-03-31 12:14:54 +02:00
case isc_nm_tcpsocket:
r = uv_read_start(&sock->uv_handle.stream, isc__nm_alloc_cb,
isc__nm_tcp_read_cb);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_read_start, r);
Use generic functions for reading and timers in TCP The TCP module has been updated to use the generic functions from netmgr.c instead of its own local copies. This brings the module mostly up to par with the TCPDNS and TLSDNS modules.
2021-03-31 12:14:54 +02:00
break;
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
case isc_nm_tcpdnssocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
r = uv_read_start(&sock->uv_handle.stream, isc__nm_alloc_cb,
isc__nm_tcpdns_read_cb);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_read_start, r);
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnssocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
r = uv_read_start(&sock->uv_handle.stream, isc__nm_alloc_cb,
isc__nm_tlsdns_read_cb);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_read_start, r);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
break;
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
atomic_store(&sock->reading, true);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
void
isc__nm_stop_reading(isc_nmsocket_t *sock) {
int r;
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
if (!atomic_load(&sock->reading)) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return;
}
switch (sock->type) {
case isc_nm_udpsocket:
r = uv_udp_recv_stop(&sock->uv_handle.udp);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_udp_recv_stop, r);
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
break;
Use generic functions for reading and timers in TCP The TCP module has been updated to use the generic functions from netmgr.c instead of its own local copies. This brings the module mostly up to par with the TCPDNS and TLSDNS modules.
2021-03-31 12:14:54 +02:00
case isc_nm_tcpsocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
case isc_nm_tcpdnssocket:
case isc_nm_tlsdnssocket:
r = uv_read_stop(&sock->uv_handle.stream);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_read_stop, r);
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
break;
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
}
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
atomic_store(&sock->reading, false);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
bool
Fix and clean up handling of connect callbacks Serveral problems were discovered and fixed after the change in the connection timeout in the previous commits: * In TLSDNS, the connection callback was not called at all under some circumstances when the TCP connection had been established, but the TLS handshake hadn't been completed yet. Additional checks have been put in place so that tls_cycle() will end early when the nmsocket is invalidated by the isc__nm_tlsdns_shutdown() call. * In TCP, TCPDNS and TLSDNS, new connections would be established even when the network manager was shutting down. The new call isc__nm_closing() has been added and is used to bail out early even before uv_tcp_connect() is attempted.
2021-03-31 11:48:41 +02:00
isc__nm_closing(isc_nmsocket_t *sock) {
return (atomic_load(&sock->mgr->closing));
}
bool
isc__nmsocket_closing(isc_nmsocket_t *sock) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return (!isc__nmsocket_active(sock) || atomic_load(&sock->closing) ||
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
isc__nm_closing(sock) ||
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
(sock->server != NULL && !isc__nmsocket_active(sock->server)));
}
static isc_result_t
processbuffer(isc_nmsocket_t *sock) {
switch (sock->type) {
case isc_nm_tcpdnssocket:
return (isc__nm_tcpdns_processbuffer(sock));
case isc_nm_tlsdnssocket:
Fix typo in processbuffer() - tcpdns vs tlsdns The processbuffer() would call isc__nm_tcpdns_processbuffer() instead of isc__nm_tlsdns_processbuffer() for the isc_nm_tlsdnssocket type of socket.
2021-03-18 21:19:18 +01:00
return (isc__nm_tlsdns_processbuffer(sock));
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
}
/*
* Process a DNS message.
*
* If we only have an incomplete DNS message, we don't touch any
* timers. If we do have a full message, reset the timer.
*
Remove the keep-response-order ACL map The keep-response-order option has been obsoleted, and in this commit, remove the keep-response-order ACL map rendering the option no-op, the call the isc_nm_sequential() and the now unused isc_nm_sequential() function itself.
2022-01-27 10:09:07 +01:00
* Stop reading if this is a client socket. In this case we'll be
* called again later by isc__nm_resume_processing().
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
*/
void
isc__nm_process_sock_buffer(isc_nmsocket_t *sock) {
for (;;) {
int_fast32_t ah = atomic_load(&sock->ah);
isc_result_t result = processbuffer(sock);
switch (result) {
case ISC_R_NOMORE:
/*
* Don't reset the timer until we have a
* full DNS message.
*/
isc__nm_start_reading(sock);
/*
* Start the timer only if there are no externally used
* active handles, there's always one active handle
* attached internally to sock->recv_handle in
* accept_connection()
*/
if (ah == 1) {
isc__nmsocket_timer_start(sock);
}
return;
case ISC_R_CANCELED:
isc__nmsocket_timer_stop(sock);
isc__nm_stop_reading(sock);
return;
case ISC_R_SUCCESS:
/*
* Stop the timer on the successful message read, this
* also allows to restart the timer when we have no more
* data.
*/
isc__nmsocket_timer_stop(sock);
Remove the keep-response-order ACL map The keep-response-order option has been obsoleted, and in this commit, remove the keep-response-order ACL map rendering the option no-op, the call the isc_nm_sequential() and the now unused isc_nm_sequential() function itself.
2022-01-27 10:09:07 +01:00
if (atomic_load(&sock->client)) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc__nm_stop_reading(sock);
return;
}
break;
default:
Consistenly use UNREACHABLE() instead of ISC_UNREACHABLE() In couple places, we have missed INSIST(0) or ISC_UNREACHABLE() replacement on some branches with UNREACHABLE(). Replace all ISC_UNREACHABLE() or INSIST(0) calls with UNREACHABLE().
2022-03-28 12:59:43 +02:00
UNREACHABLE();
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
}
}
void
isc__nm_resume_processing(void *arg) {
isc_nmsocket_t *sock = (isc_nmsocket_t *)arg;
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(sock->tid == isc_nm_tid());
REQUIRE(!atomic_load(&sock->client));
Fix and clean up handling of connect callbacks Serveral problems were discovered and fixed after the change in the connection timeout in the previous commits: * In TLSDNS, the connection callback was not called at all under some circumstances when the TCP connection had been established, but the TLS handshake hadn't been completed yet. Additional checks have been put in place so that tls_cycle() will end early when the nmsocket is invalidated by the isc__nm_tlsdns_shutdown() call. * In TCP, TCPDNS and TLSDNS, new connections would be established even when the network manager was shutting down. The new call isc__nm_closing() has been added and is used to bail out early even before uv_tcp_connect() is attempted.
2021-03-31 11:48:41 +02:00
if (isc__nmsocket_closing(sock)) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return;
}
isc__nm_process_sock_buffer(sock);
}
void
isc_nmhandle_cleartimeout(isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
switch (handle->sock->type) {
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
case isc_nm_httpsocket:
isc__nm_http_cleartimeout(handle);
return;
case isc_nm_tlssocket:
isc__nm_tls_cleartimeout(handle);
return;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
default:
handle->sock->read_timeout = 0;
Rename sock->timer to sock->read_timer Before adding the write timer, we have to remove the generic sock->timer to sock->read_timer. We don't touch the function names to limit the impact of the refactoring.
2022-02-09 10:59:08 +01:00
if (uv_is_active((uv_handle_t *)&handle->sock->read_timer)) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc__nmsocket_timer_stop(handle->sock);
}
}
}
void
isc_nmhandle_settimeout(isc_nmhandle_t *handle, uint32_t timeout) {
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
switch (handle->sock->type) {
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
case isc_nm_httpsocket:
isc__nm_http_settimeout(handle, timeout);
return;
case isc_nm_tlssocket:
isc__nm_tls_settimeout(handle, timeout);
return;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
default:
handle->sock->read_timeout = timeout;
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
isc__nmsocket_timer_restart(handle->sock);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
}
enable keepalive when the keepalive EDNS option is seen previously, receiving a keepalive option had no effect on how long named would keep the connection open; there was a place to configure the keepalive timeout but it was never used. this commit corrects that. this also fixes an error in isc__nm_{tcp,tls}dns_keepalive() in which the sense of a REQUIRE test was reversed; previously this error had not been noticed because the functions were not being used.
2020-11-02 18:33:20 -08:00
void
isc_nmhandle_keepalive(isc_nmhandle_t *handle, bool value) {
replace per-protocol keepalive functions with a common one this commit removes isc__nm_tcpdns_keepalive() and isc__nm_tlsdns_keepalive(); keepalive for these protocols and for TCP will now be set directly from isc_nmhandle_keepalive(). protocols that have an underlying TCP socket (i.e., TLS stream and HTTP), now have protocol-specific routines, called by isc_nmhandle_keeaplive(), to set the keepalive value on the underlying socket.
2021-07-14 21:12:37 -07:00
isc_nmsocket_t *sock = NULL;
enable keepalive when the keepalive EDNS option is seen previously, receiving a keepalive option had no effect on how long named would keep the connection open; there was a place to configure the keepalive timeout but it was never used. this commit corrects that. this also fixes an error in isc__nm_{tcp,tls}dns_keepalive() in which the sense of a REQUIRE test was reversed; previously this error had not been noticed because the functions were not being used.
2020-11-02 18:33:20 -08:00
REQUIRE(VALID_NMHANDLE(handle));
replace per-protocol keepalive functions with a common one this commit removes isc__nm_tcpdns_keepalive() and isc__nm_tlsdns_keepalive(); keepalive for these protocols and for TCP will now be set directly from isc_nmhandle_keepalive(). protocols that have an underlying TCP socket (i.e., TLS stream and HTTP), now have protocol-specific routines, called by isc_nmhandle_keeaplive(), to set the keepalive value on the underlying socket.
2021-07-14 21:12:37 -07:00
REQUIRE(VALID_NMSOCK(handle->sock));
enable keepalive when the keepalive EDNS option is seen previously, receiving a keepalive option had no effect on how long named would keep the connection open; there was a place to configure the keepalive timeout but it was never used. this commit corrects that. this also fixes an error in isc__nm_{tcp,tls}dns_keepalive() in which the sense of a REQUIRE test was reversed; previously this error had not been noticed because the functions were not being used.
2020-11-02 18:33:20 -08:00
replace per-protocol keepalive functions with a common one this commit removes isc__nm_tcpdns_keepalive() and isc__nm_tlsdns_keepalive(); keepalive for these protocols and for TCP will now be set directly from isc_nmhandle_keepalive(). protocols that have an underlying TCP socket (i.e., TLS stream and HTTP), now have protocol-specific routines, called by isc_nmhandle_keeaplive(), to set the keepalive value on the underlying socket.
2021-07-14 21:12:37 -07:00
sock = handle->sock;
switch (sock->type) {
case isc_nm_tcpsocket:
enable keepalive when the keepalive EDNS option is seen previously, receiving a keepalive option had no effect on how long named would keep the connection open; there was a place to configure the keepalive timeout but it was never used. this commit corrects that. this also fixes an error in isc__nm_{tcp,tls}dns_keepalive() in which the sense of a REQUIRE test was reversed; previously this error had not been noticed because the functions were not being used.
2020-11-02 18:33:20 -08:00
case isc_nm_tcpdnssocket:
case isc_nm_tlsdnssocket:
replace per-protocol keepalive functions with a common one this commit removes isc__nm_tcpdns_keepalive() and isc__nm_tlsdns_keepalive(); keepalive for these protocols and for TCP will now be set directly from isc_nmhandle_keepalive(). protocols that have an underlying TCP socket (i.e., TLS stream and HTTP), now have protocol-specific routines, called by isc_nmhandle_keeaplive(), to set the keepalive value on the underlying socket.
2021-07-14 21:12:37 -07:00
atomic_store(&sock->keepalive, value);
sock->read_timeout = value ? atomic_load(&sock->mgr->keepalive)
: atomic_load(&sock->mgr->idle);
Add TCP, TCPDNS and TLSDNS write timer When the outgoing TCP write buffers are full because the other party is not reading the data, the uv_write() could wait indefinitely on the uv_loop and never calling the callback. Add a new write timer that uses the `tcp-idle-timeout` value to interrupt the TCP connection when we are not able to send data for defined period of time.
2022-02-09 11:21:04 +01:00
sock->write_timeout = value ? atomic_load(&sock->mgr->keepalive)
: atomic_load(&sock->mgr->idle);
enable keepalive when the keepalive EDNS option is seen previously, receiving a keepalive option had no effect on how long named would keep the connection open; there was a place to configure the keepalive timeout but it was never used. this commit corrects that. this also fixes an error in isc__nm_{tcp,tls}dns_keepalive() in which the sense of a REQUIRE test was reversed; previously this error had not been noticed because the functions were not being used.
2020-11-02 18:33:20 -08:00
break;
replace per-protocol keepalive functions with a common one this commit removes isc__nm_tcpdns_keepalive() and isc__nm_tlsdns_keepalive(); keepalive for these protocols and for TCP will now be set directly from isc_nmhandle_keepalive(). protocols that have an underlying TCP socket (i.e., TLS stream and HTTP), now have protocol-specific routines, called by isc_nmhandle_keeaplive(), to set the keepalive value on the underlying socket.
2021-07-14 21:12:37 -07:00
#if HAVE_LIBNGHTTP2
case isc_nm_tlssocket:
isc__nmhandle_tls_keepalive(handle, value);
break;
case isc_nm_httpsocket:
isc__nmhandle_http_keepalive(handle, value);
break;
#endif /* HAVE_LIBNGHTTP2 */
enable keepalive when the keepalive EDNS option is seen previously, receiving a keepalive option had no effect on how long named would keep the connection open; there was a place to configure the keepalive timeout but it was never used. this commit corrects that. this also fixes an error in isc__nm_{tcp,tls}dns_keepalive() in which the sense of a REQUIRE test was reversed; previously this error had not been noticed because the functions were not being used.
2020-11-02 18:33:20 -08:00
default:
replace per-protocol keepalive functions with a common one this commit removes isc__nm_tcpdns_keepalive() and isc__nm_tlsdns_keepalive(); keepalive for these protocols and for TCP will now be set directly from isc_nmhandle_keepalive(). protocols that have an underlying TCP socket (i.e., TLS stream and HTTP), now have protocol-specific routines, called by isc_nmhandle_keeaplive(), to set the keepalive value on the underlying socket.
2021-07-14 21:12:37 -07:00
/*
* For any other protocol, this is a no-op.
*/
enable keepalive when the keepalive EDNS option is seen previously, receiving a keepalive option had no effect on how long named would keep the connection open; there was a place to configure the keepalive timeout but it was never used. this commit corrects that. this also fixes an error in isc__nm_{tcp,tls}dns_keepalive() in which the sense of a REQUIRE test was reversed; previously this error had not been noticed because the functions were not being used.
2020-11-02 18:33:20 -08:00
return;
}
}
Rewrite dns_resolver and dns_request to use netmgr timeouts - The `timeout_action` parameter to dns_dispatch_addresponse() been replaced with a netmgr callback that is called when a dispatch read times out. this callback may optionally reset the read timer and resume reading. - Added a function to convert isc_interval to milliseconds; this is used to translate fctx->interval into a value that can be passed to dns_dispatch_addresponse() as the timeout. - Note that netmgr timeouts are accurate to the millisecond, so code to check whether a timeout has been reached cannot rely on microsecond accuracy. - If serve-stale is configured, then a timeout received by the resolver may trigger it to return stale data, and then resume waiting for the read timeout. this is no longer based on a separate stale timer. - The code for canceling requests in request.c has been altered so that it can run asynchronously. - TCP timeout events apply to the dispatch, which may be shared by multiple queries. since in the event of a timeout we have no query ID to use to identify the resp we wanted, we now just send the timeout to the oldest query that was pending. - There was some additional refactoring in the resolver: combining fctx_join() and fctx_try_events() into one function to reduce code duplication, and using fixednames in fetchctx and fetchevent. - Incidental fix: new_adbaddrinfo() can't return NULL anymore, so the code can be simplified.
2021-04-22 18:58:01 -07:00
bool
isc_nmhandle_timer_running(isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
return (isc__nmsocket_timer_running(handle->sock));
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_sockaddr_t
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_peeraddr(isc_nmhandle_t *handle) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
return (handle->peer);
}
isc_sockaddr_t
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_localaddr(isc_nmhandle_t *handle) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
return (handle->local);
}
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
isc_nm_t *
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_netmgr(isc_nmhandle_t *handle) {
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
return (handle->sock->mgr);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_uvreq_t *
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nm_uvreq_get(isc_nm_t *mgr, isc_nmsocket_t *sock FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_uvreq_t *req = NULL;
REQUIRE(VALID_NM(mgr));
REQUIRE(VALID_NMSOCK(sock));
Fix the data race when read-writing sock->active by using cmpxchg
2020-10-22 10:07:56 +02:00
if (sock != NULL && isc__nmsocket_active(sock)) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/* Try to reuse one */
req = isc_astack_pop(sock->inactivereqs);
}
if (req == NULL) {
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
req = isc_mem_get(mgr->mctx, sizeof(*req));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*req = (isc__nm_uvreq_t){ .magic = 0 };
netmgr: server-side TLS support Add server-side TLS support to netmgr - that includes moving some of the isc_nm_ functions from tcp.c to a wrapper in netmgr.c calling a proper tcp or tls function, and a new isc_nm_listentls() function. Add DoT support to tcpdns - isc_nm_listentlsdns().
2020-05-13 17:37:51 +02:00
ISC_LINK_INIT(req, link);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
req->uv_req.req.data = req;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_attach(sock, &req->sock FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
req->magic = UVREQ_MAGIC;
return (req);
}
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nm_uvreq_put(isc__nm_uvreq_t **req0, isc_nmsocket_t *sock FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_uvreq_t *req = NULL;
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_t *handle = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(req0 != NULL);
REQUIRE(VALID_UVREQ(*req0));
req = *req0;
*req0 = NULL;
INSIST(sock == req->sock);
req->magic = 0;
/*
* We need to save this first to make sure that handle,
* sock, and the netmgr won't all disappear.
*/
handle = req->handle;
req->handle = NULL;
Disable inactive uvreqs caching when compiled with sanitizers When isc__nm_uvreq_t gets deactivated, it could be just put onto array stack to be reused later to save some initialization time. Unfortunately, this might hide some use-after-free errors. Disable the inactive uvreqs caching when compiled with Address or Thread Sanitizer.
2022-02-24 00:14:26 +01:00
#if !__SANITIZE_ADDRESS__ && !__SANITIZE_THREAD__
Fix the data race when read-writing sock->active by using cmpxchg
2020-10-22 10:07:56 +02:00
if (!isc__nmsocket_active(sock) ||
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
!isc_astack_trypush(sock->inactivereqs, req)) {
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
isc_mem_put(sock->mgr->mctx, req, sizeof(*req));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Disable inactive uvreqs caching when compiled with sanitizers When isc__nm_uvreq_t gets deactivated, it could be just put onto array stack to be reused later to save some initialization time. Unfortunately, this might hide some use-after-free errors. Disable the inactive uvreqs caching when compiled with Address or Thread Sanitizer.
2022-02-24 00:14:26 +01:00
#else /* !__SANITIZE_ADDRESS__ && !__SANITIZE_THREAD__ */
isc_mem_put(sock->mgr->mctx, req, sizeof(*req));
#endif /* !__SANITIZE_ADDRESS__ && !__SANITIZE_THREAD__ */
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (handle != NULL) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nmhandle_detach(&handle FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_detach(&sock FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
void
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
isc_nm_send(isc_nmhandle_t *handle, isc_region_t *region, isc_nm_cb_t cb,
apply the modified style
2020-02-13 14:44:37 -08:00
void *cbarg) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
switch (handle->sock->type) {
case isc_nm_udpsocket:
case isc_nm_udplistener:
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc__nm_udp_send(handle, region, cb, cbarg);
break;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
case isc_nm_tcpsocket:
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc__nm_tcp_send(handle, region, cb, cbarg);
break;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
case isc_nm_tcpdnssocket:
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc__nm_tcpdns_send(handle, region, cb, cbarg);
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnssocket:
isc__nm_tlsdns_send(handle, region, cb, cbarg);
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
case isc_nm_tlssocket:
isc__nm_tls_send(handle, region, cb, cbarg);
break;
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
case isc_nm_httpsocket:
nghttp2-based HTTP layer in netmgr This commit includes work-in-progress implementation of DNS-over-HTTP(S). Server-side code remains mostly untested, and there is only support for POST requests.
2020-10-31 20:42:18 +01:00
isc__nm_http_send(handle, region, cb, cbarg);
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
}
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
void
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
isc_nm_read(isc_nmhandle_t *handle, isc_nm_recv_cb_t cb, void *cbarg) {
REQUIRE(VALID_NMHANDLE(handle));
switch (handle->sock->type) {
add netmgr functions to support outgoing DNS queries - isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection. - isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a timeout argument to ensure connections time out and are correctly cleaned up on failure. - isc_nm_read() now supports UDP; it reads a single datagram and then stops until the next time it's called. - isc_nm_cancelread() now runs asynchronously to prevent assertion failure if reading is interrupted by a non-network thread (e.g. a timeout). - isc_nm_cancelread() can now apply to UDP sockets. - added shim code to support UDP connection in versions of libuv prior to 1.27, when uv_udp_connect() was added all these functions will be used to support outgoing queries in dig, xfrin, dispatch, etc.
2020-09-05 11:07:40 -07:00
case isc_nm_udpsocket:
isc__nm_udp_read(handle, cb, cbarg);
break;
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
case isc_nm_tcpsocket:
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc__nm_tcp_read(handle, cb, cbarg);
break;
add netmgr functions to support outgoing DNS queries - isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection. - isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a timeout argument to ensure connections time out and are correctly cleaned up on failure. - isc_nm_read() now supports UDP; it reads a single datagram and then stops until the next time it's called. - isc_nm_cancelread() now runs asynchronously to prevent assertion failure if reading is interrupted by a non-network thread (e.g. a timeout). - isc_nm_cancelread() can now apply to UDP sockets. - added shim code to support UDP connection in versions of libuv prior to 1.27, when uv_udp_connect() was added all these functions will be used to support outgoing queries in dig, xfrin, dispatch, etc.
2020-09-05 11:07:40 -07:00
case isc_nm_tcpdnssocket:
isc__nm_tcpdns_read(handle, cb, cbarg);
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnssocket:
isc__nm_tlsdns_read(handle, cb, cbarg);
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
case isc_nm_tlssocket:
isc__nm_tls_read(handle, cb, cbarg);
break;
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
case isc_nm_httpsocket:
isc__nm_http_read(handle, cb, cbarg);
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
}
}
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
void
isc_nm_cancelread(isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
switch (handle->sock->type) {
add netmgr functions to support outgoing DNS queries - isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection. - isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a timeout argument to ensure connections time out and are correctly cleaned up on failure. - isc_nm_read() now supports UDP; it reads a single datagram and then stops until the next time it's called. - isc_nm_cancelread() now runs asynchronously to prevent assertion failure if reading is interrupted by a non-network thread (e.g. a timeout). - isc_nm_cancelread() can now apply to UDP sockets. - added shim code to support UDP connection in versions of libuv prior to 1.27, when uv_udp_connect() was added all these functions will be used to support outgoing queries in dig, xfrin, dispatch, etc.
2020-09-05 11:07:40 -07:00
case isc_nm_udpsocket:
isc__nm_udp_cancelread(handle);
break;
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
case isc_nm_tcpsocket:
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
isc__nm_tcp_cancelread(handle);
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
break;
add netmgr functions to support outgoing DNS queries - isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection. - isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a timeout argument to ensure connections time out and are correctly cleaned up on failure. - isc_nm_read() now supports UDP; it reads a single datagram and then stops until the next time it's called. - isc_nm_cancelread() now runs asynchronously to prevent assertion failure if reading is interrupted by a non-network thread (e.g. a timeout). - isc_nm_cancelread() can now apply to UDP sockets. - added shim code to support UDP connection in versions of libuv prior to 1.27, when uv_udp_connect() was added all these functions will be used to support outgoing queries in dig, xfrin, dispatch, etc.
2020-09-05 11:07:40 -07:00
case isc_nm_tcpdnssocket:
isc__nm_tcpdns_cancelread(handle);
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnssocket:
isc__nm_tlsdns_cancelread(handle);
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
isc__nm_tls_cancelread(handle);
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
}
}
Fix the data race when read-writing sock->active by using cmpxchg
2020-10-22 10:07:56 +02:00
void
use handles for isc_nm_pauseread() and isc_nm_resumeread() by having these functions act on netmgr handles instead of socket objects, they can be used in callback functions outside the netgmr.
2020-07-01 16:17:09 -07:00
isc_nm_pauseread(isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
isc_nmsocket_t *sock = handle->sock;
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
switch (sock->type) {
case isc_nm_tcpsocket:
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_tcp_pauseread(handle);
Fix the data race when read-writing sock->active by using cmpxchg
2020-10-22 10:07:56 +02:00
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
isc__nm_tls_pauseread(handle);
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
}
}
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
void
use handles for isc_nm_pauseread() and isc_nm_resumeread() by having these functions act on netmgr handles instead of socket objects, they can be used in callback functions outside the netgmr.
2020-07-01 16:17:09 -07:00
isc_nm_resumeread(isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
isc_nmsocket_t *sock = handle->sock;
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
switch (sock->type) {
case isc_nm_tcpsocket:
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_tcp_resumeread(handle);
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
isc__nm_tls_resumeread(handle);
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
}
}
void
isc_nm_stoplistening(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
switch (sock->type) {
case isc_nm_udplistener:
isc__nm_udp_stoplistening(sock);
break;
case isc_nm_tcpdnslistener:
isc__nm_tcpdns_stoplistening(sock);
break;
case isc_nm_tcplistener:
isc__nm_tcp_stoplistening(sock);
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnslistener:
isc__nm_tlsdns_stoplistening(sock);
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#if HAVE_LIBNGHTTP2
case isc_nm_tlslistener:
isc__nm_tls_stoplistening(sock);
break;
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
case isc_nm_httplistener:
isc__nm_http_stoplistening(sock);
break;
Make the DNS over HTTPS support optional This commit adds two new autoconf options `--enable-doh` (enabled by default) and `--with-libnghttp2` (mandatory when DoH is enabled). When DoH support is disabled the library is not linked-in and support for http(s) protocol is disabled in the netmgr, named and dig.
2021-04-21 13:52:15 +02:00
#endif
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
}
}
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
void
isc__nm_connectcb(isc_nmsocket_t *sock, isc__nm_uvreq_t *uvreq,
isc_result_t eresult, bool async) {
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
if (!async) {
isc__netievent_connectcb_t ievent = { .sock = sock,
.req = uvreq,
.result = eresult };
isc__nm_async_connectcb(NULL, (isc__netievent_t *)&ievent);
isc_nm_*connect() always return via callback The isc_nm_*connect() functions were refactored to always return the connection status via the connect callback instead of sometimes returning the hard failure directly (for example, when the socket could not be created, or when the network manager was shutting down). This commit changes the connect functions in all the network manager modules, and also makes the necessary refactoring changes in places where the connect functions are called.
2021-03-31 18:32:32 +02:00
} else {
isc__netievent_connectcb_t *ievent =
isc__nm_get_netievent_connectcb(sock->mgr, sock, uvreq,
eresult);
isc__nm_enqueue_ievent(&sock->mgr->workers[sock->tid],
(isc__netievent_t *)ievent);
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
}
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
}
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
void
isc__nm_async_connectcb(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_connectcb_t *ievent = (isc__netievent_connectcb_t *)ev0;
isc_nmsocket_t *sock = ievent->sock;
isc__nm_uvreq_t *uvreq = ievent->req;
isc_result_t eresult = ievent->result;
UNUSED(worker);
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
REQUIRE(ievent->sock->tid == isc_nm_tid());
REQUIRE(uvreq->cb.connect != NULL);
uvreq->cb.connect(uvreq->handle, eresult, uvreq->cbarg);
isc__nm_uvreq_put(&uvreq, sock);
}
void
isc__nm_readcb(isc_nmsocket_t *sock, isc__nm_uvreq_t *uvreq,
isc_result_t eresult) {
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
run read callbacks synchronously on timeout when running read callbacks, if the event result is not ISC_R_SUCCESS, the callback is always run asynchronously. this is a problem on timeout, because there's no chance to reset the timer before the socket has already been destroyed. this commit allows read callbacks to run synchronously for both ISC_R_SUCCESS and ISC_R_TIMEDOUT result codes.
2021-04-20 23:27:51 -07:00
if (eresult == ISC_R_SUCCESS || eresult == ISC_R_TIMEDOUT) {
Avoid netievent allocations when the callbacks can be called directly After turning the users callbacks to be asynchronous, there was a visible performance drop. This commit prevents the unnecessary allocations while keeping the code paths same for both asynchronous and synchronous calls. The same change was done to the isc__nm_udp_{read,send} as those two functions are in the hot path.
2020-12-02 08:54:51 +01:00
isc__netievent_readcb_t ievent = { .sock = sock,
.req = uvreq,
.result = eresult };
isc__nm_async_readcb(NULL, (isc__netievent_t *)&ievent);
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
} else {
Avoid netievent allocations when the callbacks can be called directly After turning the users callbacks to be asynchronous, there was a visible performance drop. This commit prevents the unnecessary allocations while keeping the code paths same for both asynchronous and synchronous calls. The same change was done to the isc__nm_udp_{read,send} as those two functions are in the hot path.
2020-12-02 08:54:51 +01:00
isc__netievent_readcb_t *ievent = isc__nm_get_netievent_readcb(
sock->mgr, sock, uvreq, eresult);
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
isc__nm_enqueue_ievent(&sock->mgr->workers[sock->tid],
(isc__netievent_t *)ievent);
}
}
void
isc__nm_async_readcb(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_readcb_t *ievent = (isc__netievent_readcb_t *)ev0;
isc_nmsocket_t *sock = ievent->sock;
isc__nm_uvreq_t *uvreq = ievent->req;
isc_result_t eresult = ievent->result;
Address use before NULL check warning of uvreq move dereference of uvreq until the after NULL check.
2021-09-28 10:08:26 +10:00
isc_region_t region;
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
UNUSED(worker);
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
REQUIRE(sock->tid == isc_nm_tid());
Address use before NULL check warning of uvreq move dereference of uvreq until the after NULL check.
2021-09-28 10:08:26 +10:00
region.base = (unsigned char *)uvreq->uvbuf.base;
region.length = uvreq->uvbuf.len;
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
uvreq->cb.recv(uvreq->handle, eresult, &region, uvreq->cbarg);
isc__nm_uvreq_put(&uvreq, sock);
}
void
isc__nm_sendcb(isc_nmsocket_t *sock, isc__nm_uvreq_t *uvreq,
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc_result_t eresult, bool async) {
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
if (!async) {
Avoid netievent allocations when the callbacks can be called directly After turning the users callbacks to be asynchronous, there was a visible performance drop. This commit prevents the unnecessary allocations while keeping the code paths same for both asynchronous and synchronous calls. The same change was done to the isc__nm_udp_{read,send} as those two functions are in the hot path.
2020-12-02 08:54:51 +01:00
isc__netievent_sendcb_t ievent = { .sock = sock,
.req = uvreq,
.result = eresult };
isc__nm_async_sendcb(NULL, (isc__netievent_t *)&ievent);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return;
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
}
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc__netievent_sendcb_t *ievent =
isc__nm_get_netievent_sendcb(sock->mgr, sock, uvreq, eresult);
isc__nm_enqueue_ievent(&sock->mgr->workers[sock->tid],
(isc__netievent_t *)ievent);
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
}
void
isc__nm_async_sendcb(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_sendcb_t *ievent = (isc__netievent_sendcb_t *)ev0;
isc_nmsocket_t *sock = ievent->sock;
isc__nm_uvreq_t *uvreq = ievent->req;
isc_result_t eresult = ievent->result;
UNUSED(worker);
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
REQUIRE(sock->tid == isc_nm_tid());
uvreq->cb.send(uvreq->handle, eresult, uvreq->cbarg);
isc__nm_uvreq_put(&uvreq, sock);
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
static void
isc__nm_async_close(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_close_t *ievent = (isc__netievent_close_t *)ev0;
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
isc_nmsocket_t *sock = ievent->sock;
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
REQUIRE(VALID_NMSOCK(ievent->sock));
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
REQUIRE(sock->tid == isc_nm_tid());
REQUIRE(sock->closehandle_cb != NULL);
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
UNUSED(worker);
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
ievent->sock->closehandle_cb(sock);
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
}
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
void
isc__nm_async_detach(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_detach_t *ievent = (isc__netievent_detach_t *)ev0;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
FLARG_IEVENT(ievent);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
REQUIRE(VALID_NMSOCK(ievent->sock));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
REQUIRE(VALID_NMHANDLE(ievent->handle));
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
REQUIRE(ievent->sock->tid == isc_nm_tid());
UNUSED(worker);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmhandle_detach_cb(&ievent->handle FLARG_PASS);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
}
Change single write timer to per-send timers Previously, there was a single per-socket write timer that would get restarted for every new write. This turned out to be insufficient because the other side could keep reseting the timer, and never reading back the responses. Change the single write timer to per-send timer which would in turn reset the TCP connection on the first send timeout.
2022-03-10 13:51:08 +01:00
static void
reset_shutdown(uv_handle_t *handle) {
isc_nmsocket_t *sock = uv_handle_get_data(handle);
isc__nmsocket_shutdown(sock);
On shutdown, reset the established TCP connections Previously, the established TCP connections (both client and server) would be gracefully closed waiting for the write timeout. Don't wait for TCP connections to gracefully shutdown, but directly reset them for faster shutdown.
2022-03-10 13:58:58 +01:00
isc__nmsocket_detach(&sock);
Change single write timer to per-send timers Previously, there was a single per-socket write timer that would get restarted for every new write. This turned out to be insufficient because the other side could keep reseting the timer, and never reading back the responses. Change the single write timer to per-send timer which would in turn reset the TCP connection on the first send timeout.
2022-03-10 13:51:08 +01:00
}
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
void
isc__nmsocket_reset(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
switch (sock->type) {
Handle TCP sockets in isc__nmsocket_reset() The isc__nmsocket_reset() was missing a case for raw TCP sockets (used by RNDC and DoH) which would case a assertion failure when write timeout would be triggered. TCP sockets are now also properly handled in isc__nmsocket_reset().
2022-02-28 10:25:06 +01:00
case isc_nm_tcpsocket:
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
case isc_nm_tcpdnssocket:
case isc_nm_tlsdnssocket:
Handle TCP sockets in isc__nmsocket_reset() The isc__nmsocket_reset() was missing a case for raw TCP sockets (used by RNDC and DoH) which would case a assertion failure when write timeout would be triggered. TCP sockets are now also properly handled in isc__nmsocket_reset().
2022-02-28 10:25:06 +01:00
/*
* This can be called from the TCP write timeout, or
* from the TCPDNS or TLSDNS branches of isc_nm_bad_request().
*/
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
REQUIRE(sock->parent == NULL);
break;
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
break;
}
On shutdown, reset the established TCP connections Previously, the established TCP connections (both client and server) would be gracefully closed waiting for the write timeout. Don't wait for TCP connections to gracefully shutdown, but directly reset them for faster shutdown.
2022-03-10 13:58:58 +01:00
if (!uv_is_closing(&sock->uv_handle.handle) &&
uv_is_active(&sock->uv_handle.handle))
{
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
/*
* The real shutdown will be handled in the respective
* close functions.
*/
On shutdown, reset the established TCP connections Previously, the established TCP connections (both client and server) would be gracefully closed waiting for the write timeout. Don't wait for TCP connections to gracefully shutdown, but directly reset them for faster shutdown.
2022-03-10 13:58:58 +01:00
isc__nmsocket_attach(sock, &(isc_nmsocket_t *){ NULL });
Change single write timer to per-send timers Previously, there was a single per-socket write timer that would get restarted for every new write. This turned out to be insufficient because the other side could keep reseting the timer, and never reading back the responses. Change the single write timer to per-send timer which would in turn reset the TCP connection on the first send timeout.
2022-03-10 13:51:08 +01:00
int r = uv_tcp_close_reset(&sock->uv_handle.tcp,
reset_shutdown);
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
UV_RUNTIME_CHECK(uv_tcp_close_reset, r);
On shutdown, reset the established TCP connections Previously, the established TCP connections (both client and server) would be gracefully closed waiting for the write timeout. Don't wait for TCP connections to gracefully shutdown, but directly reset them for faster shutdown.
2022-03-10 13:58:58 +01:00
} else {
isc__nmsocket_shutdown(sock);
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
}
}
Fix and clean up handling of connect callbacks Serveral problems were discovered and fixed after the change in the connection timeout in the previous commits: * In TLSDNS, the connection callback was not called at all under some circumstances when the TCP connection had been established, but the TLS handshake hadn't been completed yet. Additional checks have been put in place so that tls_cycle() will end early when the nmsocket is invalidated by the isc__nm_tlsdns_shutdown() call. * In TCP, TCPDNS and TLSDNS, new connections would be established even when the network manager was shutting down. The new call isc__nm_closing() has been added and is used to bail out early even before uv_tcp_connect() is attempted.
2021-03-31 11:48:41 +02:00
void
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
isc__nmsocket_shutdown(isc_nmsocket_t *sock) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
REQUIRE(VALID_NMSOCK(sock));
switch (sock->type) {
case isc_nm_udpsocket:
Fix netmgr read/connect timeout issues - don't bother closing sockets that are already closing. - UDP read timeout timer was not stopped after reading. - improve handling of TCP connection failures.
2020-10-26 14:19:37 +01:00
isc__nm_udp_shutdown(sock);
Add isc__nm_udp_shutdown() function This function will be called during isc_nm_closedown() to ensure that all UDP sockets are closed and detached.
2020-10-26 12:30:54 +01:00
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tcpsocket:
Fix netmgr read/connect timeout issues - don't bother closing sockets that are already closing. - UDP read timeout timer was not stopped after reading. - improve handling of TCP connection failures.
2020-10-26 14:19:37 +01:00
isc__nm_tcp_shutdown(sock);
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tcpdnssocket:
isc__nm_tcpdns_shutdown(sock);
break;
case isc_nm_tlsdnssocket:
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
isc__nm_tlsdns_shutdown(sock);
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_udplistener:
case isc_nm_tcplistener:
case isc_nm_tcpdnslistener:
case isc_nm_tlsdnslistener:
return;
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
}
}
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
static void
shutdown_walk_cb(uv_handle_t *handle, void *arg) {
isc_nmsocket_t *sock = uv_handle_get_data(handle);
UNUSED(arg);
if (uv_is_closing(handle)) {
return;
}
switch (handle->type) {
case UV_UDP:
On shutdown, reset the established TCP connections Previously, the established TCP connections (both client and server) would be gracefully closed waiting for the write timeout. Don't wait for TCP connections to gracefully shutdown, but directly reset them for faster shutdown.
2022-03-10 13:58:58 +01:00
isc__nmsocket_shutdown(sock);
return;
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
case UV_TCP:
On shutdown, reset the established TCP connections Previously, the established TCP connections (both client and server) would be gracefully closed waiting for the write timeout. Don't wait for TCP connections to gracefully shutdown, but directly reset them for faster shutdown.
2022-03-10 13:58:58 +01:00
switch (sock->type) {
case isc_nm_tcpsocket:
case isc_nm_tcpdnssocket:
case isc_nm_tlsdnssocket:
if (sock->parent == NULL) {
/* Reset the TCP connections on shutdown */
isc__nmsocket_reset(sock);
return;
}
Add FALLTHROUGH macro for __attribute__((fallthrough)) Gcc 7+ and Clang 10+ have implemented __attribute__((fallthrough)) which is explicit version of the /* FALLTHROUGH */ comment we are currently using. Add and apply FALLTHROUGH macro that uses the attribute if available, but does nothing on older compilers. In one case (lib/dns/zone.c), using the macro revealed that we were using the /* FALLTHROUGH */ comment in wrong place, remove that comment.
2021-10-11 12:09:16 +02:00
FALLTHROUGH;
On shutdown, reset the established TCP connections Previously, the established TCP connections (both client and server) would be gracefully closed waiting for the write timeout. Don't wait for TCP connections to gracefully shutdown, but directly reset them for faster shutdown.
2022-03-10 13:58:58 +01:00
default:
isc__nmsocket_shutdown(sock);
}
return;
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
default:
return;
}
}
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_async_shutdown(isc__networker_t *worker, isc__netievent_t *ev0) {
shorten some names reduce line breaks and general unwieldiness by changing some function, type, and parameter names.
2019-12-09 12:24:46 -08:00
UNUSED(ev0);
Replace netievent lock-free queue with simple locked queue The current implementation of isc_queue uses Michael-Scott lock-free queue that in turn uses hazard pointers. It was discovered that the way we use the isc_queue, such complicated mechanism isn't really needed, because most of the time, we either execute the work directly when on nmthread (in case of UDP) or schedule the work from the matching nmthreads. Replace the current implementation of the isc_queue with a simple locked ISC_LIST. There's a slight improvement - since copying the whole list is very lightweight - we move the queue into a new list before we start the processing and locking just for moving the queue and not for every single item on the list. NOTE: There's a room for future improvements - since we don't guarantee the order in which the netievents are processed, we could have two lists - one unlocked that would be used when scheduling the work from the matching thread and one locked that would be used from non-matching thread.
2022-02-22 23:40:39 +01:00
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
uv_walk(&worker->loop, shutdown_walk_cb, NULL);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
bool
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_acquire_interlocked(isc_nm_t *mgr) {
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
if (!isc__nm_in_netthread()) {
return (false);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
LOCK(&mgr->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
bool success = atomic_compare_exchange_strong(
&mgr->interlocked, &(int){ ISC_NETMGR_NON_INTERLOCKED },
isc_nm_tid());
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
UNLOCK(&mgr->lock);
return (success);
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_drop_interlocked(isc_nm_t *mgr) {
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
if (!isc__nm_in_netthread()) {
return;
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
LOCK(&mgr->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
int tid = atomic_exchange(&mgr->interlocked,
ISC_NETMGR_NON_INTERLOCKED);
INSIST(tid != ISC_NETMGR_NON_INTERLOCKED);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
BROADCAST(&mgr->wkstatecond);
UNLOCK(&mgr->lock);
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_acquire_interlocked_force(isc_nm_t *mgr) {
fix shutdown deadlocks - ensure isc_nm_pause() and isc_nm_resume() work the same whether run from inside or outside of the netmgr. - promote 'stop' events to the priority event level so they can run while the netmgr is pausing or paused. - when pausing, drain the priority queue before acquiring an interlock; this prevents a deadlock when another thread is waiting for us to complete a task. - release interlock after pausing, reacquire it when resuming, so that stop events can happen. some incidental changes: - use a function to enqueue pause and resume events (this was part of a different change attempt that didn't work out; I kept it because I thought was more readable). - make mgr->nworkers a signed int to remove some annoying integer casts.
2021-05-05 14:54:53 -07:00
if (!isc__nm_in_netthread()) {
return;
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
LOCK(&mgr->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
while (!atomic_compare_exchange_strong(
&mgr->interlocked, &(int){ ISC_NETMGR_NON_INTERLOCKED },
isc_nm_tid()))
apply the modified style
2020-02-13 14:44:37 -08:00
{
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
WAIT(&mgr->wkstatecond, &mgr->lock);
}
UNLOCK(&mgr->lock);
}
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_setstats(isc_nm_t *mgr, isc_stats_t *stats) {
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
REQUIRE(VALID_NM(mgr));
REQUIRE(mgr->stats == NULL);
REQUIRE(isc_stats_ncounters(stats) == isc_sockstatscounter_max);
isc_stats_attach(stats, &mgr->stats);
}
void
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
isc__nm_incstats(isc_nmsocket_t *sock, isc__nm_statid_t id) {
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(id < STATID_MAX);
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
if (sock->statsindex != NULL && sock->mgr->stats != NULL) {
isc_stats_increment(sock->mgr->stats, sock->statsindex[id]);
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
}
}
void
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
isc__nm_decstats(isc_nmsocket_t *sock, isc__nm_statid_t id) {
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(id < STATID_MAX);
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
netmgr: refactor isc__nm_incstats() and isc__nm_decstats() After support for route/netlink sockets is merged, not all sockets will have stats counters associated with them, so it's now necessary to check whether socket stats exist before incrementing or decrementing them. rather than relying on the caller for this, we now just pass the socket and an index, and the correct stats counter will be updated if it exists.
2021-10-02 16:26:43 -07:00
if (sock->statsindex != NULL && sock->mgr->stats != NULL) {
isc_stats_decrement(sock->mgr->stats, sock->statsindex[id]);
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
}
}
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
netmgr: Don't crash if socket() returns an error in udpconnect socket() call can return an error - e.g. EMFILE, so we need to handle this nicely and not crash. Additionally wrap the socket() call inside a platform independent helper function as the Socket data type on Windows is unsigned integer: > This means, for example, that checking for errors when the socket and > accept functions return should not be done by comparing the return > value with –1, or seeing if the value is negative (both common and > legal approaches in UNIX). Instead, an application should use the > manifest constant INVALID_SOCKET as defined in the Winsock2.h header > file.
2020-11-07 20:48:37 +01:00
isc_result_t
isc__nm_socket(int domain, int type, int protocol, uv_os_sock_t *sockp) {
int sock = socket(domain, type, protocol);
if (sock < 0) {
return (isc_errno_toresult(errno));
}
Completely remove BIND 9 Windows support The Windows support has been completely removed from the source tree and BIND 9 now no longer supports native compilation on Windows. We might consider reviewing mingw-w64 port if contributed by external party, but no development efforts will be put into making BIND 9 compile and run on Windows again.
2021-05-20 15:53:50 +02:00
netmgr: Don't crash if socket() returns an error in udpconnect socket() call can return an error - e.g. EMFILE, so we need to handle this nicely and not crash. Additionally wrap the socket() call inside a platform independent helper function as the Socket data type on Windows is unsigned integer: > This means, for example, that checking for errors when the socket and > accept functions return should not be done by comparing the return > value with –1, or seeing if the value is negative (both common and > legal approaches in UNIX). Instead, an application should use the > manifest constant INVALID_SOCKET as defined in the Winsock2.h header > file.
2020-11-07 20:48:37 +01:00
*sockp = (uv_os_sock_t)sock;
return (ISC_R_SUCCESS);
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
void
isc__nm_closesocket(uv_os_sock_t sock) {
close(sock);
}
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#define setsockopt_on(socket, level, name) \
setsockopt(socket, level, name, &(int){ 1 }, sizeof(int))
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
#define setsockopt_off(socket, level, name) \
Fix copy&paste error in setsockopt_off Because of copy&paste error the setsockopt_off macro would enable the socket option instead of disabling it.
2021-06-02 17:41:17 +02:00
setsockopt(socket, level, name, &(int){ 0 }, sizeof(int))
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
isc_result_t
Use uv_os_sock_t instead of uv_os_fd_t for sockets On POSIX based systems both uv_os_sock_t and uv_os_fd_t are both typedef to int. That's not true on Windows, where uv_os_sock_t is SOCKET and uv_os_fd_t is HANDLE and they differ in level of indirection.
2020-10-05 12:25:19 +02:00
isc__nm_socket_freebind(uv_os_sock_t fd, sa_family_t sa_family) {
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
/*
* Set the IP_FREEBIND (or equivalent option) on the uv_handle.
*/
#ifdef IP_FREEBIND
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
UNUSED(sa_family);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
if (setsockopt_on(fd, IPPROTO_IP, IP_FREEBIND) == -1) {
return (ISC_R_FAILURE);
}
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
return (ISC_R_SUCCESS);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#elif defined(IP_BINDANY) || defined(IPV6_BINDANY)
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
if (sa_family == AF_INET) {
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#if defined(IP_BINDANY)
if (setsockopt_on(fd, IPPROTO_IP, IP_BINDANY) == -1) {
return (ISC_R_FAILURE);
}
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
return (ISC_R_SUCCESS);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#endif
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
} else if (sa_family == AF_INET6) {
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#if defined(IPV6_BINDANY)
if (setsockopt_on(fd, IPPROTO_IPV6, IPV6_BINDANY) == -1) {
return (ISC_R_FAILURE);
}
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
return (ISC_R_SUCCESS);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#endif
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
}
return (ISC_R_NOTIMPLEMENTED);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#elif defined(SO_BINDANY)
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
UNUSED(sa_family);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
if (setsockopt_on(fd, SOL_SOCKET, SO_BINDANY) == -1) {
return (ISC_R_FAILURE);
}
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
return (ISC_R_SUCCESS);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#else
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
UNUSED(fd);
UNUSED(sa_family);
return (ISC_R_NOTIMPLEMENTED);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#endif
}
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
isc_result_t
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
isc__nm_socket_reuse(uv_os_sock_t fd) {
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
/*
* Generally, the SO_REUSEADDR socket option allows reuse of
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
* local addresses.
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
*
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
* On the BSDs, SO_REUSEPORT implies SO_REUSEADDR but with some
* additional refinements for programs that use multicast.
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
*
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
* On Linux, SO_REUSEPORT has different semantics: it _shares_ the port
* rather than steal it from the current listener, so we don't use it
* here, but rather in isc__nm_socket_reuse_lb().
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
*
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
* On Windows, it also allows a socket to forcibly bind to a port in use
* by another socket.
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
*/
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
#if defined(SO_REUSEPORT) && !defined(__linux__)
if (setsockopt_on(fd, SOL_SOCKET, SO_REUSEPORT) == -1) {
return (ISC_R_FAILURE);
}
return (ISC_R_SUCCESS);
#elif defined(SO_REUSEADDR)
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
if (setsockopt_on(fd, SOL_SOCKET, SO_REUSEADDR) == -1) {
return (ISC_R_FAILURE);
}
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
return (ISC_R_SUCCESS);
#else
UNUSED(fd);
return (ISC_R_NOTIMPLEMENTED);
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
#endif
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
}
isc_result_t
isc__nm_socket_reuse_lb(uv_os_sock_t fd) {
/*
* On FreeBSD 12+, SO_REUSEPORT_LB socket option allows sockets to be
* bound to an identical socket address. For UDP sockets, the use of
* this option can provide better distribution of incoming datagrams to
* multiple processes (or threads) as compared to the traditional
* technique of having multiple processes compete to receive datagrams
* on the same socket.
*
* On Linux, the same thing is achieved simply with SO_REUSEPORT.
*/
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
#if defined(SO_REUSEPORT_LB)
if (setsockopt_on(fd, SOL_SOCKET, SO_REUSEPORT_LB) == -1) {
return (ISC_R_FAILURE);
} else {
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
return (ISC_R_SUCCESS);
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
}
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
#elif defined(SO_REUSEPORT) && defined(__linux__)
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
if (setsockopt_on(fd, SOL_SOCKET, SO_REUSEPORT) == -1) {
return (ISC_R_FAILURE);
} else {
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
return (ISC_R_SUCCESS);
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
}
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
#else
UNUSED(fd);
return (ISC_R_NOTIMPLEMENTED);
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
#endif
}
isc_result_t
Use uv_os_sock_t instead of uv_os_fd_t for sockets On POSIX based systems both uv_os_sock_t and uv_os_fd_t are both typedef to int. That's not true on Windows, where uv_os_sock_t is SOCKET and uv_os_fd_t is HANDLE and they differ in level of indirection.
2020-10-05 12:25:19 +02:00
isc__nm_socket_incoming_cpu(uv_os_sock_t fd) {
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
#ifdef SO_INCOMING_CPU
if (setsockopt_on(fd, SOL_SOCKET, SO_INCOMING_CPU) == -1) {
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#else
UNUSED(fd);
#endif
return (ISC_R_NOTIMPLEMENTED);
}
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
isc_result_t
Disable the Path MTU Discover on UDP Sockets Instead of disabling the fragmentation on the UDP sockets, we now disable the Path MTU Discovery by setting IP(V6)_MTU_DISCOVER socket option to IP_PMTUDISC_OMIT on Linux and disabling IP(V6)_DONTFRAG socket option on FreeBSD. This option sets DF=0 in the IP header and also ignores the Path MTU Discovery. As additional mitigation on Linux, we recommend setting net.ipv4.ip_no_pmtu_disc to Mode 3: Mode 3 is a hardend pmtu discover mode. The kernel will only accept fragmentation-needed errors if the underlying protocol can verify them besides a plain socket lookup. Current protocols for which pmtu events will be honored are TCP, SCTP and DCCP as they verify e.g. the sequence number or the association. This mode should not be enabled globally but is only intended to secure e.g. name servers in namespaces where TCP path mtu must still work but path MTU information of other protocols should be discarded. If enabled globally this mode could break other protocols.
2021-07-28 15:55:46 +02:00
isc__nm_socket_disable_pmtud(uv_os_sock_t fd, sa_family_t sa_family) {
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
/*
Disable the Path MTU Discover on UDP Sockets Instead of disabling the fragmentation on the UDP sockets, we now disable the Path MTU Discovery by setting IP(V6)_MTU_DISCOVER socket option to IP_PMTUDISC_OMIT on Linux and disabling IP(V6)_DONTFRAG socket option on FreeBSD. This option sets DF=0 in the IP header and also ignores the Path MTU Discovery. As additional mitigation on Linux, we recommend setting net.ipv4.ip_no_pmtu_disc to Mode 3: Mode 3 is a hardend pmtu discover mode. The kernel will only accept fragmentation-needed errors if the underlying protocol can verify them besides a plain socket lookup. Current protocols for which pmtu events will be honored are TCP, SCTP and DCCP as they verify e.g. the sequence number or the association. This mode should not be enabled globally but is only intended to secure e.g. name servers in namespaces where TCP path mtu must still work but path MTU information of other protocols should be discarded. If enabled globally this mode could break other protocols.
2021-07-28 15:55:46 +02:00
* Disable the Path MTU Discovery on IP packets
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
*/
if (sa_family == AF_INET6) {
#if defined(IPV6_DONTFRAG)
Disable the Path MTU Discover on UDP Sockets Instead of disabling the fragmentation on the UDP sockets, we now disable the Path MTU Discovery by setting IP(V6)_MTU_DISCOVER socket option to IP_PMTUDISC_OMIT on Linux and disabling IP(V6)_DONTFRAG socket option on FreeBSD. This option sets DF=0 in the IP header and also ignores the Path MTU Discovery. As additional mitigation on Linux, we recommend setting net.ipv4.ip_no_pmtu_disc to Mode 3: Mode 3 is a hardend pmtu discover mode. The kernel will only accept fragmentation-needed errors if the underlying protocol can verify them besides a plain socket lookup. Current protocols for which pmtu events will be honored are TCP, SCTP and DCCP as they verify e.g. the sequence number or the association. This mode should not be enabled globally but is only intended to secure e.g. name servers in namespaces where TCP path mtu must still work but path MTU information of other protocols should be discarded. If enabled globally this mode could break other protocols.
2021-07-28 15:55:46 +02:00
if (setsockopt_off(fd, IPPROTO_IPV6, IPV6_DONTFRAG) == -1) {
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
Disable the Path MTU Discover on UDP Sockets Instead of disabling the fragmentation on the UDP sockets, we now disable the Path MTU Discovery by setting IP(V6)_MTU_DISCOVER socket option to IP_PMTUDISC_OMIT on Linux and disabling IP(V6)_DONTFRAG socket option on FreeBSD. This option sets DF=0 in the IP header and also ignores the Path MTU Discovery. As additional mitigation on Linux, we recommend setting net.ipv4.ip_no_pmtu_disc to Mode 3: Mode 3 is a hardend pmtu discover mode. The kernel will only accept fragmentation-needed errors if the underlying protocol can verify them besides a plain socket lookup. Current protocols for which pmtu events will be honored are TCP, SCTP and DCCP as they verify e.g. the sequence number or the association. This mode should not be enabled globally but is only intended to secure e.g. name servers in namespaces where TCP path mtu must still work but path MTU information of other protocols should be discarded. If enabled globally this mode could break other protocols.
2021-07-28 15:55:46 +02:00
#elif defined(IPV6_MTU_DISCOVER) && defined(IP_PMTUDISC_OMIT)
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
if (setsockopt(fd, IPPROTO_IPV6, IPV6_MTU_DISCOVER,
Disable the Path MTU Discover on UDP Sockets Instead of disabling the fragmentation on the UDP sockets, we now disable the Path MTU Discovery by setting IP(V6)_MTU_DISCOVER socket option to IP_PMTUDISC_OMIT on Linux and disabling IP(V6)_DONTFRAG socket option on FreeBSD. This option sets DF=0 in the IP header and also ignores the Path MTU Discovery. As additional mitigation on Linux, we recommend setting net.ipv4.ip_no_pmtu_disc to Mode 3: Mode 3 is a hardend pmtu discover mode. The kernel will only accept fragmentation-needed errors if the underlying protocol can verify them besides a plain socket lookup. Current protocols for which pmtu events will be honored are TCP, SCTP and DCCP as they verify e.g. the sequence number or the association. This mode should not be enabled globally but is only intended to secure e.g. name servers in namespaces where TCP path mtu must still work but path MTU information of other protocols should be discarded. If enabled globally this mode could break other protocols.
2021-07-28 15:55:46 +02:00
&(int){ IP_PMTUDISC_OMIT }, sizeof(int)) == -1)
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
{
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#else
UNUSED(fd);
#endif
} else if (sa_family == AF_INET) {
#if defined(IP_DONTFRAG)
Disable the Path MTU Discover on UDP Sockets Instead of disabling the fragmentation on the UDP sockets, we now disable the Path MTU Discovery by setting IP(V6)_MTU_DISCOVER socket option to IP_PMTUDISC_OMIT on Linux and disabling IP(V6)_DONTFRAG socket option on FreeBSD. This option sets DF=0 in the IP header and also ignores the Path MTU Discovery. As additional mitigation on Linux, we recommend setting net.ipv4.ip_no_pmtu_disc to Mode 3: Mode 3 is a hardend pmtu discover mode. The kernel will only accept fragmentation-needed errors if the underlying protocol can verify them besides a plain socket lookup. Current protocols for which pmtu events will be honored are TCP, SCTP and DCCP as they verify e.g. the sequence number or the association. This mode should not be enabled globally but is only intended to secure e.g. name servers in namespaces where TCP path mtu must still work but path MTU information of other protocols should be discarded. If enabled globally this mode could break other protocols.
2021-07-28 15:55:46 +02:00
if (setsockopt_off(fd, IPPROTO_IP, IP_DONTFRAG) == -1) {
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
Disable the Path MTU Discover on UDP Sockets Instead of disabling the fragmentation on the UDP sockets, we now disable the Path MTU Discovery by setting IP(V6)_MTU_DISCOVER socket option to IP_PMTUDISC_OMIT on Linux and disabling IP(V6)_DONTFRAG socket option on FreeBSD. This option sets DF=0 in the IP header and also ignores the Path MTU Discovery. As additional mitigation on Linux, we recommend setting net.ipv4.ip_no_pmtu_disc to Mode 3: Mode 3 is a hardend pmtu discover mode. The kernel will only accept fragmentation-needed errors if the underlying protocol can verify them besides a plain socket lookup. Current protocols for which pmtu events will be honored are TCP, SCTP and DCCP as they verify e.g. the sequence number or the association. This mode should not be enabled globally but is only intended to secure e.g. name servers in namespaces where TCP path mtu must still work but path MTU information of other protocols should be discarded. If enabled globally this mode could break other protocols.
2021-07-28 15:55:46 +02:00
#elif defined(IP_MTU_DISCOVER) && defined(IP_PMTUDISC_OMIT)
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
if (setsockopt(fd, IPPROTO_IP, IP_MTU_DISCOVER,
Disable the Path MTU Discover on UDP Sockets Instead of disabling the fragmentation on the UDP sockets, we now disable the Path MTU Discovery by setting IP(V6)_MTU_DISCOVER socket option to IP_PMTUDISC_OMIT on Linux and disabling IP(V6)_DONTFRAG socket option on FreeBSD. This option sets DF=0 in the IP header and also ignores the Path MTU Discovery. As additional mitigation on Linux, we recommend setting net.ipv4.ip_no_pmtu_disc to Mode 3: Mode 3 is a hardend pmtu discover mode. The kernel will only accept fragmentation-needed errors if the underlying protocol can verify them besides a plain socket lookup. Current protocols for which pmtu events will be honored are TCP, SCTP and DCCP as they verify e.g. the sequence number or the association. This mode should not be enabled globally but is only intended to secure e.g. name servers in namespaces where TCP path mtu must still work but path MTU information of other protocols should be discarded. If enabled globally this mode could break other protocols.
2021-07-28 15:55:46 +02:00
&(int){ IP_PMTUDISC_OMIT }, sizeof(int)) == -1)
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
{
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#else
UNUSED(fd);
#endif
} else {
return (ISC_R_FAMILYNOSUPPORT);
}
return (ISC_R_NOTIMPLEMENTED);
}
Explicitly enable IPV6_V6ONLY on the netmgr sockets Some operating systems (OpenBSD and DragonFly BSD) don't restrict the IPv6 sockets to sending and receiving IPv6 packets only. Explicitly enable the IPV6_V6ONLY socket option on the IPv6 sockets to prevent failures from using the IPv4-mapped IPv6 address.
2022-01-13 13:24:55 +01:00
isc_result_t
isc__nm_socket_v6only(uv_os_sock_t fd, sa_family_t sa_family) {
/*
* Enable the IPv6-only option on IPv6 sockets
*/
if (sa_family == AF_INET6) {
#if defined(IPV6_V6ONLY)
if (setsockopt_on(fd, IPPROTO_IPV6, IPV6_V6ONLY) == -1) {
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#else
UNUSED(fd);
#endif
}
return (ISC_R_NOTIMPLEMENTED);
}
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
isc_result_t
isc_nm_checkaddr(const isc_sockaddr_t *addr, isc_socktype_t type) {
int proto, pf, addrlen, fd, r;
REQUIRE(addr != NULL);
switch (type) {
case isc_socktype_tcp:
proto = SOCK_STREAM;
break;
case isc_socktype_udp:
proto = SOCK_DGRAM;
break;
default:
return (ISC_R_NOTIMPLEMENTED);
}
pf = isc_sockaddr_pf(addr);
if (pf == AF_INET) {
addrlen = sizeof(struct sockaddr_in);
} else {
addrlen = sizeof(struct sockaddr_in6);
}
fd = socket(pf, proto, 0);
if (fd < 0) {
return (isc_errno_toresult(errno));
}
r = bind(fd, (const struct sockaddr *)&addr->type.sa, addrlen);
if (r < 0) {
close(fd);
return (isc_errno_toresult(errno));
}
close(fd);
return (ISC_R_SUCCESS);
}
Completely remove BIND 9 Windows support The Windows support has been completely removed from the source tree and BIND 9 now no longer supports native compilation on Windows. We might consider reviewing mingw-w64 port if contributed by external party, but no development efforts will be put into making BIND 9 compile and run on Windows again.
2021-05-20 15:53:50 +02:00
#if defined(TCP_CONNECTIONTIMEOUT)
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
#define TIMEOUT_TYPE int
#define TIMEOUT_DIV 1000
#define TIMEOUT_OPTNAME TCP_CONNECTIONTIMEOUT
#elif defined(TCP_RXT_CONNDROPTIME)
#define TIMEOUT_TYPE int
#define TIMEOUT_DIV 1000
#define TIMEOUT_OPTNAME TCP_RXT_CONNDROPTIME
#elif defined(TCP_USER_TIMEOUT)
#define TIMEOUT_TYPE unsigned int
#define TIMEOUT_DIV 1
#define TIMEOUT_OPTNAME TCP_USER_TIMEOUT
Add FreeBSD connection timeout socket option On FreeBSD, the option to configure connection timeout is called TCP_KEEPINIT, use it to configure the connection timeout there. This also fixes the dangling socket problems in the unit test, so re-enable them.
2020-12-02 21:54:25 +01:00
#elif defined(TCP_KEEPINIT)
#define TIMEOUT_TYPE int
#define TIMEOUT_DIV 1000
#define TIMEOUT_OPTNAME TCP_KEEPINIT
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
#endif
isc_result_t
isc__nm_socket_connectiontimeout(uv_os_sock_t fd, int timeout_ms) {
#if defined(TIMEOUT_OPTNAME)
TIMEOUT_TYPE timeout = timeout_ms / TIMEOUT_DIV;
if (timeout == 0) {
timeout = 1;
}
if (setsockopt(fd, IPPROTO_TCP, TIMEOUT_OPTNAME, &timeout,
sizeof(timeout)) == -1)
{
return (ISC_R_FAILURE);
}
return (ISC_R_SUCCESS);
#else
UNUSED(fd);
UNUSED(timeout_ms);
return (ISC_R_SUCCESS);
#endif
}
Disable Nagle's algorithm for HTTP/2 connections It is advisable to disable Nagle's algorithm for HTTP/2 connections because multiple HTTP/2 streams could be multiplexed over one transport connection. Thus, delays when delivering small packets could bring down performance for the whole session. HTTP/2 is meant to be used this way.
2021-02-28 19:33:16 +02:00
isc_result_t
isc__nm_socket_tcp_nodelay(uv_os_sock_t fd) {
#ifdef TCP_NODELAY
if (setsockopt_on(fd, IPPROTO_TCP, TCP_NODELAY) == -1) {
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#else
UNUSED(fd);
return (ISC_R_SUCCESS);
#endif
}
Set TCP maximum segment size to minimum size of 1220 Previously the socket code would set the TCPv6 maximum segment size to minimum value to prevent IP fragmentation for TCP. This was not yet implemented for the network manager. Implement network manager functions to set and use minimum MTU socket option and set the TCP_MAXSEG socket option for both IPv4 and IPv6 and use those to clamp the TCP maximum segment size for TCP, TCPDNS and TLSDNS layers in the network manager to 1220 bytes, that is 1280 (IPv6 minimum link MTU) minus 40 (IPv6 fixed header) minus 20 (TCP fixed header) We already rely on a similar value for UDP to prevent IP fragmentation and it make sense to use the same value for IPv4 and IPv6 because the modern networks are required to support IPv6 packet sizes. If there's need for small TCP segment values, the MTU on the interfaces needs to be properly configured.
2021-10-05 22:30:55 +02:00
isc_result_t
isc__nm_socket_tcp_maxseg(uv_os_sock_t fd, int size) {
#ifdef TCP_MAXSEG
if (setsockopt(fd, IPPROTO_TCP, TCP_MAXSEG, (void *)&size,
sizeof(size))) {
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#else
UNUSED(fd);
UNUSED(size);
return (ISC_R_SUCCESS);
#endif
}
Set minimum MTU (1280) on IPv6 sockets The IPV6_USE_MIN_MTU socket option directs the IP layer to limit the IPv6 packet size to the minimum required supported MTU from the base IPv6 specification, i.e. 1280 bytes. Many implementations of TCP running over IPv6 neglect to check the IPV6_USE_MIN_MTU value when performing MSS negotiation and when constructing a TCP segment despite MSS being defined to be the MTU less the IP and TCP header sizes (60 bytes for IPv6). This leads to oversized IPv6 packets being sent resulting in unintended Path Maximum Transport Unit Discovery (PMTUD) being performed and to fragmented IPv6 packets being sent. Add and use a function to set socket option to limit the MTU on IPv6 sockets to the minimum MTU (1280) both for UDP and TCP.
2021-10-05 22:30:55 +02:00
isc_result_t
isc__nm_socket_min_mtu(uv_os_sock_t fd, sa_family_t sa_family) {
if (sa_family != AF_INET6) {
return (ISC_R_SUCCESS);
}
#ifdef IPV6_USE_MIN_MTU
if (setsockopt_on(fd, IPPROTO_IPV6, IPV6_USE_MIN_MTU) == -1) {
return (ISC_R_FAILURE);
}
#elif defined(IPV6_MTU)
if (setsockopt(fd, IPPROTO_IPV6, IPV6_MTU, &(int){ 1280 },
sizeof(int)) == -1) {
return (ISC_R_FAILURE);
}
#else
UNUSED(fd);
#endif
return (ISC_R_SUCCESS);
}
Add configuration option to set send/recv buffers on the nm sockets This commit adds a new configuration option to set the receive and send buffer sizes on the TCP and UDP netmgr sockets. The default is `0` which doesn't set any value and just uses the value set by the operating system. There's no magic value here - set it too small and the performance will drop, set it too large, the buffers can fill-up with queries that have already timeouted on the client side and nobody is interested for the answer and this would just make the server clog up even more by making it produce useless work. The `netstat -su` can be used on POSIX systems to monitor the receive and send buffer errors.
2020-12-02 20:51:38 +01:00
void
isc__nm_set_network_buffers(isc_nm_t *nm, uv_handle_t *handle) {
int32_t recv_buffer_size = 0;
int32_t send_buffer_size = 0;
switch (handle->type) {
case UV_TCP:
recv_buffer_size =
atomic_load_relaxed(&nm->recv_tcp_buffer_size);
send_buffer_size =
atomic_load_relaxed(&nm->send_tcp_buffer_size);
break;
case UV_UDP:
recv_buffer_size =
atomic_load_relaxed(&nm->recv_udp_buffer_size);
send_buffer_size =
atomic_load_relaxed(&nm->send_udp_buffer_size);
break;
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Add configuration option to set send/recv buffers on the nm sockets This commit adds a new configuration option to set the receive and send buffer sizes on the TCP and UDP netmgr sockets. The default is `0` which doesn't set any value and just uses the value set by the operating system. There's no magic value here - set it too small and the performance will drop, set it too large, the buffers can fill-up with queries that have already timeouted on the client side and nobody is interested for the answer and this would just make the server clog up even more by making it produce useless work. The `netstat -su` can be used on POSIX systems to monitor the receive and send buffer errors.
2020-12-02 20:51:38 +01:00
}
if (recv_buffer_size > 0) {
int r = uv_recv_buffer_size(handle, &recv_buffer_size);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_recv_buffer_size, r);
Add configuration option to set send/recv buffers on the nm sockets This commit adds a new configuration option to set the receive and send buffer sizes on the TCP and UDP netmgr sockets. The default is `0` which doesn't set any value and just uses the value set by the operating system. There's no magic value here - set it too small and the performance will drop, set it too large, the buffers can fill-up with queries that have already timeouted on the client side and nobody is interested for the answer and this would just make the server clog up even more by making it produce useless work. The `netstat -su` can be used on POSIX systems to monitor the receive and send buffer errors.
2020-12-02 20:51:38 +01:00
}
if (send_buffer_size > 0) {
int r = uv_send_buffer_size(handle, &send_buffer_size);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_send_buffer_size, r);
Add configuration option to set send/recv buffers on the nm sockets This commit adds a new configuration option to set the receive and send buffer sizes on the TCP and UDP netmgr sockets. The default is `0` which doesn't set any value and just uses the value set by the operating system. There's no magic value here - set it too small and the performance will drop, set it too large, the buffers can fill-up with queries that have already timeouted on the client side and nobody is interested for the answer and this would just make the server clog up even more by making it produce useless work. The `netstat -su` can be used on POSIX systems to monitor the receive and send buffer errors.
2020-12-02 20:51:38 +01:00
}
}
Add asynchronous work API to the network manager The libuv has a support for running long running tasks in the dedicated threadpools, so it doesn't affect networking IO. This commit adds isc_nm_work_enqueue() wrapper that would wraps around the libuv API and runs it on top of associated worker loop. The only limitation is that the function must be called from inside network manager thread, so the call to the function should be wrapped inside a (bound) task.
2021-05-27 09:45:07 +02:00
static isc_threadresult_t
isc__nm_work_run(isc_threadarg_t arg) {
isc__nm_work_t *work = (isc__nm_work_t *)arg;
work->cb(work->data);
return ((isc_threadresult_t)0);
}
static void
isc__nm_work_cb(uv_work_t *req) {
isc__nm_work_t *work = uv_req_get_data((uv_req_t *)req);
if (isc_tid_v == SIZE_MAX) {
isc__trampoline_t *trampoline_arg =
isc__trampoline_get(isc__nm_work_run, work);
(void)isc__trampoline_run(trampoline_arg);
} else {
(void)isc__nm_work_run((isc_threadarg_t)work);
}
}
static void
isc__nm_after_work_cb(uv_work_t *req, int status) {
isc_result_t result = ISC_R_SUCCESS;
isc__nm_work_t *work = uv_req_get_data((uv_req_t *)req);
isc_nm_t *netmgr = work->netmgr;
if (status != 0) {
result = isc__nm_uverr2result(status);
}
work->after_cb(work->data, result);
isc_mem_put(netmgr->mctx, work, sizeof(*work));
isc_nm_detach(&netmgr);
}
void
isc_nm_work_offload(isc_nm_t *netmgr, isc_nm_workcb_t work_cb,
isc_nm_after_workcb_t after_work_cb, void *data) {
isc__networker_t *worker = NULL;
isc__nm_work_t *work = NULL;
int r;
REQUIRE(isc__nm_in_netthread());
REQUIRE(VALID_NM(netmgr));
worker = &netmgr->workers[isc_nm_tid()];
work = isc_mem_get(netmgr->mctx, sizeof(*work));
*work = (isc__nm_work_t){
.cb = work_cb,
.after_cb = after_work_cb,
.data = data,
};
isc_nm_attach(netmgr, &work->netmgr);
uv_req_set_data((uv_req_t *)&work->req, work);
r = uv_queue_work(&worker->loop, &work->req, isc__nm_work_cb,
isc__nm_after_work_cb);
Use UV_RUNTIME_CHECK() as appropriate Replace the RUNTIME_CHECK() calls for libuv API calls with UV_RUNTIME_CHECK() to get more detailed error message when something fails and should not.
2022-02-15 14:51:02 +01:00
UV_RUNTIME_CHECK(uv_queue_work, r);
Add asynchronous work API to the network manager The libuv has a support for running long running tasks in the dedicated threadpools, so it doesn't affect networking IO. This commit adds isc_nm_work_enqueue() wrapper that would wraps around the libuv API and runs it on top of associated worker loop. The only limitation is that the function must be called from inside network manager thread, so the call to the function should be wrapped inside a (bound) task.
2021-05-27 09:45:07 +02:00
}
Return HTTP status code for small/malformed requests This commit makes BIND return HTTP status codes for malformed or too small requests. DNS request processing code would ignore such requests. Such an approach works well for other DNS transport but does not make much sense for HTTP, not allowing it to complete the request/response sequence. Suppose execution has reached the point where DNS message handling code has been called. In that case, it means that the HTTP request has been successfully processed, and, thus, we are expected to respond to it either with a message containing some DNS payload or at least to return an error status code. This commit ensures that BIND behaves this way.
2021-07-06 16:36:17 +03:00
void
isc_nm_bad_request(isc_nmhandle_t *handle) {
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
isc_nmsocket_t *sock = NULL;
Return HTTP status code for small/malformed requests This commit makes BIND return HTTP status codes for malformed or too small requests. DNS request processing code would ignore such requests. Such an approach works well for other DNS transport but does not make much sense for HTTP, not allowing it to complete the request/response sequence. Suppose execution has reached the point where DNS message handling code has been called. In that case, it means that the HTTP request has been successfully processed, and, thus, we are expected to respond to it either with a message containing some DNS payload or at least to return an error status code. This commit ensures that BIND behaves this way.
2021-07-06 16:36:17 +03:00
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
sock = handle->sock;
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
switch (sock->type) {
Return HTTP status code for small/malformed requests This commit makes BIND return HTTP status codes for malformed or too small requests. DNS request processing code would ignore such requests. Such an approach works well for other DNS transport but does not make much sense for HTTP, not allowing it to complete the request/response sequence. Suppose execution has reached the point where DNS message handling code has been called. In that case, it means that the HTTP request has been successfully processed, and, thus, we are expected to respond to it either with a message containing some DNS payload or at least to return an error status code. This commit ensures that BIND behaves this way.
2021-07-06 16:36:17 +03:00
case isc_nm_udpsocket:
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
return;
Return HTTP status code for small/malformed requests This commit makes BIND return HTTP status codes for malformed or too small requests. DNS request processing code would ignore such requests. Such an approach works well for other DNS transport but does not make much sense for HTTP, not allowing it to complete the request/response sequence. Suppose execution has reached the point where DNS message handling code has been called. In that case, it means that the HTTP request has been successfully processed, and, thus, we are expected to respond to it either with a message containing some DNS payload or at least to return an error status code. This commit ensures that BIND behaves this way.
2021-07-06 16:36:17 +03:00
case isc_nm_tcpdnssocket:
case isc_nm_tlsdnssocket:
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
REQUIRE(sock->parent == NULL);
isc__nmsocket_reset(sock);
Return HTTP status code for small/malformed requests This commit makes BIND return HTTP status codes for malformed or too small requests. DNS request processing code would ignore such requests. Such an approach works well for other DNS transport but does not make much sense for HTTP, not allowing it to complete the request/response sequence. Suppose execution has reached the point where DNS message handling code has been called. In that case, it means that the HTTP request has been successfully processed, and, thus, we are expected to respond to it either with a message containing some DNS payload or at least to return an error status code. This commit ensures that BIND behaves this way.
2021-07-06 16:36:17 +03:00
return;
Reset the TCP connection when garbage is received When invalid DNS message is received, there was a handling mechanism for DoH that would be called to return proper HTTP response. Reuse this mechanism and reset the TCP connection when the client is blackholed, DNS message is completely bogus or the ns_client receives response instead of query.
2022-02-15 14:41:15 +01:00
#if HAVE_LIBNGHTTP2
case isc_nm_httpsocket:
isc__nm_http_bad_request(handle);
return;
#endif /* HAVE_LIBNGHTTP2 */
Return HTTP status code for small/malformed requests This commit makes BIND return HTTP status codes for malformed or too small requests. DNS request processing code would ignore such requests. Such an approach works well for other DNS transport but does not make much sense for HTTP, not allowing it to complete the request/response sequence. Suppose execution has reached the point where DNS message handling code has been called. In that case, it means that the HTTP request has been successfully processed, and, thus, we are expected to respond to it either with a message containing some DNS payload or at least to return an error status code. This commit ensures that BIND behaves this way.
2021-07-06 16:36:17 +03:00
case isc_nm_tcpsocket:
#if HAVE_LIBNGHTTP2
case isc_nm_tlssocket:
#endif /* HAVE_LIBNGHTTP2 */
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Return HTTP status code for small/malformed requests This commit makes BIND return HTTP status codes for malformed or too small requests. DNS request processing code would ignore such requests. Such an approach works well for other DNS transport but does not make much sense for HTTP, not allowing it to complete the request/response sequence. Suppose execution has reached the point where DNS message handling code has been called. In that case, it means that the HTTP request has been successfully processed, and, thus, we are expected to respond to it either with a message containing some DNS payload or at least to return an error status code. This commit ensures that BIND behaves this way.
2021-07-06 16:36:17 +03:00
break;
}
}
Add isc_nm_xfr_allowed() function The intention of having this function is to have a predicate to check if a zone transfer could be performed over the given handle. In most cases we can assume that we can do zone transfers over any stream transport except DoH, but this assumption will not work for zone transfers over DoT (XoT), as the RFC9103 requires ALPN to happen, which might not be the case for all deployments of DoT.
2021-08-26 15:07:20 +03:00
bool
isc_nm_xfr_allowed(isc_nmhandle_t *handle) {
isc_nmsocket_t *sock;
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
sock = handle->sock;
switch (sock->type) {
case isc_nm_tcpdnssocket:
return (true);
case isc_nm_tlsdnssocket:
return (isc__nm_tlsdns_xfr_allowed(sock));
default:
return (false);
}
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
Add isc_nm_xfr_allowed() function The intention of having this function is to have a predicate to check if a zone transfer could be performed over the given handle. In most cases we can assume that we can do zone transfers over any stream transport except DoH, but this assumption will not work for zone transfers over DoT (XoT), as the RFC9103 requires ALPN to happen, which might not be the case for all deployments of DoT.
2021-08-26 15:07:20 +03:00
return (false);
}
address '--disable-doh' failures Change 5756 (GL #2854) introduced build errors when using 'configure --disable-doh'. To fix this, isc_nm_is_http_handle() is now defined in all builds, not just builds that have DoH enabled. Missing code comments were added both for that function and for isc_nm_is_tlsdns_handle().
2021-11-08 12:44:55 -08:00
bool
isc_nm_is_http_handle(isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
return (handle->sock->type == isc_nm_httpsocket);
}
DoH: Add isc_nm_set_min_answer_ttl() This commit adds an isc_nm_set_min_answer_ttl() function which is intended to to be used to give a hint to the underlying transport regarding the answer TTL. The interface is intentionally kept generic because over time more transports might benefit from this functionality, but currently it is intended for DoH to set "max-age" value within "Cache-Control" HTTP header (as recommended in the RFC8484, section 5.1 "Cache Interaction"). It is no-op for other DNS transports for the time being.
2021-10-06 14:09:53 +03:00
void
isc_nm_set_maxage(isc_nmhandle_t *handle, const uint32_t ttl) {
address '--disable-doh' failures Change 5756 (GL #2854) introduced build errors when using 'configure --disable-doh'. To fix this, isc_nm_is_http_handle() is now defined in all builds, not just builds that have DoH enabled. Missing code comments were added both for that function and for isc_nm_is_tlsdns_handle().
2021-11-08 12:44:55 -08:00
isc_nmsocket_t *sock = NULL;
DoH: Add isc_nm_set_min_answer_ttl() This commit adds an isc_nm_set_min_answer_ttl() function which is intended to to be used to give a hint to the underlying transport regarding the answer TTL. The interface is intentionally kept generic because over time more transports might benefit from this functionality, but currently it is intended for DoH to set "max-age" value within "Cache-Control" HTTP header (as recommended in the RFC8484, section 5.1 "Cache Interaction"). It is no-op for other DNS transports for the time being.
2021-10-06 14:09:53 +03:00
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
REQUIRE(!atomic_load(&handle->sock->client));
address '--disable-doh' failures Change 5756 (GL #2854) introduced build errors when using 'configure --disable-doh'. To fix this, isc_nm_is_http_handle() is now defined in all builds, not just builds that have DoH enabled. Missing code comments were added both for that function and for isc_nm_is_tlsdns_handle().
2021-11-08 12:44:55 -08:00
#if !HAVE_LIBNGHTTP2
UNUSED(ttl);
#endif
DoH: Add isc_nm_set_min_answer_ttl() This commit adds an isc_nm_set_min_answer_ttl() function which is intended to to be used to give a hint to the underlying transport regarding the answer TTL. The interface is intentionally kept generic because over time more transports might benefit from this functionality, but currently it is intended for DoH to set "max-age" value within "Cache-Control" HTTP header (as recommended in the RFC8484, section 5.1 "Cache Interaction"). It is no-op for other DNS transports for the time being.
2021-10-06 14:09:53 +03:00
sock = handle->sock;
switch (sock->type) {
#if HAVE_LIBNGHTTP2
case isc_nm_httpsocket:
isc__nm_http_set_maxage(handle, ttl);
break;
#endif /* HAVE_LIBNGHTTP2 */
case isc_nm_udpsocket:
case isc_nm_tcpdnssocket:
case isc_nm_tlsdnssocket:
return;
break;
case isc_nm_tcpsocket:
#if HAVE_LIBNGHTTP2
case isc_nm_tlssocket:
#endif /* HAVE_LIBNGHTTP2 */
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
DoH: Add isc_nm_set_min_answer_ttl() This commit adds an isc_nm_set_min_answer_ttl() function which is intended to to be used to give a hint to the underlying transport regarding the answer TTL. The interface is intentionally kept generic because over time more transports might benefit from this functionality, but currently it is intended for DoH to set "max-age" value within "Cache-Control" HTTP header (as recommended in the RFC8484, section 5.1 "Cache Interaction"). It is no-op for other DNS transports for the time being.
2021-10-06 14:09:53 +03:00
break;
}
}
Add isc_nm_socket_type() This commit adds an isc_nm_socket_type() function which can be used to obtain a handle's socket type. This change obsoletes isc_nm_is_tlsdns_handle() and isc_nm_is_http_handle(). However, it was decided to keep the latter as we eventually might end up supporting multiple HTTP versions.
2021-11-15 17:42:15 +02:00
isc_nmsocket_type
isc_nm_socket_type(const isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
return (handle->sock->type);
}
Add isc_nm_has_encryption() This commit adds an isc_nm_has_encryption() function intended to check if a given handle is backed by a connection which uses encryption.
2021-11-16 13:35:37 +02:00
bool
isc_nm_has_encryption(const isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
switch (handle->sock->type) {
case isc_nm_tlsdnssocket:
#if HAVE_LIBNGHTTP2
case isc_nm_tlssocket:
#endif /* HAVE_LIBNGHTTP2 */
return (true);
#if HAVE_LIBNGHTTP2
case isc_nm_httpsocket:
return (isc__nm_http_has_encryption(handle));
#endif /* HAVE_LIBNGHTTP2 */
default:
return (false);
};
return (false);
}
Make netmgr the authority on number of threads running Instead of passing the "workers" variable back and forth along with passing the single isc_nm_t instance, add isc_nm_getnworkers() function that returns the number of netmgr threads are running. Change the ns_interfacemgr and ns_taskmgr to utilize the newly acquired knowledge.
2022-03-18 09:50:58 +01:00
uint32_t
isc_nm_getnworkers(const isc_nm_t *netmgr) {
REQUIRE(VALID_NM(netmgr));
return (netmgr->nworkers);
}
Add ISC_R_TLSBADPEERCERT error code to the TLS related code This commit adds support for ISC_R_TLSBADPEERCERT error code, which is supposed to be used to signal for TLS peer certificates verification in dig and other code. The support for this error code is added to our TLS and TLS DNS implementations. This commit also adds isc_nm_verify_tls_peer_result_string() function which is supposed to be used to get a textual description of the reason for getting a ISC_R_TLSBADPEERCERT error.
2022-01-13 14:35:24 +02:00
const char *
isc_nm_verify_tls_peer_result_string(const isc_nmhandle_t *handle) {
isc_nmsocket_t *sock;
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
sock = handle->sock;
switch (sock->type) {
case isc_nm_tlsdnssocket:
return (isc__nm_tlsdns_verify_tls_peer_result_string(handle));
break;
#if HAVE_LIBNGHTTP2
case isc_nm_tlssocket:
return (isc__nm_tls_verify_tls_peer_result_string(handle));
break;
case isc_nm_httpsocket:
return (isc__nm_http_verify_tls_peer_result_string(handle));
break;
#endif /* HAVE_LIBNGHTTP2 */
default:
break;
}
return (NULL);
}
Add isc_nmsocket_set_tlsctx() This commit adds isc_nmsocket_set_tlsctx() - an asynchronous function that replaces the TLS context within a given TLS-enabled listener socket object. It is based on the newly added reference counting functionality. The intention of adding this function is to add functionality to replace a TLS context without recreating the whole socket object, including the underlying TCP listener socket, as a BIND process might not have enough permissions to re-create it fully on reconfiguration.
2022-03-31 13:47:48 +03:00
void
isc__nm_async_settlsctx(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent__tlsctx_t *ev_tlsctx = (isc__netievent__tlsctx_t *)ev0;
const int tid = isc_nm_tid();
isc_nmsocket_t *listener = ev_tlsctx->sock;
isc_tlsctx_t *tlsctx = ev_tlsctx->tlsctx;
UNUSED(worker);
switch (listener->type) {
case isc_nm_tlsdnslistener:
isc__nm_async_tlsdns_set_tlsctx(listener, tlsctx, tid);
break;
#if HAVE_LIBNGHTTP2
case isc_nm_tlslistener:
isc__nm_async_tls_set_tlsctx(listener, tlsctx, tid);
break;
#endif /* HAVE_LIBNGHTTP2 */
default:
UNREACHABLE();
break;
};
}
static void
set_tlsctx_workers(isc_nmsocket_t *listener, isc_tlsctx_t *tlsctx) {
/* Update the TLS context reference for every worker thread. */
for (size_t i = 0; i < isc_nm_getnworkers(listener->mgr); i++) {
isc__netievent__tlsctx_t *ievent =
isc__nm_get_netievent_settlsctx(listener->mgr, listener,
tlsctx);
isc__nm_enqueue_ievent(&listener->mgr->workers[i],
(isc__netievent_t *)ievent);
}
}
void
isc_nmsocket_set_tlsctx(isc_nmsocket_t *listener, isc_tlsctx_t *tlsctx) {
REQUIRE(VALID_NMSOCK(listener));
REQUIRE(tlsctx != NULL);
switch (listener->type) {
#if HAVE_LIBNGHTTP2
case isc_nm_httplistener:
/*
* We handle HTTP listener sockets differently, as they rely
* on underlying TLS sockets for networking. The TLS context
* will get passed to these underlying sockets via the call to
* isc__nm_http_set_tlsctx().
*/
isc__nm_http_set_tlsctx(listener, tlsctx);
break;
case isc_nm_tlslistener:
set_tlsctx_workers(listener, tlsctx);
break;
#endif /* HAVE_LIBNGHTTP2 */
case isc_nm_tlsdnslistener:
set_tlsctx_workers(listener, tlsctx);
break;
default:
UNREACHABLE();
break;
};
}
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
/*
* Dump all active sockets in netmgr. We output to stderr
* as the logger might be already shut down.
*/
static const char *
nmsocket_type_totext(isc_nmsocket_type type) {
switch (type) {
case isc_nm_udpsocket:
return ("isc_nm_udpsocket");
case isc_nm_udplistener:
return ("isc_nm_udplistener");
case isc_nm_tcpsocket:
return ("isc_nm_tcpsocket");
case isc_nm_tcplistener:
return ("isc_nm_tcplistener");
case isc_nm_tcpdnslistener:
return ("isc_nm_tcpdnslistener");
case isc_nm_tcpdnssocket:
return ("isc_nm_tcpdnssocket");
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
return ("isc_nm_tlssocket");
case isc_nm_tlslistener:
return ("isc_nm_tlslistener");
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnslistener:
return ("isc_nm_tlsdnslistener");
case isc_nm_tlsdnssocket:
return ("isc_nm_tlsdnssocket");
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
case isc_nm_httplistener:
return ("isc_nm_httplistener");
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
case isc_nm_httpsocket:
return ("isc_nm_httpsocket");
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
}
}
static void
nmhandle_dump(isc_nmhandle_t *handle) {
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
fprintf(stderr, "Active handle %p, refs %" PRIuFAST32 "\n", handle,
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
isc_refcount_current(&handle->references));
fprintf(stderr, "Created by:\n");
Add support for generating backtraces on Windows This commit adds support for generating backtraces on Windows and refactors the isc_backtrace API to match the Linux/BSD API (without the isc_ prefix) * isc_backtrace_gettrace() was renamed to isc_backtrace(), the third argument was removed and the return type was changed to int * isc_backtrace_symbols() was added * isc_backtrace_symbols_fd() was added and used as appropriate
2021-04-27 16:20:03 +02:00
isc_backtrace_symbols_fd(handle->backtrace, handle->backtrace_size,
STDERR_FILENO);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
fprintf(stderr, "\n\n");
}
static void
nmsocket_dump(isc_nmsocket_t *sock) {
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc_nmhandle_t *handle = NULL;
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
LOCK(&sock->lock);
fprintf(stderr, "\n=================\n");
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
fprintf(stderr, "Active %s socket %p, type %s, refs %" PRIuFAST32 "\n",
atomic_load(&sock->client) ? "client" : "server", sock,
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
nmsocket_type_totext(sock->type),
isc_refcount_current(&sock->references));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
fprintf(stderr,
Add asynchronous work API to the network manager The libuv has a support for running long running tasks in the dedicated threadpools, so it doesn't affect networking IO. This commit adds isc_nm_work_enqueue() wrapper that would wraps around the libuv API and runs it on top of associated worker loop. The only limitation is that the function must be called from inside network manager thread, so the call to the function should be wrapped inside a (bound) task.
2021-05-27 09:45:07 +02:00
"Parent %p, listener %p, server %p, statichandle = "
"%p\n",
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
sock->parent, sock->listener, sock->server, sock->statichandle);
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
fprintf(stderr, "Flags:%s%s%s%s%s\n",
atomic_load(&sock->active) ? " active" : "",
atomic_load(&sock->closing) ? " closing" : "",
atomic_load(&sock->destroying) ? " destroying" : "",
atomic_load(&sock->connecting) ? " connecting" : "",
netmgr fixes needed for dispatch - The read timer must always be stopped when reading stops. - Read callbacks can now call isc_nm_read() again in TCP, TCPDNS and TLSDNS; previously this caused an assertion. - The wrong failure code could be sent after a UDP recv failure because the if statements were in the wrong order. the check for a NULL address needs to be after the check for an error code, otherwise the result will always be set to ISC_R_EOF. - When aborting a read or connect because the netmgr is shutting down, use ISC_R_SHUTTINGDOWN. (ISC_R_CANCELED is now reserved for when the read has been canceled by the caller.) - A new function isc_nmhandle_timer_running() has been added enabling a callback to check whether the timer has been reset after processing a timeout. - Incidental netmgr fix: always use isc__nm_closing() instead of referencing sock->mgr->closing directly - Corrected a few comments that used outdated function names.
2021-07-26 13:14:41 +02:00
atomic_load(&sock->accepting) ? " accepting" : "");
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
fprintf(stderr, "Created by:\n");
Add support for generating backtraces on Windows This commit adds support for generating backtraces on Windows and refactors the isc_backtrace API to match the Linux/BSD API (without the isc_ prefix) * isc_backtrace_gettrace() was renamed to isc_backtrace(), the third argument was removed and the return type was changed to int * isc_backtrace_symbols() was added * isc_backtrace_symbols_fd() was added and used as appropriate
2021-04-27 16:20:03 +02:00
isc_backtrace_symbols_fd(sock->backtrace, sock->backtrace_size,
STDERR_FILENO);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
fprintf(stderr, "\n");
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
for (handle = ISC_LIST_HEAD(sock->active_handles); handle != NULL;
handle = ISC_LIST_NEXT(handle, active_link))
{
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
static bool first = true;
if (first) {
fprintf(stderr, "Active handles:\n");
first = false;
}
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
nmhandle_dump(handle);
}
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
fprintf(stderr, "\n");
UNLOCK(&sock->lock);
}
void
isc__nm_dump_active(isc_nm_t *nm) {
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc_nmsocket_t *sock = NULL;
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
REQUIRE(VALID_NM(nm));
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
LOCK(&nm->lock);
for (sock = ISC_LIST_HEAD(nm->active_sockets); sock != NULL;
sock = ISC_LIST_NEXT(sock, active_link))
{
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
static bool first = true;
if (first) {
fprintf(stderr, "Outstanding sockets\n");
first = false;
}
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
nmsocket_dump(sock);
}
UNLOCK(&nm->lock);
}
#endif
Reference in New Issue Copy Permalink