mir/bind
2
0
Fork 0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-23 18:49:54 +00:00
Code Issues Releases Wiki Activity
bind/lib/isc/netmgr/netmgr.c

3153 lines
77 KiB
C
Raw Normal View History

netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
* Copyright (C) Internet Systems Consortium, Inc. ("ISC")
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
update all copyright headers to eliminate the typo
2020-09-14 16:20:40 -07:00
* file, you can obtain one at https://mozilla.org/MPL/2.0/.
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*
* See the COPYRIGHT file distributed with this work for additional
* information regarding copyright ownership.
*/
#include <inttypes.h>
#include <unistd.h>
#include <uv.h>
#include <isc/atomic.h>
Add support for generating backtraces on Windows This commit adds support for generating backtraces on Windows and refactors the isc_backtrace API to match the Linux/BSD API (without the isc_ prefix) * isc_backtrace_gettrace() was renamed to isc_backtrace(), the third argument was removed and the return type was changed to int * isc_backtrace_symbols() was added * isc_backtrace_symbols_fd() was added and used as appropriate
2021-04-27 16:20:03 +02:00
#include <isc/backtrace.h>
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
#include <isc/buffer.h>
#include <isc/condition.h>
netmgr: Don't crash if socket() returns an error in udpconnect socket() call can return an error - e.g. EMFILE, so we need to handle this nicely and not crash. Additionally wrap the socket() call inside a platform independent helper function as the Socket data type on Windows is unsigned integer: > This means, for example, that checking for errors when the socket and > accept functions return should not be done by comparing the return > value with –1, or seeing if the value is negative (both common and > legal approaches in UNIX). Instead, an application should use the > manifest constant INVALID_SOCKET as defined in the Winsock2.h header > file.
2020-11-07 20:48:37 +01:00
#include <isc/errno.h>
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
#include <isc/log.h>
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
#include <isc/magic.h>
#include <isc/mem.h>
#include <isc/netmgr.h>
#include <isc/print.h>
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
#include <isc/quota.h>
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
#include <isc/random.h>
#include <isc/refcount.h>
#include <isc/region.h>
#include <isc/result.h>
#include <isc/sockaddr.h>
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
#include <isc/stats.h>
netmgr: Don't crash if socket() returns an error in udpconnect socket() call can return an error - e.g. EMFILE, so we need to handle this nicely and not crash. Additionally wrap the socket() call inside a platform independent helper function as the Socket data type on Windows is unsigned integer: > This means, for example, that checking for errors when the socket and > accept functions return should not be done by comparing the return > value with –1, or seeing if the value is negative (both common and > legal approaches in UNIX). Instead, an application should use the > manifest constant INVALID_SOCKET as defined in the Winsock2.h header > file.
2020-11-07 20:48:37 +01:00
#include <isc/strerr.h>
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
#include <isc/task.h>
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
#include <isc/thread.h>
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
#include <isc/tls.h>
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
#include <isc/util.h>
#include "netmgr-int.h"
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
#include "netmgr_p.h"
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
#include "openssl_shim.h"
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
#include "uv-compat.h"
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Increase inactivehandles and inactivereqs size for better reuse.
2020-01-29 15:16:02 +01:00
/*%
* How many isc_nmhandles and isc_nm_uvreqs will we be
* caching for reuse in a socket.
*/
#define ISC_NM_HANDLES_STACK_SIZE 600
#define ISC_NM_REQS_STACK_SIZE 600
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
/*%
* Shortcut index arrays to get access to statistics counters.
*/
static const isc_statscounter_t udp4statsindex[] = {
isc_sockstatscounter_udp4open,
isc_sockstatscounter_udp4openfail,
isc_sockstatscounter_udp4close,
isc_sockstatscounter_udp4bindfail,
isc_sockstatscounter_udp4connectfail,
isc_sockstatscounter_udp4connect,
-1,
-1,
isc_sockstatscounter_udp4sendfail,
isc_sockstatscounter_udp4recvfail,
isc_sockstatscounter_udp4active
};
static const isc_statscounter_t udp6statsindex[] = {
isc_sockstatscounter_udp6open,
isc_sockstatscounter_udp6openfail,
isc_sockstatscounter_udp6close,
isc_sockstatscounter_udp6bindfail,
isc_sockstatscounter_udp6connectfail,
isc_sockstatscounter_udp6connect,
-1,
-1,
isc_sockstatscounter_udp6sendfail,
isc_sockstatscounter_udp6recvfail,
isc_sockstatscounter_udp6active
};
static const isc_statscounter_t tcp4statsindex[] = {
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
isc_sockstatscounter_tcp4open, isc_sockstatscounter_tcp4openfail,
isc_sockstatscounter_tcp4close, isc_sockstatscounter_tcp4bindfail,
isc_sockstatscounter_tcp4connectfail, isc_sockstatscounter_tcp4connect,
isc_sockstatscounter_tcp4acceptfail, isc_sockstatscounter_tcp4accept,
isc_sockstatscounter_tcp4sendfail, isc_sockstatscounter_tcp4recvfail,
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
isc_sockstatscounter_tcp4active
};
static const isc_statscounter_t tcp6statsindex[] = {
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
isc_sockstatscounter_tcp6open, isc_sockstatscounter_tcp6openfail,
isc_sockstatscounter_tcp6close, isc_sockstatscounter_tcp6bindfail,
isc_sockstatscounter_tcp6connectfail, isc_sockstatscounter_tcp6connect,
isc_sockstatscounter_tcp6acceptfail, isc_sockstatscounter_tcp6accept,
isc_sockstatscounter_tcp6sendfail, isc_sockstatscounter_tcp6recvfail,
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
isc_sockstatscounter_tcp6active
};
#if 0
/* XXX: not currently used */
static const isc_statscounter_t unixstatsindex[] = {
isc_sockstatscounter_unixopen,
isc_sockstatscounter_unixopenfail,
isc_sockstatscounter_unixclose,
isc_sockstatscounter_unixbindfail,
isc_sockstatscounter_unixconnectfail,
isc_sockstatscounter_unixconnect,
isc_sockstatscounter_unixacceptfail,
isc_sockstatscounter_unixaccept,
isc_sockstatscounter_unixsendfail,
isc_sockstatscounter_unixrecvfail,
isc_sockstatscounter_unixactive
};
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
#endif /* if 0 */
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
* libuv is not thread safe, but has mechanisms to pass messages
* between threads. Each socket is owned by a thread. For UDP
* sockets we have a set of sockets for each interface and we can
* choose a sibling and send the message directly. For TCP, or if
* we're calling from a non-networking thread, we need to pass the
* request using async_cb.
*/
Complete rewrite the BIND 9 build system The rewrite of BIND 9 build system is a large work and cannot be reasonable split into separate merge requests. Addition of the automake has a positive effect on the readability and maintainability of the build system as it is more declarative, it allows conditional and we are able to drop all of the custom make code that BIND 9 developed over the years to overcome the deficiencies of autoconf + custom Makefile.in files. This squashed commit contains following changes: - conversion (or rather fresh rewrite) of all Makefile.in files to Makefile.am by using automake - the libtool is now properly integrated with automake (the way we used it was rather hackish as the only official way how to use libtool is via automake - the dynamic module loading was rewritten from a custom patchwork to libtool's libltdl (which includes the patchwork to support module loading on different systems internally) - conversion of the unit test executor from kyua to automake parallel driver - conversion of the system test executor from custom make/shell to automake parallel driver - The GSSAPI has been refactored, the custom SPNEGO on the basis that all major KRB5/GSSAPI (mit-krb5, heimdal and Windows) implementations support SPNEGO mechanism. - The various defunct tests from bin/tests have been removed: bin/tests/optional and bin/tests/pkcs11 - The text files generated from the MD files have been removed, the MarkDown has been designed to be readable by both humans and computers - The xsl header is now generated by a simple sed command instead of perl helper - The <irs/platform.h> header has been removed - cleanups of configure.ac script to make it more simpler, addition of multiple macros (there's still work to be done though) - the tarball can now be prepared with `make dist` - the system tests are partially able to run in oot build Here's a list of unfinished work that needs to be completed in subsequent merge requests: - `make distcheck` doesn't yet work (because of system tests oot run is not yet finished) - documentation is not yet built, there's a different merge request with docbook to sphinx-build rst conversion that needs to be rebased and adapted on top of the automake - msvc build is non functional yet and we need to decide whether we will just cross-compile bind9 using mingw-w64 or fix the msvc build - contributed dlz modules are not included neither in the autoconf nor automake
2018-08-07 16:46:53 +02:00
static thread_local int isc__nm_tid_v = ISC_NETMGR_TID_UNKNOWN;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Reformat using the new rules
2020-02-14 08:14:03 +01:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_maybe_destroy(isc_nmsocket_t *sock FLARG);
Reformat using the new rules
2020-02-14 08:14:03 +01:00
static void
nmhandle_free(isc_nmsocket_t *sock, isc_nmhandle_t *handle);
static isc_threadresult_t
nm_thread(isc_threadarg_t worker0);
static void
async_cb(uv_async_t *handle);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static bool
Reformat using the new rules
2020-02-14 08:14:03 +01:00
process_queue(isc__networker_t *worker, isc_queue_t *queue);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static bool
process_priority_queue(isc__networker_t *worker);
static void
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
process_privilege_queue(isc__networker_t *worker);
static void
process_tasks_queue(isc__networker_t *worker);
static void
process_normal_queue(isc__networker_t *worker);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_async_stop(isc__networker_t *worker, isc__netievent_t *ev0);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_async_pause(isc__networker_t *worker, isc__netievent_t *ev0);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_async_resume(isc__networker_t *worker, isc__netievent_t *ev0);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
static void
isc__nm_async_detach(isc__networker_t *worker, isc__netievent_t *ev0);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
static void
isc__nm_async_close(isc__networker_t *worker, isc__netievent_t *ev0);
/*%<
* Issue a 'handle closed' callback on the socket.
*/
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmhandle_detach_cb(isc_nmhandle_t **handlep FLARG);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
int
Complete rewrite the BIND 9 build system The rewrite of BIND 9 build system is a large work and cannot be reasonable split into separate merge requests. Addition of the automake has a positive effect on the readability and maintainability of the build system as it is more declarative, it allows conditional and we are able to drop all of the custom make code that BIND 9 developed over the years to overcome the deficiencies of autoconf + custom Makefile.in files. This squashed commit contains following changes: - conversion (or rather fresh rewrite) of all Makefile.in files to Makefile.am by using automake - the libtool is now properly integrated with automake (the way we used it was rather hackish as the only official way how to use libtool is via automake - the dynamic module loading was rewritten from a custom patchwork to libtool's libltdl (which includes the patchwork to support module loading on different systems internally) - conversion of the unit test executor from kyua to automake parallel driver - conversion of the system test executor from custom make/shell to automake parallel driver - The GSSAPI has been refactored, the custom SPNEGO on the basis that all major KRB5/GSSAPI (mit-krb5, heimdal and Windows) implementations support SPNEGO mechanism. - The various defunct tests from bin/tests have been removed: bin/tests/optional and bin/tests/pkcs11 - The text files generated from the MD files have been removed, the MarkDown has been designed to be readable by both humans and computers - The xsl header is now generated by a simple sed command instead of perl helper - The <irs/platform.h> header has been removed - cleanups of configure.ac script to make it more simpler, addition of multiple macros (there's still work to be done though) - the tarball can now be prepared with `make dist` - the system tests are partially able to run in oot build Here's a list of unfinished work that needs to be completed in subsequent merge requests: - `make distcheck` doesn't yet work (because of system tests oot run is not yet finished) - documentation is not yet built, there's a different merge request with docbook to sphinx-build rst conversion that needs to be rebased and adapted on top of the automake - msvc build is non functional yet and we need to decide whether we will just cross-compile bind9 using mingw-w64 or fix the msvc build - contributed dlz modules are not included neither in the autoconf nor automake
2018-08-07 16:46:53 +02:00
isc_nm_tid(void) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
return (isc__nm_tid_v);
}
bool
Complete rewrite the BIND 9 build system The rewrite of BIND 9 build system is a large work and cannot be reasonable split into separate merge requests. Addition of the automake has a positive effect on the readability and maintainability of the build system as it is more declarative, it allows conditional and we are able to drop all of the custom make code that BIND 9 developed over the years to overcome the deficiencies of autoconf + custom Makefile.in files. This squashed commit contains following changes: - conversion (or rather fresh rewrite) of all Makefile.in files to Makefile.am by using automake - the libtool is now properly integrated with automake (the way we used it was rather hackish as the only official way how to use libtool is via automake - the dynamic module loading was rewritten from a custom patchwork to libtool's libltdl (which includes the patchwork to support module loading on different systems internally) - conversion of the unit test executor from kyua to automake parallel driver - conversion of the system test executor from custom make/shell to automake parallel driver - The GSSAPI has been refactored, the custom SPNEGO on the basis that all major KRB5/GSSAPI (mit-krb5, heimdal and Windows) implementations support SPNEGO mechanism. - The various defunct tests from bin/tests have been removed: bin/tests/optional and bin/tests/pkcs11 - The text files generated from the MD files have been removed, the MarkDown has been designed to be readable by both humans and computers - The xsl header is now generated by a simple sed command instead of perl helper - The <irs/platform.h> header has been removed - cleanups of configure.ac script to make it more simpler, addition of multiple macros (there's still work to be done though) - the tarball can now be prepared with `make dist` - the system tests are partially able to run in oot build Here's a list of unfinished work that needs to be completed in subsequent merge requests: - `make distcheck` doesn't yet work (because of system tests oot run is not yet finished) - documentation is not yet built, there's a different merge request with docbook to sphinx-build rst conversion that needs to be rebased and adapted on top of the automake - msvc build is non functional yet and we need to decide whether we will just cross-compile bind9 using mingw-w64 or fix the msvc build - contributed dlz modules are not included neither in the autoconf nor automake
2018-08-07 16:46:53 +02:00
isc__nm_in_netthread(void) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
return (isc__nm_tid_v >= 0);
}
Make netmgr initialize and cleanup Winsock itself On Windows, WSAStartup() needs to be called to initialize Winsock before any sockets are created or else socket() calls will return error code 10093 (WSANOTINITIALISED). Since BIND's Network Manager is intended to work as a reusable networking library, it should take care of calling WSAStartup() - and its cleanup counterpart, WSACleanup() - itself rather than relying on external code to do it. Add the necessary WSAStartup() and WSACleanup() calls to isc_nm_start() and isc_nm_destroy(), respectively.
2020-12-02 22:36:23 +01:00
#ifdef WIN32
static void
isc__nm_winsock_initialize(void) {
WORD wVersionRequested = MAKEWORD(2, 2);
WSADATA wsaData;
int result;
result = WSAStartup(wVersionRequested, &wsaData);
if (result != 0) {
char strbuf[ISC_STRERRORSIZE];
strerror_r(result, strbuf, sizeof(strbuf));
UNEXPECTED_ERROR(__FILE__, __LINE__,
"WSAStartup() failed with error code %lu: %s",
result, strbuf);
}
/*
* Confirm that the WinSock DLL supports version 2.2. Note that if the
* DLL supports versions greater than 2.2 in addition to 2.2, it will
* still return 2.2 in wVersion since that is the version we requested.
*/
if (LOBYTE(wsaData.wVersion) != 2 || HIBYTE(wsaData.wVersion) != 2) {
UNEXPECTED_ERROR(__FILE__, __LINE__,
"Unusable WinSock DLL version: %u.%u",
LOBYTE(wsaData.wVersion),
HIBYTE(wsaData.wVersion));
}
}
static void
isc__nm_winsock_destroy(void) {
WSACleanup();
}
#endif /* WIN32 */
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
void
isc__netmgr_create(isc_mem_t *mctx, uint32_t workers, isc_nm_t **netmgrp) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nm_t *mgr = NULL;
apply the modified style
2020-02-13 14:44:37 -08:00
char name[32];
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
REQUIRE(workers > 0);
Make netmgr initialize and cleanup Winsock itself On Windows, WSAStartup() needs to be called to initialize Winsock before any sockets are created or else socket() calls will return error code 10093 (WSANOTINITIALISED). Since BIND's Network Manager is intended to work as a reusable networking library, it should take care of calling WSAStartup() - and its cleanup counterpart, WSACleanup() - itself rather than relying on external code to do it. Add the necessary WSAStartup() and WSACleanup() calls to isc_nm_start() and isc_nm_destroy(), respectively.
2020-12-02 22:36:23 +01:00
#ifdef WIN32
isc__nm_winsock_initialize();
#endif /* WIN32 */
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
mgr = isc_mem_get(mctx, sizeof(*mgr));
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*mgr = (isc_nm_t){ .nworkers = workers };
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_mem_attach(mctx, &mgr->mctx);
isc_mutex_init(&mgr->lock);
isc_condition_init(&mgr->wkstatecond);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
isc_condition_init(&mgr->wkpausecond);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_refcount_init(&mgr->references, 1);
atomic_init(&mgr->maxudp, 0);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
atomic_init(&mgr->interlocked, ISC_NETMGR_NON_INTERLOCKED);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
ISC_LIST_INIT(mgr->active_sockets);
#endif
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
/*
* Default TCP timeout values.
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
* May be updated by isc_nm_tcptimeouts().
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
*/
Fix the data race in accessing the isc_nm_t timers The following TSAN report about accessing the mgr timers (mgr->init, mgr->idle, mgr->keepalive and mgr->advertised) has been fixed in this commit: ================== WARNING: ThreadSanitizer: data race (pid=2746) Read of size 4 at 0x7b440008a948 by thread T18: #0 isc__nm_tcpdns_read /home/ondrej/Projects/bind9/lib/isc/netmgr/tcpdns.c:849:25 (libisc.so.1706+0x2ba0f) #1 isc_nm_read /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1679:3 (libisc.so.1706+0x22258) #2 tcpdns_connect_connect_cb /home/ondrej/Projects/bind9/lib/isc/tests/tcpdns_test.c:363:2 (tcpdns_test+0x4bc5fb) #3 isc__nm_async_connectcb /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1816:2 (libisc.so.1706+0x228c9) #4 isc__nm_connectcb /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1791:3 (libisc.so.1706+0x22713) #5 tcpdns_connect_cb /home/ondrej/Projects/bind9/lib/isc/netmgr/tcpdns.c:343:2 (libisc.so.1706+0x2d89d) #6 uv__stream_connect /home/ondrej/Projects/tsan/libuv/src/unix/stream.c:1381:5 (libuv.so.1+0x27c18) #7 uv__stream_io /home/ondrej/Projects/tsan/libuv/src/unix/stream.c:1298:5 (libuv.so.1+0x25977) #8 uv__io_poll /home/ondrej/Projects/tsan/libuv/src/unix/linux-core.c:462:11 (libuv.so.1+0x2e795) #9 uv_run /home/ondrej/Projects/tsan/libuv/src/unix/core.c:385:5 (libuv.so.1+0x158ec) #10 nm_thread /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:530:11 (libisc.so.1706+0x1c94a) Previous write of size 4 at 0x7b440008a948 by main thread: #0 isc_nm_settimeouts /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:490:12 (libisc.so.1706+0x1dda5) #1 tcpdns_recv_two /home/ondrej/Projects/bind9/lib/isc/tests/tcpdns_test.c:601:2 (tcpdns_test+0x4bad0e) #2 cmocka_run_one_test_or_fixture <null> (libcmocka.so.0+0x70be) #3 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16 (libc.so.6+0x2409a) Location is heap block of size 281 at 0x7b440008a840 allocated by main thread: #0 malloc <null> (tcpdns_test+0x42864b) #1 default_memalloc /home/ondrej/Projects/bind9/lib/isc/mem.c:713:8 (libisc.so.1706+0x6d261) #2 mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:622:8 (libisc.so.1706+0x69b9c) #3 isc___mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:1044:9 (libisc.so.1706+0x6d379) #4 isc__mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:2432:10 (libisc.so.1706+0x6889e) #5 isc_nm_start /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:203:8 (libisc.so.1706+0x1c219) #6 nm_setup /home/ondrej/Projects/bind9/lib/isc/tests/tcpdns_test.c:244:11 (tcpdns_test+0x4baaa4) #7 cmocka_run_one_test_or_fixture <null> (libcmocka.so.0+0x70fd) #8 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16 (libc.so.6+0x2409a) Thread T18 'isc-net-0000' (tid=3513, running) created by main thread at: #0 pthread_create <null> (tcpdns_test+0x429e7b) #1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/pthreads/thread.c:73:8 (libisc.so.1706+0x8476a) #2 isc_nm_start /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:271:3 (libisc.so.1706+0x1c66a) #3 nm_setup /home/ondrej/Projects/bind9/lib/isc/tests/tcpdns_test.c:244:11 (tcpdns_test+0x4baaa4) #4 cmocka_run_one_test_or_fixture <null> (libcmocka.so.0+0x70fd) #5 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16 (libc.so.6+0x2409a) SUMMARY: ThreadSanitizer: data race /home/ondrej/Projects/bind9/lib/isc/netmgr/tcpdns.c:849:25 in isc__nm_tcpdns_read ================== ThreadSanitizer: reported 1 warnings
2020-12-02 09:52:39 +01:00
atomic_init(&mgr->init, 30000);
atomic_init(&mgr->idle, 30000);
atomic_init(&mgr->keepalive, 30000);
atomic_init(&mgr->advertised, 30000);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
isc_mutex_init(&mgr->reqlock);
isc_mempool_create(mgr->mctx, sizeof(isc__nm_uvreq_t), &mgr->reqpool);
isc_mempool_setname(mgr->reqpool, "nm_reqpool");
Don't limit the size of uvreq/nmhandle pool artificially. There was a hard limit set on number of uvreq and nmhandles that can be allocated by a pool, but we don't handle a situation where we can't get an uvreq. Don't limit the number at all, let the OS deal with it.
2020-02-10 14:00:36 +01:00
isc_mempool_setfreemax(mgr->reqpool, 4096);
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
isc_mempool_associatelock(mgr->reqpool, &mgr->reqlock);
isc_mempool_setfillcount(mgr->reqpool, 32);
isc_mutex_init(&mgr->evlock);
isc_mempool_create(mgr->mctx, sizeof(isc__netievent_storage_t),
&mgr->evpool);
isc_mempool_setname(mgr->evpool, "nm_evpool");
Don't limit the size of uvreq/nmhandle pool artificially. There was a hard limit set on number of uvreq and nmhandles that can be allocated by a pool, but we don't handle a situation where we can't get an uvreq. Don't limit the number at all, let the OS deal with it.
2020-02-10 14:00:36 +01:00
isc_mempool_setfreemax(mgr->evpool, 4096);
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
isc_mempool_associatelock(mgr->evpool, &mgr->evlock);
isc_mempool_setfillcount(mgr->evpool, 32);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
mgr->workers = isc_mem_get(mctx, workers * sizeof(isc__networker_t));
for (size_t i = 0; i < workers; i++) {
apply the modified style
2020-02-13 14:44:37 -08:00
int r;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__networker_t *worker = &mgr->workers[i];
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*worker = (isc__networker_t){
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
.mgr = mgr,
.id = i,
};
r = uv_loop_init(&worker->loop);
RUNTIME_CHECK(r == 0);
worker->loop.data = &mgr->workers[i];
r = uv_async_init(&worker->loop, &worker->async, async_cb);
RUNTIME_CHECK(r == 0);
isc_mutex_init(&worker->lock);
isc_condition_init(&worker->cond);
worker->ievents = isc_queue_new(mgr->mctx, 128);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
worker->ievents_priv = isc_queue_new(mgr->mctx, 128);
worker->ievents_task = isc_queue_new(mgr->mctx, 128);
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
worker->ievents_prio = isc_queue_new(mgr->mctx, 128);
Make nm->recvbuf larger and heap allocated, to allow uv_recvmmsg usage. Upcoming version of libuv will suport uv_recvmmsg and uv_sendmmsg. To use uv_recvmmsg we need to provide a larger buffer and be able to properly free it.
2020-01-29 13:16:04 +01:00
worker->recvbuf = isc_mem_get(mctx, ISC_NETMGR_RECVBUF_SIZE);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
worker->sendbuf = isc_mem_get(mctx, ISC_NETMGR_SENDBUF_SIZE);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
* We need to do this here and not in nm_thread to avoid a
* race - we could exit isc_nm_start, launch nm_destroy,
* and nm_thread would still not be up.
*/
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
mgr->workers_running++;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_thread_create(nm_thread, &mgr->workers[i], &worker->thread);
snprintf(name, sizeof(name), "isc-net-%04zu", i);
isc_thread_setname(worker->thread, name);
}
mgr->magic = NM_MAGIC;
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
*netmgrp = mgr;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
/*
* Free the resources of the network manager.
*/
static void
apply the modified style
2020-02-13 14:44:37 -08:00
nm_destroy(isc_nm_t **mgr0) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NM(*mgr0));
REQUIRE(!isc__nm_in_netthread());
isc_nm_t *mgr = *mgr0;
Clear the pointer to destroyed object early using the semantic patch Also disable the semantic patch as the code needs tweaks here and there because some destroy functions might not destroy the object and return early if the object is still in use.
2020-02-08 04:37:54 -08:00
*mgr0 = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Add isc_refcount_destroy() call to nm_destroy()
2019-12-10 10:47:08 +01:00
isc_refcount_destroy(&mgr->references);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
mgr->magic = 0;
for (size_t i = 0; i < mgr->nworkers; i++) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc__networker_t *worker = &mgr->workers[i];
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__netievent_t *event = isc__nm_get_netievent_stop(mgr);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc__nm_enqueue_ievent(worker, event);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
address lock order inversion
2019-12-12 17:43:03 +11:00
LOCK(&mgr->lock);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
while (mgr->workers_running > 0) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
WAIT(&mgr->wkstatecond, &mgr->lock);
}
UNLOCK(&mgr->lock);
for (size_t i = 0; i < mgr->nworkers; i++) {
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
isc__networker_t *worker = &mgr->workers[i];
style, comments
2019-12-03 00:07:59 -08:00
isc__netievent_t *ievent = NULL;
apply the modified style
2020-02-13 14:44:37 -08:00
int r;
style, comments
2019-12-03 00:07:59 -08:00
/* Empty the async event queues */
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
while ((ievent = (isc__netievent_t *)isc_queue_dequeue(
apply the modified style
2020-02-13 14:44:37 -08:00
worker->ievents)) != NULL)
{
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
isc_mempool_put(mgr->evpool, ievent);
}
style, comments
2019-12-03 00:07:59 -08:00
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
INSIST(isc_queue_dequeue(worker->ievents_priv) ==
(uintptr_t)NULL);
INSIST(isc_queue_dequeue(worker->ievents_task) ==
(uintptr_t)NULL);
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
while ((ievent = (isc__netievent_t *)isc_queue_dequeue(
apply the modified style
2020-02-13 14:44:37 -08:00
worker->ievents_prio)) != NULL)
{
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
isc_mempool_put(mgr->evpool, ievent);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
style, comments
2019-12-03 00:07:59 -08:00
r = uv_loop_close(&worker->loop);
netmgr: close uv_loop properly close the uv_handle for the worker async channel, and call uv_loop_close() on shutdown to ensure that the event loop's internal resources are properly freed.
2019-11-15 12:15:03 +01:00
INSIST(r == 0);
style, comments
2019-12-03 00:07:59 -08:00
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
isc_queue_destroy(worker->ievents);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
isc_queue_destroy(worker->ievents_priv);
isc_queue_destroy(worker->ievents_task);
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
isc_queue_destroy(worker->ievents_prio);
Add missing isc_mutex_destroy and isc_conditional_destroy calls. While harmless on Linux, missing isc_{mutex,conditional}_destroy causes a memory leak on *BSD. Missing calls were added.
2020-05-28 12:34:37 +02:00
isc_mutex_destroy(&worker->lock);
isc_condition_destroy(&worker->cond);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
isc_mem_put(mgr->mctx, worker->sendbuf,
ISC_NETMGR_SENDBUF_SIZE);
Make nm->recvbuf larger and heap allocated, to allow uv_recvmmsg usage. Upcoming version of libuv will suport uv_recvmmsg and uv_sendmmsg. To use uv_recvmmsg we need to provide a larger buffer and be able to properly free it.
2020-01-29 13:16:04 +01:00
isc_mem_put(mgr->mctx, worker->recvbuf,
ISC_NETMGR_RECVBUF_SIZE);
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
isc_thread_join(worker->thread, NULL);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
if (mgr->stats != NULL) {
isc_stats_detach(&mgr->stats);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_condition_destroy(&mgr->wkstatecond);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
isc_condition_destroy(&mgr->wkpausecond);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_mutex_destroy(&mgr->lock);
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
isc_mempool_destroy(&mgr->evpool);
isc_mutex_destroy(&mgr->evlock);
isc_mempool_destroy(&mgr->reqpool);
isc_mutex_destroy(&mgr->reqlock);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_mem_put(mgr->mctx, mgr->workers,
mgr->nworkers * sizeof(isc__networker_t));
isc_mem_putanddetach(&mgr->mctx, mgr, sizeof(*mgr));
Make netmgr initialize and cleanup Winsock itself On Windows, WSAStartup() needs to be called to initialize Winsock before any sockets are created or else socket() calls will return error code 10093 (WSANOTINITIALISED). Since BIND's Network Manager is intended to work as a reusable networking library, it should take care of calling WSAStartup() - and its cleanup counterpart, WSACleanup() - itself rather than relying on external code to do it. Add the necessary WSAStartup() and WSACleanup() calls to isc_nm_start() and isc_nm_destroy(), respectively.
2020-12-02 22:36:23 +01:00
#ifdef WIN32
isc__nm_winsock_destroy();
#endif /* WIN32 */
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_pause(isc_nm_t *mgr) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NM(mgr));
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
uint_fast32_t pausing = 0;
REQUIRE(!atomic_load(&mgr->paused));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_acquire_interlocked_force(mgr);
for (size_t i = 0; i < mgr->nworkers; i++) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc__networker_t *worker = &mgr->workers[i];
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
if (i != (size_t)isc_nm_tid()) {
isc__netievent_resume_t *event =
isc__nm_get_netievent_pause(mgr);
pausing++;
isc__nm_enqueue_ievent(worker,
(isc__netievent_t *)event);
} else {
isc__nm_async_pause(worker, NULL);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
LOCK(&mgr->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
while (mgr->workers_paused != pausing) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
WAIT(&mgr->wkstatecond, &mgr->lock);
}
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
REQUIRE(atomic_compare_exchange_strong(&mgr->paused, &(bool){ false },
true));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
UNLOCK(&mgr->lock);
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_resume(isc_nm_t *mgr) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NM(mgr));
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
REQUIRE(atomic_load(&mgr->paused));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
for (size_t i = 0; i < mgr->nworkers; i++) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc__networker_t *worker = &mgr->workers[i];
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
if (i != (size_t)isc_nm_tid()) {
isc__netievent_resume_t *event =
isc__nm_get_netievent_resume(mgr);
isc__nm_enqueue_ievent(worker,
(isc__netievent_t *)event);
} else {
isc__nm_async_resume(worker, NULL);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
LOCK(&mgr->lock);
while (mgr->workers_paused != 0) {
WAIT(&mgr->wkstatecond, &mgr->lock);
}
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
REQUIRE(atomic_compare_exchange_strong(&mgr->paused, &(bool){ true },
false));
BROADCAST(&mgr->wkpausecond);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
UNLOCK(&mgr->lock);
isc__nm_drop_interlocked(mgr);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_attach(isc_nm_t *mgr, isc_nm_t **dst) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NM(mgr));
REQUIRE(dst != NULL && *dst == NULL);
Use isc_refcount_increment0() when reusing handle or socket; remove extra DbC checks
2020-01-14 09:43:37 +01:00
isc_refcount_increment(&mgr->references);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*dst = mgr;
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_detach(isc_nm_t **mgr0) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nm_t *mgr = NULL;
REQUIRE(mgr0 != NULL);
REQUIRE(VALID_NM(*mgr0));
mgr = *mgr0;
*mgr0 = NULL;
Use isc_refcount_increment0() when reusing handle or socket; remove extra DbC checks
2020-01-14 09:43:37 +01:00
if (isc_refcount_decrement(&mgr->references) == 1) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
nm_destroy(&mgr);
}
}
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_closedown(isc_nm_t *mgr) {
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
REQUIRE(VALID_NM(mgr));
atomic_store(&mgr->closing, true);
for (size_t i = 0; i < mgr->nworkers; i++) {
isc__netievent_t *event = NULL;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
event = isc__nm_get_netievent_shutdown(mgr);
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
isc__nm_enqueue_ievent(&mgr->workers[i], event);
}
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
isc__netmgr_destroy(isc_nm_t **netmgrp) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nm_t *mgr = NULL;
limit the time we wait for netmgr to be destroyed if more than 10 seconds pass while we wait for netmgr events to finish running on shutdown, something is almost certainly wrong and we should assert and crash.
2020-07-15 17:57:58 -07:00
int counter = 0;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
REQUIRE(VALID_NM(*netmgrp));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
mgr = *netmgrp;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
/*
* Close active connections.
*/
isc_nm_closedown(mgr);
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
* Wait for the manager to be dereferenced elsewhere.
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*/
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
while (isc_refcount_current(&mgr->references) > 1 && counter++ < 1000) {
Use uv_sleep in the netmgr code libuv added uv_sleep(unsigned int msec) to the API since 1.34.0. Use that in the netmgr code and define usleep based shim for libuv << 1.34.0.
2021-05-03 12:10:03 +02:00
uv_sleep(10);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
#ifdef NETMGR_TRACE
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
if (isc_refcount_current(&mgr->references) > 1) {
isc__nm_dump_active(mgr);
INSIST(0);
ISC_UNREACHABLE();
}
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
#endif
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
/*
* Now just patiently wait
*/
while (isc_refcount_current(&mgr->references) > 1) {
Use uv_sleep in the netmgr code libuv added uv_sleep(unsigned int msec) to the API since 1.34.0. Use that in the netmgr code and define usleep based shim for libuv << 1.34.0.
2021-05-03 12:10:03 +02:00
uv_sleep(10);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
}
limit the time we wait for netmgr to be destroyed if more than 10 seconds pass while we wait for netmgr events to finish running on shutdown, something is almost certainly wrong and we should assert and crash.
2020-07-15 17:57:58 -07:00
netmgr: add shutdown function - new function isc_nm_shutdown() shuts down all active TCP connections, but does not destroy the netmgr.
2019-11-22 15:57:42 -08:00
/*
* Detach final reference.
*/
Add new isc_managers API to simplify <*>mgr create/destroy Previously, netmgr, taskmgr, timermgr and socketmgr all had their own isc_<*>mgr_create() and isc_<*>mgr_destroy() functions. The new isc_managers_create() and isc_managers_destroy() fold all four into a single function and makes sure the objects are created and destroy in correct order. Especially now, when taskmgr runs on top of netmgr, the correct order is important and when the code was duplicated at many places it's easy to make mistake. The former isc_<*>mgr_create() and isc_<*>mgr_destroy() functions were made private and a single call to isc_managers_create() and isc_managers_destroy() is required at the program startup / shutdown.
2021-04-27 00:07:43 +02:00
isc_nm_detach(netmgrp);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_maxudp(isc_nm_t *mgr, uint32_t maxudp) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NM(mgr));
atomic_store(&mgr->maxudp, maxudp);
}
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_nm_settimeouts(isc_nm_t *mgr, uint32_t init, uint32_t idle,
uint32_t keepalive, uint32_t advertised) {
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
REQUIRE(VALID_NM(mgr));
Change the isc_nm_(get|set)timeouts() to work with milliseconds The RFC7828 specifies the keepalive interval to be 16-bit, specified in units of 100 milliseconds and the configuration options tcp-*-timeouts are following the suit. The units of 100 milliseconds are very unintuitive and while we can't change the configuration and presentation format, we should not follow this weird unit in the API. This commit changes the isc_nm_(get|set)timeouts() functions to work with milliseconds and convert the values to milliseconds before passing them to the function, not just internally.
2021-03-18 11:16:45 +01:00
atomic_store(&mgr->init, init);
atomic_store(&mgr->idle, idle);
atomic_store(&mgr->keepalive, keepalive);
atomic_store(&mgr->advertised, advertised);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
}
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_nm_gettimeouts(isc_nm_t *mgr, uint32_t *initial, uint32_t *idle,
uint32_t *keepalive, uint32_t *advertised) {
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
REQUIRE(VALID_NM(mgr));
if (initial != NULL) {
Change the isc_nm_(get|set)timeouts() to work with milliseconds The RFC7828 specifies the keepalive interval to be 16-bit, specified in units of 100 milliseconds and the configuration options tcp-*-timeouts are following the suit. The units of 100 milliseconds are very unintuitive and while we can't change the configuration and presentation format, we should not follow this weird unit in the API. This commit changes the isc_nm_(get|set)timeouts() functions to work with milliseconds and convert the values to milliseconds before passing them to the function, not just internally.
2021-03-18 11:16:45 +01:00
*initial = atomic_load(&mgr->init);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
}
if (idle != NULL) {
Change the isc_nm_(get|set)timeouts() to work with milliseconds The RFC7828 specifies the keepalive interval to be 16-bit, specified in units of 100 milliseconds and the configuration options tcp-*-timeouts are following the suit. The units of 100 milliseconds are very unintuitive and while we can't change the configuration and presentation format, we should not follow this weird unit in the API. This commit changes the isc_nm_(get|set)timeouts() functions to work with milliseconds and convert the values to milliseconds before passing them to the function, not just internally.
2021-03-18 11:16:45 +01:00
*idle = atomic_load(&mgr->idle);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
}
if (keepalive != NULL) {
Change the isc_nm_(get|set)timeouts() to work with milliseconds The RFC7828 specifies the keepalive interval to be 16-bit, specified in units of 100 milliseconds and the configuration options tcp-*-timeouts are following the suit. The units of 100 milliseconds are very unintuitive and while we can't change the configuration and presentation format, we should not follow this weird unit in the API. This commit changes the isc_nm_(get|set)timeouts() functions to work with milliseconds and convert the values to milliseconds before passing them to the function, not just internally.
2021-03-18 11:16:45 +01:00
*keepalive = atomic_load(&mgr->keepalive);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
}
if (advertised != NULL) {
Change the isc_nm_(get|set)timeouts() to work with milliseconds The RFC7828 specifies the keepalive interval to be 16-bit, specified in units of 100 milliseconds and the configuration options tcp-*-timeouts are following the suit. The units of 100 milliseconds are very unintuitive and while we can't change the configuration and presentation format, we should not follow this weird unit in the API. This commit changes the isc_nm_(get|set)timeouts() functions to work with milliseconds and convert the values to milliseconds before passing them to the function, not just internally.
2021-03-18 11:16:45 +01:00
*advertised = atomic_load(&mgr->advertised);
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
}
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
* nm_thread is a single worker thread, that runs uv_run event loop
* until asked to stop.
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
*
* There are four queues for asynchronous events:
*
* 1. priority queue - netievents on the priority queue are run even when
* the taskmgr enters exclusive mode and the netmgr is paused. This
* is needed to properly start listening on the interfaces, free
* resources on shutdown, or resume from a pause.
*
* 2. privileged task queue - only privileged tasks are queued here and
* this is the first queue that gets processed when network manager
* is unpaused using isc_nm_resume(). All netmgr workers need to
* clean the privileged task queue before they all proceed to normal
* operation. Both task queues are processed when the workers are
* shutting down.
*
* 3. task queue - only (traditional) tasks are scheduled here, and this
* queue and the privileged task queue are both processed when the
* netmgr workers are finishing. This is needed to process the task
* shutdown events.
*
* 4. normal queue - this is the queue with netmgr events, e.g. reading,
* sending, callbacks, etc.
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*/
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
Use isc_threadresult_t instead of pthread specific void * return type The ISC thread API already defines isc_threadresult_t type, but we are using a pthread specific return type (void *).
2019-12-04 10:41:40 +01:00
static isc_threadresult_t
apply the modified style
2020-02-13 14:44:37 -08:00
nm_thread(isc_threadarg_t worker0) {
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
isc__networker_t *worker = (isc__networker_t *)worker0;
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
isc_nm_t *mgr = worker->mgr;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_tid_v = worker->id;
isc_thread_setaffinity(isc__nm_tid_v);
while (true) {
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
/*
* uv_run() runs async_cb() in a loop, which processes
* all four event queues until a "pause" or "stop" event
* is encountered. On pause, we process only priority and
* privileged events until resuming.
*/
apply the modified style
2020-02-13 14:44:37 -08:00
int r = uv_run(&worker->loop, UV_RUN_DEFAULT);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
INSIST(r > 0 || worker->finished);
if (worker->paused) {
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
INSIST(atomic_load(&mgr->interlocked) != isc_nm_tid());
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
/*
* We need to lock the worker first; otherwise
* isc_nm_resume() might slip in before WAIT() in
* the while loop starts, then the signal never
* gets delivered and we are stuck forever in the
* paused loop.
*/
LOCK(&worker->lock);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
LOCK(&mgr->lock);
mgr->workers_paused++;
SIGNAL(&mgr->wkstatecond);
UNLOCK(&mgr->lock);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
while (worker->paused) {
WAIT(&worker->cond, &worker->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
UNLOCK(&worker->lock);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
(void)process_priority_queue(worker);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
LOCK(&worker->lock);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
LOCK(&mgr->lock);
mgr->workers_paused--;
SIGNAL(&mgr->wkstatecond);
UNLOCK(&mgr->lock);
UNLOCK(&worker->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
/*
* All workers must run the privileged event
* queue before we resume from pause.
*/
process_privilege_queue(worker);
LOCK(&mgr->lock);
while (atomic_load(&mgr->paused)) {
WAIT(&mgr->wkpausecond, &mgr->lock);
}
UNLOCK(&mgr->lock);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
if (r == 0) {
INSIST(worker->finished);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
break;
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
INSIST(!worker->finished);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
* We've fully resumed from pause. Drain the normal
* asynchronous event queues before resuming the uv_run()
* loop. (This is not strictly necessary, it just ensures
* that all pending events are processed before another
* pause can slip in.)
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*/
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
process_tasks_queue(worker);
process_normal_queue(worker);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
/*
* We are shutting down. Process the task queues
* (they may include shutdown events) but do not process
* the netmgr event queue.
*/
process_privilege_queue(worker);
process_tasks_queue(worker);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
LOCK(&mgr->lock);
mgr->workers_running--;
SIGNAL(&mgr->wkstatecond);
UNLOCK(&mgr->lock);
Use isc_threadresult_t instead of pthread specific void * return type The ISC thread API already defines isc_threadresult_t type, but we are using a pthread specific return type (void *).
2019-12-04 10:41:40 +01:00
return ((isc_threadresult_t)0);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
/*
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
* async_cb() is a universal callback for 'async' events sent to event loop.
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
* It's the only way to safely pass data to the libuv event loop. We use a
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
* single async event and a set of lockless queues of 'isc__netievent_t'
* structures passed from other threads.
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*/
static void
apply the modified style
2020-02-13 14:44:37 -08:00
async_cb(uv_async_t *handle) {
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
isc__networker_t *worker = (isc__networker_t *)handle->loop->data;
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
/*
* process_priority_queue() returns false when pausing or stopping,
* so we don't want to process the other queues in that case.
*/
if (!process_priority_queue(worker)) {
return;
}
process_privilege_queue(worker);
process_tasks_queue(worker);
process_normal_queue(worker);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
}
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_async_stop(isc__networker_t *worker, isc__netievent_t *ev0) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
UNUSED(ev0);
worker->finished = true;
/* Close the async handler */
uv_close((uv_handle_t *)&worker->async, NULL);
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
}
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_async_pause(isc__networker_t *worker, isc__netievent_t *ev0) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
UNUSED(ev0);
REQUIRE(worker->paused == false);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
worker->paused = true;
uv_stop(&worker->loop);
}
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_async_resume(isc__networker_t *worker, isc__netievent_t *ev0) {
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
UNUSED(ev0);
REQUIRE(worker->paused == true);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
worker->paused = false;
}
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
void
isc_nm_task_enqueue(isc_nm_t *nm, isc_task_t *task, int threadid) {
isc__netievent_t *event = NULL;
int tid;
isc__networker_t *worker = NULL;
if (threadid == -1) {
tid = (int)isc_random_uniform(nm->nworkers);
} else {
tid = threadid % nm->nworkers;
}
worker = &nm->workers[tid];
if (isc_task_privilege(task)) {
event = (isc__netievent_t *)
isc__nm_get_netievent_privilegedtask(nm, task);
} else {
event = (isc__netievent_t *)isc__nm_get_netievent_task(nm,
task);
}
isc__nm_enqueue_ievent(worker, event);
}
#define isc__nm_async_privilegedtask(worker, ev0) \
isc__nm_async_task(worker, ev0)
static void
isc__nm_async_task(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_task_t *ievent = (isc__netievent_task_t *)ev0;
isc_result_t result;
UNUSED(worker);
result = isc_task_run(ievent->task);
switch (result) {
case ISC_R_QUOTA:
isc_nm_task_enqueue(worker->mgr, (isc_task_t *)ievent->task,
isc_nm_tid());
return;
case ISC_R_SUCCESS:
return;
default:
INSIST(0);
ISC_UNREACHABLE();
}
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static bool
process_priority_queue(isc__networker_t *worker) {
return (process_queue(worker, worker->ievents_prio));
}
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
static void
process_privilege_queue(isc__networker_t *worker) {
(void)process_queue(worker, worker->ievents_priv);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
}
static void
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
process_tasks_queue(isc__networker_t *worker) {
(void)process_queue(worker, worker->ievents_task);
}
static void
process_normal_queue(isc__networker_t *worker) {
(void)process_queue(worker, worker->ievents);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
/*
* The two macros here generate the individual cases for the process_netievent()
* function. The NETIEVENT_CASE(type) macro is the common case, and
* NETIEVENT_CASE_NOMORE(type) is a macro that causes the loop in the
* process_queue() to stop, e.g. it's only used for the netievent that
* stops/pauses processing the enqueued netievents.
*/
#define NETIEVENT_CASE(type) \
case netievent_##type: { \
isc__nm_async_##type(worker, ievent); \
isc__nm_put_netievent_##type( \
worker->mgr, (isc__netievent_##type##_t *)ievent); \
return (true); \
}
#define NETIEVENT_CASE_NOMORE(type) \
case netievent_##type: { \
isc__nm_async_##type(worker, ievent); \
isc__nm_put_netievent_##type(worker->mgr, ievent); \
return (false); \
}
static bool
process_netievent(isc__networker_t *worker, isc__netievent_t *ievent) {
REQUIRE(worker->id == isc_nm_tid());
switch (ievent->type) {
/* Don't process more ievents when we are stopping */
NETIEVENT_CASE_NOMORE(stop);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
NETIEVENT_CASE(privilegedtask);
NETIEVENT_CASE(task);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_CASE(udpconnect);
NETIEVENT_CASE(udplisten);
NETIEVENT_CASE(udpstop);
NETIEVENT_CASE(udpsend);
NETIEVENT_CASE(udpread);
NETIEVENT_CASE(udpcancel);
NETIEVENT_CASE(udpclose);
NETIEVENT_CASE(tcpaccept);
NETIEVENT_CASE(tcpconnect);
NETIEVENT_CASE(tcplisten);
NETIEVENT_CASE(tcpstartread);
NETIEVENT_CASE(tcppauseread);
NETIEVENT_CASE(tcpsend);
NETIEVENT_CASE(tcpstop);
NETIEVENT_CASE(tcpcancel);
NETIEVENT_CASE(tcpclose);
NETIEVENT_CASE(tcpdnsaccept);
NETIEVENT_CASE(tcpdnslisten);
NETIEVENT_CASE(tcpdnsconnect);
NETIEVENT_CASE(tcpdnssend);
NETIEVENT_CASE(tcpdnscancel);
NETIEVENT_CASE(tcpdnsclose);
NETIEVENT_CASE(tcpdnsread);
NETIEVENT_CASE(tcpdnsstop);
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
NETIEVENT_CASE(tlsstartread);
NETIEVENT_CASE(tlssend);
NETIEVENT_CASE(tlsclose);
NETIEVENT_CASE(tlsdobio);
NETIEVENT_CASE(tlscancel);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
NETIEVENT_CASE(tlsdnscycle);
NETIEVENT_CASE(tlsdnsaccept);
NETIEVENT_CASE(tlsdnslisten);
NETIEVENT_CASE(tlsdnsconnect);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_CASE(tlsdnssend);
NETIEVENT_CASE(tlsdnscancel);
NETIEVENT_CASE(tlsdnsclose);
NETIEVENT_CASE(tlsdnsread);
NETIEVENT_CASE(tlsdnsstop);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
NETIEVENT_CASE(tlsdnsshutdown);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
NETIEVENT_CASE(httpstop);
NETIEVENT_CASE(httpsend);
NETIEVENT_CASE(httpclose);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_CASE(connectcb);
NETIEVENT_CASE(readcb);
NETIEVENT_CASE(sendcb);
NETIEVENT_CASE(close);
NETIEVENT_CASE(detach);
NETIEVENT_CASE(shutdown);
NETIEVENT_CASE(resume);
NETIEVENT_CASE_NOMORE(pause);
default:
INSIST(0);
ISC_UNREACHABLE();
}
return (true);
}
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
static bool
apply the modified style
2020-02-13 14:44:37 -08:00
process_queue(isc__networker_t *worker, isc_queue_t *queue) {
style, comments
2019-12-03 00:07:59 -08:00
isc__netievent_t *ievent = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
apply the modified style
2020-02-13 14:44:37 -08:00
while ((ievent = (isc__netievent_t *)isc_queue_dequeue(queue)) != NULL)
{
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
if (!process_netievent(worker, ievent)) {
return (false);
Refactor the pausing/unpausing and finishing the nm_thread The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from nm_destroy() has been refactored, so all use the netievents instead of directly touching the worker structure members. This allows us to remove most of the locking as the .paused and .finished members are always accessed from the matching nm_thread. When shutting down the nm_thread(), instead of issuing uv_stop(), we just shutdown the .async handler, so all uv_loop_t events are properly finished first and uv_run() ends gracefully with no outstanding active handles in the loop.
2020-09-23 21:49:46 +02:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
return (true);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void *
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_get_netievent(isc_nm_t *mgr, isc__netievent_type type) {
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
isc__netievent_storage_t *event = isc_mempool_get(mgr->evpool);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*event = (isc__netievent_storage_t){ .ni.type = type };
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
return (event);
}
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_put_netievent(isc_nm_t *mgr, void *ievent) {
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
isc_mempool_put(mgr->evpool, ievent);
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_DEF(tcpclose);
NETIEVENT_SOCKET_DEF(tcplisten);
NETIEVENT_SOCKET_DEF(tcppauseread);
NETIEVENT_SOCKET_DEF(tcpstartread);
NETIEVENT_SOCKET_DEF(tcpstop);
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
NETIEVENT_SOCKET_DEF(tlsclose);
NETIEVENT_SOCKET_DEF(tlsconnect);
NETIEVENT_SOCKET_DEF(tlsdobio);
NETIEVENT_SOCKET_DEF(tlsstartread);
NETIEVENT_SOCKET_HANDLE_DEF(tlscancel);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_DEF(udpclose);
NETIEVENT_SOCKET_DEF(udplisten);
NETIEVENT_SOCKET_DEF(udpread);
NETIEVENT_SOCKET_DEF(udpsend);
NETIEVENT_SOCKET_DEF(udpstop);
NETIEVENT_SOCKET_DEF(tcpdnsclose);
NETIEVENT_SOCKET_DEF(tcpdnsread);
NETIEVENT_SOCKET_DEF(tcpdnsstop);
NETIEVENT_SOCKET_DEF(tcpdnslisten);
NETIEVENT_SOCKET_REQ_DEF(tcpdnsconnect);
NETIEVENT_SOCKET_REQ_DEF(tcpdnssend);
NETIEVENT_SOCKET_HANDLE_DEF(tcpdnscancel);
NETIEVENT_SOCKET_QUOTA_DEF(tcpdnsaccept);
NETIEVENT_SOCKET_DEF(tlsdnsclose);
NETIEVENT_SOCKET_DEF(tlsdnsread);
NETIEVENT_SOCKET_DEF(tlsdnsstop);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
NETIEVENT_SOCKET_DEF(tlsdnslisten);
NETIEVENT_SOCKET_REQ_DEF(tlsdnsconnect);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_REQ_DEF(tlsdnssend);
NETIEVENT_SOCKET_HANDLE_DEF(tlsdnscancel);
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
NETIEVENT_SOCKET_QUOTA_DEF(tlsdnsaccept);
NETIEVENT_SOCKET_DEF(tlsdnscycle);
NETIEVENT_SOCKET_DEF(tlsdnsshutdown);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
NETIEVENT_SOCKET_DEF(httpstop);
NETIEVENT_SOCKET_REQ_DEF(httpsend);
NETIEVENT_SOCKET_DEF(httpclose);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_REQ_DEF(tcpconnect);
NETIEVENT_SOCKET_REQ_DEF(tcpsend);
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
NETIEVENT_SOCKET_REQ_DEF(tlssend);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETIEVENT_SOCKET_REQ_DEF(udpconnect);
NETIEVENT_SOCKET_REQ_RESULT_DEF(connectcb);
NETIEVENT_SOCKET_REQ_RESULT_DEF(readcb);
NETIEVENT_SOCKET_REQ_RESULT_DEF(sendcb);
NETIEVENT_SOCKET_DEF(detach);
NETIEVENT_SOCKET_HANDLE_DEF(tcpcancel);
NETIEVENT_SOCKET_HANDLE_DEF(udpcancel);
NETIEVENT_SOCKET_QUOTA_DEF(tcpaccept);
NETIEVENT_SOCKET_DEF(close);
NETIEVENT_DEF(pause);
NETIEVENT_DEF(resume);
NETIEVENT_DEF(shutdown);
NETIEVENT_DEF(stop);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
NETIEVENT_TASK_DEF(task);
NETIEVENT_TASK_DEF(privilegedtask);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
void
isc__nm_maybe_enqueue_ievent(isc__networker_t *worker,
isc__netievent_t *event) {
/*
* If we are already in the matching nmthread, process the ievent
* directly.
*/
if (worker->id == isc_nm_tid()) {
process_netievent(worker, event);
return;
}
isc__nm_enqueue_ievent(worker, event);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_enqueue_ievent(isc__networker_t *worker, isc__netievent_t *event) {
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
if (event->type > netievent_prio) {
/*
* We need to make sure this signal will be delivered and
* the queue will be processed.
*/
LOCK(&worker->lock);
isc_queue_enqueue(worker->ievents_prio, (uintptr_t)event);
SIGNAL(&worker->cond);
UNLOCK(&worker->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
} else if (event->type == netievent_privilegedtask) {
isc_queue_enqueue(worker->ievents_priv, (uintptr_t)event);
} else if (event->type == netievent_task) {
isc_queue_enqueue(worker->ievents_task, (uintptr_t)event);
- Add separate priority event queue for events that must be processed even when worker is paused (e.g. interface reconfiguration). This is needed to prevent deadlocks when reconfiguring interfaces - as network manager is paused then, but we still need to stop/start listening. - Proper handling of TCP listen errors in netmgr - bind to the socket first, then return the error code.
2019-12-02 13:54:44 +01:00
} else {
isc_queue_enqueue(worker->ievents, (uintptr_t)event);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
uv_async_send(&worker->async);
}
netmgr: don't send to an inactive (closing) udp socket We had a race in which n UDP socket could have been already closing by libuv but we still sent data to it. Mark socket as not-active when stopping listening and verify that socket is not active when trying to send data to it.
2020-01-16 11:52:58 +01:00
bool
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nmsocket_active(isc_nmsocket_t *sock) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMSOCK(sock));
if (sock->parent != NULL) {
return (atomic_load(&sock->parent->active));
}
return (atomic_load(&sock->active));
}
Fix the data race when read-writing sock->active by using cmpxchg
2020-10-22 10:07:56 +02:00
bool
isc__nmsocket_deactivate(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
if (sock->parent != NULL) {
return (atomic_compare_exchange_strong(&sock->parent->active,
&(bool){ true }, false));
}
return (atomic_compare_exchange_strong(&sock->active, &(bool){ true },
false));
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_attach(isc_nmsocket_t *sock, isc_nmsocket_t **target FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(target != NULL && *target == NULL);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_nmsocket_t *rsock = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (sock->parent != NULL) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
rsock = sock->parent;
INSIST(rsock->parent == NULL); /* sanity check */
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
} else {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
rsock = sock;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc__nmsocket_attach():%p->references = %" PRIuFAST32
"\n",
rsock, isc_refcount_current(&rsock->references) + 1);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_refcount_increment0(&rsock->references);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
*target = sock;
}
/*
* Free all resources inside a socket (including its children if any).
*/
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_cleanup(isc_nmsocket_t *sock, bool dofree FLARG) {
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_t *handle = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_uvreq_t *uvreq = NULL;
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(!isc__nmsocket_active(sock));
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("nmsocket_cleanup():%p->references = %" PRIuFAST32
"\n",
sock, isc_refcount_current(&sock->references));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
atomic_store(&sock->destroying, true);
if (sock->parent == NULL && sock->children != NULL) {
/*
* We shouldn't be here unless there are no active handles,
* so we can clean up and free the children.
*/
Use sock->nchildren instead of mgr->nworkers when initializing NM On Windows, we were limiting the number of listening children to just 1, but we were then iterating on mgr->nworkers. That lead to scheduling more async_*listen() than actually allocated and out-of-bound read-write operation on the heap.
2020-12-03 17:58:10 +01:00
for (size_t i = 0; i < sock->nchildren; i++) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (!atomic_load(&sock->children[i].destroying)) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_cleanup(&sock->children[i],
false FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
}
/*
* This was a parent socket; free the children.
*/
isc_mem_put(sock->mgr->mctx, sock->children,
sock->nchildren * sizeof(*sock));
sock->children = NULL;
sock->nchildren = 0;
}
Proper accounting of active TCP connections
2020-02-28 11:57:51 +01:00
if (sock->statsindex != NULL) {
Limit TCP connection quota logging to 1/s
2020-02-28 13:58:13 +01:00
isc__nm_decstats(sock->mgr, sock->statsindex[STATID_ACTIVE]);
Proper accounting of active TCP connections
2020-02-28 11:57:51 +01:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
sock->statichandle = NULL;
modify reference counting within netmgr - isc__nmhandle_get() now attaches to the sock in the nmhandle object. the caller is responsible for dereferencing the original socket pointer when necessary. - tcpdns listener sockets attach sock->outer to the outer tcp listener socket. tcpdns connected sockets attach sock->outerhandle to the handle for the tcp connected socket. - only listener sockets need to be attached/detached directly. connected sockets should only be accessed and reference-counted via their associated handles.
2020-06-04 23:13:54 -07:00
if (sock->outerhandle != NULL) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nmhandle_detach(&sock->outerhandle FLARG_PASS);
modify reference counting within netmgr - isc__nmhandle_get() now attaches to the sock in the nmhandle object. the caller is responsible for dereferencing the original socket pointer when necessary. - tcpdns listener sockets attach sock->outer to the outer tcp listener socket. tcpdns connected sockets attach sock->outerhandle to the handle for the tcp connected socket. - only listener sockets need to be attached/detached directly. connected sockets should only be accessed and reference-counted via their associated handles.
2020-06-04 23:13:54 -07:00
}
if (sock->outer != NULL) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_detach(&sock->outer FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
while ((handle = isc_astack_pop(sock->inactivehandles)) != NULL) {
nmhandle_free(sock, handle);
}
if (sock->buf != NULL) {
place a limit on pipelined queries that can be processed simultaneously when the TCPDNS_CLIENTS_PER_CONN limit has been exceeded for a TCP DNS connection, switch to sequential mode to ensure that memory cannot be exhausted by too many simultaneous queries.
2019-11-08 10:52:49 -08:00
isc_mem_free(sock->mgr->mctx, sock->buf);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
if (sock->quota != NULL) {
isc_quota_detach(&sock->quota);
}
netmgr: fix TCP backlog and client quota count - add support for TCP backlog, using the value provided by config. - don't attach to TCP client quota for listening sockets, only connected sockets.
2019-11-22 13:19:45 +01:00
sock->pquota = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_astack_destroy(sock->inactivehandles);
while ((uvreq = isc_astack_pop(sock->inactivereqs)) != NULL) {
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
isc_mempool_put(sock->mgr->reqpool, uvreq);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
isc_astack_destroy(sock->inactivereqs);
Clear sock->magic to 0 when destroying a netmgr socket
2020-05-28 14:47:04 +02:00
sock->magic = 0;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_mem_free(sock->mgr->mctx, sock->ah_frees);
isc_mem_free(sock->mgr->mctx, sock->ah_handles);
Add missing isc_mutex_destroy and isc_conditional_destroy calls. While harmless on Linux, missing isc_{mutex,conditional}_destroy causes a memory leak on *BSD. Missing calls were added.
2020-05-28 12:34:37 +02:00
isc_mutex_destroy(&sock->lock);
isc_condition_destroy(&sock->cond);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_condition_destroy(&sock->scond);
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
isc__nm_tls_cleanup_data(sock);
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
isc__nm_http_cleanup_data(sock);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
LOCK(&sock->mgr->lock);
ISC_LIST_UNLINK(sock->mgr->active_sockets, sock, active_link);
UNLOCK(&sock->mgr->lock);
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (dofree) {
isc_nm_t *mgr = sock->mgr;
isc_mem_put(mgr->mctx, sock, sizeof(*sock));
isc_nm_detach(&mgr);
} else {
isc_nm_detach(&sock->mgr);
}
}
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_maybe_destroy(isc_nmsocket_t *sock FLARG) {
apply the modified style
2020-02-13 14:44:37 -08:00
int active_handles;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
bool destroy = false;
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
NETMGR_TRACE_LOG("%s():%p->references = %" PRIuFAST32 "\n", __func__,
sock, isc_refcount_current(&sock->references));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (sock->parent != NULL) {
/*
* This is a child socket and cannot be destroyed except
* as a side effect of destroying the parent, so let's go
* see if the parent is ready to be destroyed.
*/
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_maybe_destroy(sock->parent FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
return;
}
/*
* This is a parent socket (or a standalone). See whether the
* children have active handles before deciding whether to
* accept destruction.
*/
LOCK(&sock->lock);
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
if (atomic_load(&sock->active) || atomic_load(&sock->destroying) ||
apply the modified style
2020-02-13 14:44:37 -08:00
!atomic_load(&sock->closed) || atomic_load(&sock->references) != 0)
{
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
UNLOCK(&sock->lock);
return;
}
active_handles = atomic_load(&sock->ah);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (sock->children != NULL) {
Use sock->nchildren instead of mgr->nworkers when initializing NM On Windows, we were limiting the number of listening children to just 1, but we were then iterating on mgr->nworkers. That lead to scheduling more async_*listen() than actually allocated and out-of-bound read-write operation on the heap.
2020-12-03 17:58:10 +01:00
for (size_t i = 0; i < sock->nchildren; i++) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
LOCK(&sock->children[i].lock);
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
active_handles += atomic_load(&sock->children[i].ah);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
UNLOCK(&sock->children[i].lock);
}
}
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
if (active_handles == 0 || sock->statichandle != NULL) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
destroy = true;
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
NETMGR_TRACE_LOG("%s:%p->active_handles = %d, .statichandle = %p\n",
__func__, sock, active_handles, sock->statichandle);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (destroy) {
Fix a shutdown race in netmgr udp We need to mark the socket as inactive early (and synchronously) in the stoplistening process; otherwise we might destroy the callback argument before we actually stop listening, and call the callback on bad memory.
2020-06-22 15:46:11 -07:00
atomic_store(&sock->destroying, true);
UNLOCK(&sock->lock);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_cleanup(sock, true FLARG_PASS);
Fix a shutdown race in netmgr udp We need to mark the socket as inactive early (and synchronously) in the stoplistening process; otherwise we might destroy the callback argument before we actually stop listening, and call the callback on bad memory.
2020-06-22 15:46:11 -07:00
} else {
UNLOCK(&sock->lock);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
}
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_prep_destroy(isc_nmsocket_t *sock FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(sock->parent == NULL);
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc___nmsocket_prep_destroy():%p->references = "
"%" PRIuFAST32 "\n",
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
sock, isc_refcount_current(&sock->references));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/*
* The final external reference to the socket is gone. We can try
* destroying the socket, but we have to wait for all the inflight
* handles to finish first.
*/
atomic_store(&sock->active, false);
/*
* If the socket has children, they'll need to be marked inactive
* so they can be cleaned up too.
*/
if (sock->children != NULL) {
Use sock->nchildren instead of mgr->nworkers when initializing NM On Windows, we were limiting the number of listening children to just 1, but we were then iterating on mgr->nworkers. That lead to scheduling more async_*listen() than actually allocated and out-of-bound read-write operation on the heap.
2020-12-03 17:58:10 +01:00
for (size_t i = 0; i < sock->nchildren; i++) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
atomic_store(&sock->children[i].active, false);
}
}
/*
* If we're here then we already stopped listening; otherwise
* we'd have a hanging reference from the listening process.
*
* If it's a regular socket we may need to close it.
*/
if (!atomic_load(&sock->closed)) {
switch (sock->type) {
add netmgr functions to support outgoing DNS queries - isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection. - isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a timeout argument to ensure connections time out and are correctly cleaned up on failure. - isc_nm_read() now supports UDP; it reads a single datagram and then stops until the next time it's called. - isc_nm_cancelread() now runs asynchronously to prevent assertion failure if reading is interrupted by a non-network thread (e.g. a timeout). - isc_nm_cancelread() can now apply to UDP sockets. - added shim code to support UDP connection in versions of libuv prior to 1.27, when uv_udp_connect() was added all these functions will be used to support outgoing queries in dig, xfrin, dispatch, etc.
2020-09-05 11:07:40 -07:00
case isc_nm_udpsocket:
isc__nm_udp_close(sock);
return;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
case isc_nm_tcpsocket:
isc__nm_tcp_close(sock);
Don't destroy a non-closed socket, wait for all the callbacks. We erroneously tried to destroy a socket after issuing isc__nm_tcp{,dns}_close. Under some (race) circumstances we could get nm_socket_cleanup to be called twice for the same socket, causing an access to a dead memory.
2020-07-01 00:49:12 -07:00
return;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
case isc_nm_tcpdnssocket:
isc__nm_tcpdns_close(sock);
Don't destroy a non-closed socket, wait for all the callbacks. We erroneously tried to destroy a socket after issuing isc__nm_tcp{,dns}_close. Under some (race) circumstances we could get nm_socket_cleanup to be called twice for the same socket, causing an access to a dead memory.
2020-07-01 00:49:12 -07:00
return;
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
isc__nm_tls_close(sock);
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnssocket:
isc__nm_tlsdns_close(sock);
return;
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
case isc_nm_httpsocket:
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
isc__nm_http_close(sock);
return;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
default:
break;
}
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmsocket_maybe_destroy(sock FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_detach(isc_nmsocket_t **sockp FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(sockp != NULL && *sockp != NULL);
REQUIRE(VALID_NMSOCK(*sockp));
isc_nmsocket_t *sock = *sockp, *rsock = NULL;
*sockp = NULL;
/*
* If the socket is a part of a set (a child socket) we are
* counting references for the whole set at the parent.
*/
if (sock->parent != NULL) {
rsock = sock->parent;
INSIST(rsock->parent == NULL); /* Sanity check */
} else {
rsock = sock;
}
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc__nmsocket_detach():%p->references = %" PRIuFAST32
"\n",
rsock, isc_refcount_current(&rsock->references) - 1);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
Use isc_refcount_increment0() when reusing handle or socket; remove extra DbC checks
2020-01-14 09:43:37 +01:00
if (isc_refcount_decrement(&rsock->references) == 1) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_prep_destroy(rsock FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
}
make isc_nmsocket_{attach,detach}{} functions private there is no need for a caller to reference-count socket objects. they need tto be able tto close listener sockets (i.e., those returned by isc_nm_listen{udp,tcp,tcpdns}), and an isc_nmsocket_close() function has been added for that. other sockets are only accessed via handles.
2020-06-04 14:54:36 -07:00
void
isc_nmsocket_close(isc_nmsocket_t **sockp) {
REQUIRE(sockp != NULL);
REQUIRE(VALID_NMSOCK(*sockp));
REQUIRE((*sockp)->type == isc_nm_udplistener ||
(*sockp)->type == isc_nm_tcplistener ||
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
(*sockp)->type == isc_nm_tcpdnslistener ||
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
(*sockp)->type == isc_nm_tlsdnslistener ||
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
(*sockp)->type == isc_nm_tlslistener ||
(*sockp)->type == isc_nm_httplistener);
make isc_nmsocket_{attach,detach}{} functions private there is no need for a caller to reference-count socket objects. they need tto be able tto close listener sockets (i.e., those returned by isc_nm_listen{udp,tcp,tcpdns}), and an isc_nmsocket_close() function has been added for that. other sockets are only accessed via handles.
2020-06-04 14:54:36 -07:00
isc__nmsocket_detach(sockp);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_init(isc_nmsocket_t *sock, isc_nm_t *mgr, isc_nmsocket_type type,
isc_nmiface_t *iface FLARG) {
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
uint16_t family;
REQUIRE(sock != NULL);
REQUIRE(mgr != NULL);
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
REQUIRE(iface != NULL);
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
family = iface->addr.type.sa.sa_family;
Increase inactivehandles and inactivereqs size for better reuse.
2020-01-29 15:16:02 +01:00
*sock = (isc_nmsocket_t){ .type = type,
.iface = iface,
.fd = -1,
.ah_size = 32,
.inactivehandles = isc_astack_new(
mgr->mctx, ISC_NM_HANDLES_STACK_SIZE),
.inactivereqs = isc_astack_new(
mgr->mctx, ISC_NM_REQS_STACK_SIZE) };
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
#if NETMGR_TRACE
Add support for generating backtraces on Windows This commit adds support for generating backtraces on Windows and refactors the isc_backtrace API to match the Linux/BSD API (without the isc_ prefix) * isc_backtrace_gettrace() was renamed to isc_backtrace(), the third argument was removed and the return type was changed to int * isc_backtrace_symbols() was added * isc_backtrace_symbols_fd() was added and used as appropriate
2021-04-27 16:20:03 +02:00
sock->backtrace_size = isc_backtrace(sock->backtrace, TRACE_SIZE);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
ISC_LINK_INIT(sock, active_link);
ISC_LIST_INIT(sock->active_handles);
LOCK(&mgr->lock);
ISC_LIST_APPEND(mgr->active_sockets, sock, active_link);
UNLOCK(&mgr->lock);
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nm_attach(mgr, &sock->mgr);
sock->uv_handle.handle.data = sock;
apply the modified style
2020-02-13 14:44:37 -08:00
sock->ah_frees = isc_mem_allocate(mgr->mctx,
sock->ah_size * sizeof(size_t));
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
sock->ah_handles = isc_mem_allocate(
mgr->mctx, sock->ah_size * sizeof(isc_nmhandle_t *));
Redesigned TCP accepting: one listen/accept loop, passing the connected socket. Instead of using bind() and passing the listening socket to the children threads using uv_export/uv_import use one thread that does the accepting, and then passes the connected socket using uv_export/uv_import to a random worker. The previous solution had thundering herd problems (all workers waking up on one connection and trying to accept()), this one avoids this and is simpler. The tcp clients quota is simplified with isc_quota_attach_cb - a callback is issued when the quota is available.
2020-03-24 13:38:51 +01:00
ISC_LINK_INIT(&sock->quotacb, link);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
for (size_t i = 0; i < 32; i++) {
sock->ah_frees[i] = i;
sock->ah_handles[i] = NULL;
}
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
switch (type) {
case isc_nm_udpsocket:
case isc_nm_udplistener:
if (family == AF_INET) {
sock->statsindex = udp4statsindex;
} else {
sock->statsindex = udp6statsindex;
}
count statistics in netmgr UDP code - also restored a test in the statistics test which was changed when the netmgr was introduced because active sockets were not being counted.
2020-01-06 20:26:47 -08:00
isc__nm_incstats(sock->mgr, sock->statsindex[STATID_ACTIVE]);
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
break;
case isc_nm_tcpsocket:
case isc_nm_tcplistener:
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tcpdnssocket:
case isc_nm_tcpdnslistener:
case isc_nm_tlsdnssocket:
case isc_nm_tlsdnslistener:
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
case isc_nm_httpsocket:
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
case isc_nm_httplistener:
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
if (family == AF_INET) {
sock->statsindex = tcp4statsindex;
} else {
sock->statsindex = tcp6statsindex;
}
count statistics in netmgr UDP code - also restored a test in the statistics test which was changed when the netmgr was introduced because active sockets were not being counted.
2020-01-06 20:26:47 -08:00
isc__nm_incstats(sock->mgr, sock->statsindex[STATID_ACTIVE]);
break was on wrong line. 959 break; CID 1457872 (#1 of 1): Structurally dead code (UNREACHABLE) unreachable: This code cannot be reached: isc__nm_incstats(sock->mgr,.... 960 isc__nm_incstats(sock->mgr, sock->statsindex[STATID_ACTIVE]); 961 default:
2020-02-05 15:50:29 +11:00
break;
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
default:
break;
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_mutex_init(&sock->lock);
isc_condition_init(&sock->cond);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc_condition_init(&sock->scond);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_refcount_init(&sock->references, 1);
place a limit on pipelined queries that can be processed simultaneously when the TCPDNS_CLIENTS_PER_CONN limit has been exceeded for a TCP DNS connection, switch to sequential mode to ensure that memory cannot be exhausted by too many simultaneous queries.
2019-11-08 10:52:49 -08:00
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
memset(&sock->tlsstream, 0, sizeof(sock->tlsstream));
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc__nmsocket_init():%p->references = %" PRIuFAST32
"\n",
sock, isc_refcount_current(&sock->references));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
atomic_init(&sock->active, true);
place a limit on pipelined queries that can be processed simultaneously when the TCPDNS_CLIENTS_PER_CONN limit has been exceeded for a TCP DNS connection, switch to sequential mode to ensure that memory cannot be exhausted by too many simultaneous queries.
2019-11-08 10:52:49 -08:00
atomic_init(&sock->sequential, false);
atomic_init(&sock->readpaused, false);
Fix more races between connect and shutdown There were more races that could happen while connecting to a socket while closing or shutting down the same socket. This commit introduces a .closing flag to guard the socket from being closed twice.
2020-10-29 12:04:00 +01:00
atomic_init(&sock->closing, false);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
atomic_store(&sock->active_child_connections, 0);
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
isc__nm_http_initsocket(sock);
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
sock->magic = NMSOCK_MAGIC;
}
clean up outerhandle when a tcpdns socket is disconnected this prevents a crash when some non-netmgr thread, such as a recursive lookup, times out after the TCP socket is already disconnected.
2020-06-20 15:03:05 -07:00
void
isc__nmsocket_clearcb(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
Clone the csock in accept_connection(), not in callback If we clone the csock (children socket) in TCP accept_connection() instead of passing the ssock (server socket) to the call back and cloning it there we unbreak the assumption that every socket is handled inside it's own worker thread and therefore we can get rid of (at least) callback locking.
2020-09-18 12:27:40 +02:00
REQUIRE(!isc__nm_in_netthread() || sock->tid == isc_nm_tid());
clean up outerhandle when a tcpdns socket is disconnected this prevents a crash when some non-netmgr thread, such as a recursive lookup, times out after the TCP socket is already disconnected.
2020-06-20 15:03:05 -07:00
properly lock the setting/unsetting of callbacks in isc_nmsocket_t changes to socket callback functions were not thread safe.
2020-09-11 10:53:31 +02:00
sock->recv_cb = NULL;
sock->recv_cbarg = NULL;
sock->accept_cb = NULL;
clean up outerhandle when a tcpdns socket is disconnected this prevents a crash when some non-netmgr thread, such as a recursive lookup, times out after the TCP socket is already disconnected.
2020-06-20 15:03:05 -07:00
sock->accept_cbarg = NULL;
Fix more races between connect and shutdown There were more races that could happen while connecting to a socket while closing or shutting down the same socket. This commit introduces a .closing flag to guard the socket from being closed twice.
2020-10-29 12:04:00 +01:00
sock->connect_cb = NULL;
sock->connect_cbarg = NULL;
clean up outerhandle when a tcpdns socket is disconnected this prevents a crash when some non-netmgr thread, such as a recursive lookup, times out after the TCP socket is already disconnected.
2020-06-20 15:03:05 -07:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_free_uvbuf(isc_nmsocket_t *sock, const uv_buf_t *buf) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__networker_t *worker = NULL;
REQUIRE(VALID_NMSOCK(sock));
place a limit on pipelined queries that can be processed simultaneously when the TCPDNS_CLIENTS_PER_CONN limit has been exceeded for a TCP DNS connection, switch to sequential mode to ensure that memory cannot be exhausted by too many simultaneous queries.
2019-11-08 10:52:49 -08:00
if (buf->base == NULL) {
/* Empty buffer: might happen in case of error. */
return;
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
worker = &sock->mgr->workers[sock->tid];
clean up comments
2019-11-15 13:22:13 -08:00
REQUIRE(worker->recvbuf_inuse);
Use different allocators for UDP and TCP Each worker has a receive buffer with space for 20 DNS messages of up to 2^16 bytes each, and the allocator function passed to uv_read_start() or uv_udp_recv_start() will reserve a portion of it for use by sockets. UDP can use recvmmsg() and so it needs that entire space, but TCP reads one message at a time. This commit introduces separate allocator functions for TCP and UDP setting different buffer size limits, so that libuv will provide the correct buffer sizes to each of them.
2020-07-02 16:27:38 +02:00
if (sock->type == isc_nm_udpsocket && buf->base > worker->recvbuf &&
Make nm->recvbuf larger and heap allocated, to allow uv_recvmmsg usage. Upcoming version of libuv will suport uv_recvmmsg and uv_sendmmsg. To use uv_recvmmsg we need to provide a larger buffer and be able to properly free it.
2020-01-29 13:16:04 +01:00
buf->base <= worker->recvbuf + ISC_NETMGR_RECVBUF_SIZE)
{
Don't free udp recv buffer if UV_UDP_MMSG_CHUNK is set
2020-04-29 15:19:32 +02:00
/* Can happen in case of out-of-order recvmmsg in libuv1.36 */
Make nm->recvbuf larger and heap allocated, to allow uv_recvmmsg usage. Upcoming version of libuv will suport uv_recvmmsg and uv_sendmmsg. To use uv_recvmmsg we need to provide a larger buffer and be able to properly free it.
2020-01-29 13:16:04 +01:00
return;
}
clean up comments
2019-11-15 13:22:13 -08:00
REQUIRE(buf->base == worker->recvbuf);
worker->recvbuf_inuse = false;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
static isc_nmhandle_t *
apply the modified style
2020-02-13 14:44:37 -08:00
alloc_handle(isc_nmsocket_t *sock) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nmhandle_t *handle =
isc_mem_get(sock->mgr->mctx,
sizeof(isc_nmhandle_t) + sock->extrahandlesize);
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*handle = (isc_nmhandle_t){ .magic = NMHANDLE_MAGIC };
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
ISC_LINK_INIT(handle, active_link);
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_refcount_init(&handle->references, 1);
return (handle);
}
isc_nmhandle_t *
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmhandle_get(isc_nmsocket_t *sock, isc_sockaddr_t *peer,
isc_sockaddr_t *local FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_nmhandle_t *handle = NULL;
apply the modified style
2020-02-13 14:44:37 -08:00
size_t handlenum;
int pos;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMSOCK(sock));
handle = isc_astack_pop(sock->inactivehandles);
if (handle == NULL) {
handle = alloc_handle(sock);
} else {
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
isc_refcount_init(&handle->references, 1);
Don't destroy a non-closed socket, wait for all the callbacks. We erroneously tried to destroy a socket after issuing isc__nm_tcp{,dns}_close. Under some (race) circumstances we could get nm_socket_cleanup to be called twice for the same socket, causing an access to a dead memory.
2020-07-01 00:49:12 -07:00
INSIST(VALID_NMHANDLE(handle));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG(
"isc__nmhandle_get():handle %p->references = %" PRIuFAST32 "\n",
handle, isc_refcount_current(&handle->references));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_attach(sock, &handle->sock FLARG_PASS);
modify reference counting within netmgr - isc__nmhandle_get() now attaches to the sock in the nmhandle object. the caller is responsible for dereferencing the original socket pointer when necessary. - tcpdns listener sockets attach sock->outer to the outer tcp listener socket. tcpdns connected sockets attach sock->outerhandle to the handle for the tcp connected socket. - only listener sockets need to be attached/detached directly. connected sockets should only be accessed and reference-counted via their associated handles.
2020-06-04 23:13:54 -07:00
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
#if NETMGR_TRACE
Add support for generating backtraces on Windows This commit adds support for generating backtraces on Windows and refactors the isc_backtrace API to match the Linux/BSD API (without the isc_ prefix) * isc_backtrace_gettrace() was renamed to isc_backtrace(), the third argument was removed and the return type was changed to int * isc_backtrace_symbols() was added * isc_backtrace_symbols_fd() was added and used as appropriate
2021-04-27 16:20:03 +02:00
handle->backtrace_size = isc_backtrace(handle->backtrace, TRACE_SIZE);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
if (peer != NULL) {
netmgr: server-side TLS support Add server-side TLS support to netmgr - that includes moving some of the isc_nm_ functions from tcp.c to a wrapper in netmgr.c calling a proper tcp or tls function, and a new isc_nm_listentls() function. Add DoT support to tcpdns - isc_nm_listentlsdns().
2020-05-13 17:37:51 +02:00
memmove(&handle->peer, peer, sizeof(isc_sockaddr_t));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
} else {
netmgr: server-side TLS support Add server-side TLS support to netmgr - that includes moving some of the isc_nm_ functions from tcp.c to a wrapper in netmgr.c calling a proper tcp or tls function, and a new isc_nm_listentls() function. Add DoT support to tcpdns - isc_nm_listentlsdns().
2020-05-13 17:37:51 +02:00
memmove(&handle->peer, &sock->peer, sizeof(isc_sockaddr_t));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
if (local != NULL) {
netmgr: server-side TLS support Add server-side TLS support to netmgr - that includes moving some of the isc_nm_ functions from tcp.c to a wrapper in netmgr.c calling a proper tcp or tls function, and a new isc_nm_listentls() function. Add DoT support to tcpdns - isc_nm_listentlsdns().
2020-05-13 17:37:51 +02:00
memmove(&handle->local, local, sizeof(isc_sockaddr_t));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
} else if (sock->iface != NULL) {
netmgr: server-side TLS support Add server-side TLS support to netmgr - that includes moving some of the isc_nm_ functions from tcp.c to a wrapper in netmgr.c calling a proper tcp or tls function, and a new isc_nm_listentls() function. Add DoT support to tcpdns - isc_nm_listentlsdns().
2020-05-13 17:37:51 +02:00
memmove(&handle->local, &sock->iface->addr,
sizeof(isc_sockaddr_t));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
} else {
INSIST(0);
ISC_UNREACHABLE();
}
LOCK(&sock->lock);
/* We need to add this handle to the list of active handles */
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
if ((size_t)atomic_load(&sock->ah) == sock->ah_size) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
sock->ah_frees =
isc_mem_reallocate(sock->mgr->mctx, sock->ah_frees,
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
sock->ah_size * 2 * sizeof(size_t));
sock->ah_handles = isc_mem_reallocate(
sock->mgr->mctx, sock->ah_handles,
sock->ah_size * 2 * sizeof(isc_nmhandle_t *));
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
for (size_t i = sock->ah_size; i < sock->ah_size * 2; i++) {
sock->ah_frees[i] = i;
sock->ah_handles[i] = NULL;
}
sock->ah_size *= 2;
}
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
handlenum = atomic_fetch_add(&sock->ah, 1);
pos = sock->ah_frees[handlenum];
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
INSIST(sock->ah_handles[pos] == NULL);
sock->ah_handles[pos] = handle;
handle->ah_pos = pos;
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
ISC_LIST_APPEND(sock->active_handles, handle, active_link);
#endif
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
UNLOCK(&sock->lock);
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
switch (sock->type) {
case isc_nm_udpsocket:
case isc_nm_tcpdnssocket:
case isc_nm_tlsdnssocket:
if (!atomic_load(&sock->client)) {
break;
}
/* fallthrough */
case isc_nm_tcpsocket:
case isc_nm_tlssocket:
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
INSIST(sock->statichandle == NULL);
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
/*
* statichandle must be assigned, not attached;
* otherwise, if a handle was detached elsewhere
* it could never reach 0 references, and the
* handle and socket would never be freed.
*/
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
sock->statichandle = handle;
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
break;
default:
break;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
if (sock->type == isc_nm_httpsocket && sock->h2.session) {
isc__nm_httpsession_attach(sock->h2.session,
&handle->httpsession);
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
return (handle);
}
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nmhandle_attach(isc_nmhandle_t *handle, isc_nmhandle_t **handlep FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
REQUIRE(handlep != NULL && *handlep == NULL);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc__nmhandle_attach():handle %p->references = "
"%" PRIuFAST32 "\n",
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
handle, isc_refcount_current(&handle->references) + 1);
Use isc_refcount_increment0() when reusing handle or socket; remove extra DbC checks
2020-01-14 09:43:37 +01:00
isc_refcount_increment(&handle->references);
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
*handlep = handle;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
bool
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_is_stream(isc_nmhandle_t *handle) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
return (handle->sock->type == isc_nm_tcpsocket ||
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
handle->sock->type == isc_nm_tcpdnssocket ||
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
handle->sock->type == isc_nm_tlssocket ||
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
handle->sock->type == isc_nm_tlsdnssocket);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
static void
apply the modified style
2020-02-13 14:44:37 -08:00
nmhandle_free(isc_nmsocket_t *sock, isc_nmhandle_t *handle) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
size_t extra = sock->extrahandlesize;
Add isc_refcount_destroy() call to nm_handle_free()
2019-12-10 11:09:56 +01:00
isc_refcount_destroy(&handle->references);
place a limit on pipelined queries that can be processed simultaneously when the TCPDNS_CLIENTS_PER_CONN limit has been exceeded for a TCP DNS connection, switch to sequential mode to ensure that memory cannot be exhausted by too many simultaneous queries.
2019-11-08 10:52:49 -08:00
if (handle->dofree != NULL) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
handle->dofree(handle->opaque);
}
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*handle = (isc_nmhandle_t){ .magic = 0 };
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc_mem_put(sock->mgr->mctx, handle, sizeof(isc_nmhandle_t) + extra);
}
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
static void
apply the modified style
2020-02-13 14:44:37 -08:00
nmhandle_deactivate(isc_nmsocket_t *sock, isc_nmhandle_t *handle) {
implement isc_nm_tcpconnect() the isc_nm_tcpconnect() function establishes a client connection via TCP. once the connection is esablished, a callback function will be called with a newly created network manager handle.
2019-12-16 18:24:55 -08:00
size_t handlenum;
bool reuse = false;
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
/*
* We do all of this under lock to avoid races with socket
* destruction. We have to do this now, because at this point the
* socket is either unused or still attached to event->sock.
*/
LOCK(&sock->lock);
INSIST(sock->ah_handles[handle->ah_pos] == handle);
INSIST(sock->ah_size > handle->ah_pos);
INSIST(atomic_load(&sock->ah) > 0);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
ISC_LIST_UNLINK(sock->active_handles, handle, active_link);
#endif
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
sock->ah_handles[handle->ah_pos] = NULL;
implement isc_nm_tcpconnect() the isc_nm_tcpconnect() function establishes a client connection via TCP. once the connection is esablished, a callback function will be called with a newly created network manager handle.
2019-12-16 18:24:55 -08:00
handlenum = atomic_fetch_sub(&sock->ah, 1) - 1;
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
sock->ah_frees[handlenum] = handle->ah_pos;
handle->ah_pos = 0;
if (atomic_load(&sock->active)) {
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
reuse = isc_astack_trypush(sock->inactivehandles, handle);
Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref
2019-12-08 22:44:08 +01:00
}
if (!reuse) {
nmhandle_free(sock, handle);
}
UNLOCK(&sock->lock);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nmhandle_detach(isc_nmhandle_t **handlep FLARG) {
modify reference counting within netmgr - isc__nmhandle_get() now attaches to the sock in the nmhandle object. the caller is responsible for dereferencing the original socket pointer when necessary. - tcpdns listener sockets attach sock->outer to the outer tcp listener socket. tcpdns connected sockets attach sock->outerhandle to the handle for the tcp connected socket. - only listener sockets need to be attached/detached directly. connected sockets should only be accessed and reference-counted via their associated handles.
2020-06-04 23:13:54 -07:00
isc_nmsocket_t *sock = NULL;
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
isc_nmhandle_t *handle = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
REQUIRE(handlep != NULL);
REQUIRE(VALID_NMHANDLE(*handlep));
handle = *handlep;
*handlep = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
sock = handle->sock;
if (sock->tid == isc_nm_tid()) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmhandle_detach_cb(&handle FLARG_PASS);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
} else {
isc__netievent_detach_t *event =
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_get_netievent_detach(sock->mgr, sock);
/*
* we are using implicit "attach" as the last reference
* need to be destroyed explicitly in the async callback
*/
event->handle = handle;
FLARG_IEVENT_PASS(event);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
isc__nm_enqueue_ievent(&sock->mgr->workers[sock->tid],
(isc__netievent_t *)event);
}
}
Fix and clean up handling of connect callbacks Serveral problems were discovered and fixed after the change in the connection timeout in the previous commits: * In TLSDNS, the connection callback was not called at all under some circumstances when the TCP connection had been established, but the TLS handshake hadn't been completed yet. Additional checks have been put in place so that tls_cycle() will end early when the nmsocket is invalidated by the isc__nm_tlsdns_shutdown() call. * In TCP, TCPDNS and TLSDNS, new connections would be established even when the network manager was shutting down. The new call isc__nm_closing() has been added and is used to bail out early even before uv_tcp_connect() is attempted.
2021-03-31 11:48:41 +02:00
void
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
isc__nmsocket_shutdown(isc_nmsocket_t *sock);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
static void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmhandle_detach_cb(isc_nmhandle_t **handlep FLARG) {
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
isc_nmsocket_t *sock = NULL;
isc_nmhandle_t *handle = NULL;
REQUIRE(handlep != NULL);
REQUIRE(VALID_NMHANDLE(*handlep));
handle = *handlep;
*handlep = NULL;
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
NETMGR_TRACE_LOG("isc__nmhandle_detach():%p->references = %" PRIuFAST32
"\n",
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
handle, isc_refcount_current(&handle->references) - 1);
Use isc_refcount_increment0() when reusing handle or socket; remove extra DbC checks
2020-01-14 09:43:37 +01:00
if (isc_refcount_decrement(&handle->references) > 1) {
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
return;
}
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
Add missing acquire memory barrier in isc_nmhandle_unref The ThreadSanitizer uses system synchronization primitives to check for data race. The netmgr handle->references was missing acquire memory barrier before resetting and reusing the memory occupied by isc_nmhandle_t.
2020-06-11 13:01:26 +02:00
/* We need an acquire memory barrier here */
(void)isc_refcount_current(&handle->references);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
sock = handle->sock;
handle->sock = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
if (handle->doreset != NULL) {
handle->doreset(handle->opaque);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
if (sock->type == isc_nm_httpsocket && handle->httpsession != NULL) {
isc__nm_httpsession_detach(&handle->httpsession);
}
Deactivate the handle before sending the async close callback. We could have a race between handle closing and processing async callback. Deactivate the handle before issuing the callback - we have the socket referenced anyway so it's not a problem.
2020-03-26 14:25:06 +01:00
nmhandle_deactivate(sock, handle);
/*
* The handle is gone now. If the socket has a callback configured
netmgr: fix a race in socket destruction, happening if we close the socket externally and, at the same time, a timeout timer callback was called.
2019-12-06 20:45:00 +01:00
* for that (e.g., to perform cleanup after request processing),
Fix a race in tcpdns close with uv_close on timer stop timers before closing netmgr: tcpdns_close needs to be asynchronous, it manipulates sock->timer
2019-12-06 22:25:52 +01:00
* call it now, or schedule it to run asynchronously.
netmgr: fix a race in socket destruction, happening if we close the socket externally and, at the same time, a timeout timer callback was called.
2019-12-06 20:45:00 +01:00
*/
if (sock->closehandle_cb != NULL) {
if (sock->tid == isc_nm_tid()) {
sock->closehandle_cb(sock);
} else {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__netievent_close_t *event =
isc__nm_get_netievent_close(sock->mgr, sock);
netmgr: fix a race in socket destruction, happening if we close the socket externally and, at the same time, a timeout timer callback was called.
2019-12-06 20:45:00 +01:00
isc__nm_enqueue_ievent(&sock->mgr->workers[sock->tid],
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
(isc__netievent_t *)event);
netmgr: fix a race in socket destruction, happening if we close the socket externally and, at the same time, a timeout timer callback was called.
2019-12-06 20:45:00 +01:00
}
}
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
if (handle == sock->statichandle) {
change from isc_nmhandle_ref/unref to isc_nmhandle attach/detach Attaching and detaching handle pointers will make it easier to determine where and why reference counting errors have occurred. A handle needs to be referenced more than once when multiple asynchronous operations are in flight, so callers must now maintain multiple handle pointers for each pending operation. For example, ns_client objects now contain: - reqhandle: held while waiting for a request callback (query, notify, update) - sendhandle: held while waiting for a send callback - fetchhandle: held while waiting for a recursive fetch to complete - updatehandle: held while waiting for an update-forwarding task to complete control channel connection objects now contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback - cmdhandle: held while an rndc command is running httpd connections contain: - readhandle: held while waiting for a read callback - sendhandle: held while waiting for a send callback
2020-09-03 13:31:27 -07:00
/* statichandle is assigned, not attached. */
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
sock->statichandle = NULL;
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_detach(&sock FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
void *
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_getdata(isc_nmhandle_t *handle) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
return (handle->opaque);
}
void
isc_nmhandle_setdata(isc_nmhandle_t *handle, void *arg,
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_opaquecb_t doreset, isc_nm_opaquecb_t dofree) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
handle->opaque = arg;
handle->doreset = doreset;
handle->dofree = dofree;
}
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
void
isc__nm_alloc_dnsbuf(isc_nmsocket_t *sock, size_t len) {
REQUIRE(len <= NM_BIG_BUF);
if (sock->buf == NULL) {
/* We don't have the buffer at all */
size_t alloc_len = len < NM_REG_BUF ? NM_REG_BUF : NM_BIG_BUF;
sock->buf = isc_mem_allocate(sock->mgr->mctx, alloc_len);
sock->buf_size = alloc_len;
} else {
/* We have the buffer but it's too small */
sock->buf = isc_mem_reallocate(sock->mgr->mctx, sock->buf,
NM_BIG_BUF);
sock->buf_size = NM_BIG_BUF;
}
}
void
isc__nm_failed_send_cb(isc_nmsocket_t *sock, isc__nm_uvreq_t *req,
isc_result_t eresult) {
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(req));
if (req->cb.send != NULL) {
isc__nm_sendcb(sock, req, eresult, true);
} else {
isc__nm_uvreq_put(&req, sock);
}
}
void
isc__nm_failed_accept_cb(isc_nmsocket_t *sock, isc_result_t eresult) {
REQUIRE(sock->accepting);
REQUIRE(sock->server);
/*
* Detach the quota early to make room for other connections;
* otherwise it'd be detached later asynchronously, and clog
* the quota unnecessarily.
*/
if (sock->quota != NULL) {
isc_quota_detach(&sock->quota);
}
isc__nmsocket_detach(&sock->server);
sock->accepting = false;
switch (eresult) {
case ISC_R_NOTCONNECTED:
/* IGNORE: The client disconnected before we could accept */
break;
default:
isc_log_write(isc_lctx, ISC_LOGCATEGORY_GENERAL,
ISC_LOGMODULE_NETMGR, ISC_LOG_ERROR,
"Accepting TCP connection failed: %s",
isc_result_totext(eresult));
}
}
void
isc__nm_failed_connect_cb(isc_nmsocket_t *sock, isc__nm_uvreq_t *req,
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc_result_t eresult, bool async) {
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(req));
REQUIRE(sock->tid == isc_nm_tid());
REQUIRE(req->cb.connect != NULL);
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
isc__nmsocket_timer_stop(sock);
Fix and clean up handling of connect callbacks Serveral problems were discovered and fixed after the change in the connection timeout in the previous commits: * In TLSDNS, the connection callback was not called at all under some circumstances when the TCP connection had been established, but the TLS handshake hadn't been completed yet. Additional checks have been put in place so that tls_cycle() will end early when the nmsocket is invalidated by the isc__nm_tlsdns_shutdown() call. * In TCP, TCPDNS and TLSDNS, new connections would be established even when the network manager was shutting down. The new call isc__nm_closing() has been added and is used to bail out early even before uv_tcp_connect() is attempted.
2021-03-31 11:48:41 +02:00
uv_handle_set_data((uv_handle_t *)&sock->timer, sock);
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
INSIST(atomic_compare_exchange_strong(&sock->connecting,
&(bool){ true }, false));
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
isc__nmsocket_clearcb(sock);
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc__nm_connectcb(sock, req, eresult, async);
Merge the common parts between udp, tcpdns and tlsdns protocol The udp, tcpdns and tlsdns contained lot of cut&paste code or code that was very similar making the stack harder to maintain as any change to one would have to be copied to the the other protocols. In this commit, we merge the common parts into the common functions under isc__nm_<foo> namespace and just keep the little differences based on the socket type.
2021-03-18 09:27:38 +01:00
isc__nmsocket_prep_destroy(sock);
}
void
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc__nm_failed_read_cb(isc_nmsocket_t *sock, isc_result_t result, bool async) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
REQUIRE(VALID_NMSOCK(sock));
switch (sock->type) {
case isc_nm_udpsocket:
isc__nm_udp_failed_read_cb(sock, result);
return;
case isc_nm_tcpsocket:
isc__nm_tcp_failed_read_cb(sock, result);
return;
case isc_nm_tcpdnssocket:
isc__nm_tcpdns_failed_read_cb(sock, result);
return;
case isc_nm_tlsdnssocket:
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc__nm_tlsdns_failed_read_cb(sock, result, async);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return;
default:
INSIST(0);
ISC_UNREACHABLE();
}
}
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
void
isc__nmsocket_connecttimeout_cb(uv_timer_t *timer) {
uv_connect_t *uvreq = uv_handle_get_data((uv_handle_t *)timer);
isc_nmsocket_t *sock = uv_handle_get_data((uv_handle_t *)uvreq->handle);
isc__nm_uvreq_t *req = uv_handle_get_data((uv_handle_t *)uvreq);
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(sock->tid == isc_nm_tid());
REQUIRE(atomic_load(&sock->connecting));
REQUIRE(VALID_UVREQ(req));
REQUIRE(VALID_NMHANDLE(req->handle));
isc__nmsocket_timer_stop(sock);
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
if (sock->tls.pending_req != NULL) {
REQUIRE(req == sock->tls.pending_req);
sock->tls.pending_req = NULL;
}
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
/* Call the connect callback directly */
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
req->cb.connect(req->handle, ISC_R_TIMEDOUT, req->cbarg);
/* Timer is not running, cleanup and shutdown everything */
if (!isc__nmsocket_timer_running(sock)) {
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
INSIST(atomic_compare_exchange_strong(&sock->connecting,
&(bool){ true }, false));
isc__nm_uvreq_put(&req, sock);
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
isc__nmsocket_clearcb(sock);
isc__nmsocket_shutdown(sock);
}
}
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
static void
isc__nmsocket_readtimeout_cb(uv_timer_t *timer) {
isc_nmsocket_t *sock = uv_handle_get_data((uv_handle_t *)timer);
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(sock->tid == isc_nm_tid());
REQUIRE(sock->reading);
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
if (atomic_load(&sock->client)) {
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
uv_timer_stop(timer);
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
if (sock->recv_cb != NULL) {
isc__nm_uvreq_t *req = isc__nm_get_read_req(sock, NULL);
isc__nm_readcb(sock, req, ISC_R_TIMEDOUT);
}
if (!isc__nmsocket_timer_running(sock)) {
isc__nmsocket_clearcb(sock);
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc__nm_failed_read_cb(sock, ISC_R_CANCELED, false);
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
}
} else {
Refactor async callbacks and fix the double tlsdnsconnect callback The isc_nm_tlsdnsconnect() call could end up with two connect callbacks called when the timeout fired and the TCP connection was aborted, but the TLS handshake was not complete yet. isc__nm_connecttimeout_cb() forgot to clean up sock->tls.pending_req when the connect callback was called with ISC_R_TIMEDOUT, leading to a second callback running later. A new argument has been added to the isc__nm_*_failed_connect_cb and isc__nm_*_failed_read_cb functions, to indicate whether the callback needs to run asynchronously or not.
2021-04-06 18:27:38 +02:00
isc__nm_failed_read_cb(sock, ISC_R_TIMEDOUT, false);
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
}
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
void
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc__nmsocket_timer_restart(isc_nmsocket_t *sock) {
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
int r = 0;
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
REQUIRE(VALID_NMSOCK(sock));
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
if (atomic_load(&sock->connecting)) {
if (sock->connect_timeout == 0) {
return;
}
r = uv_timer_start(&sock->timer,
isc__nmsocket_connecttimeout_cb,
sock->connect_timeout + 10, 0);
} else {
if (sock->read_timeout == 0) {
return;
}
r = uv_timer_start(&sock->timer, isc__nmsocket_readtimeout_cb,
sock->read_timeout, 0);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
RUNTIME_CHECK(r == 0);
}
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
bool
isc__nmsocket_timer_running(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
return (uv_is_active((uv_handle_t *)&sock->timer));
}
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
void
isc__nmsocket_timer_start(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
if (isc__nmsocket_timer_running(sock)) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return;
}
isc__nmsocket_timer_restart(sock);
}
void
isc__nmsocket_timer_stop(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
/* uv_timer_stop() is idempotent, no need to check if running */
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
int r = uv_timer_stop(&sock->timer);
RUNTIME_CHECK(r == 0);
}
isc__nm_uvreq_t *
isc__nm_get_read_req(isc_nmsocket_t *sock, isc_sockaddr_t *sockaddr) {
isc__nm_uvreq_t *req = NULL;
req = isc__nm_uvreq_get(sock->mgr, sock);
req->cb.recv = sock->recv_cb;
req->cbarg = sock->recv_cbarg;
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
switch (sock->type) {
case isc_nm_tcpsocket:
case isc_nm_tlssocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc_nmhandle_attach(sock->statichandle, &req->handle);
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
break;
default:
if (atomic_load(&sock->client)) {
isc_nmhandle_attach(sock->statichandle, &req->handle);
} else {
req->handle = isc__nmhandle_get(sock, sockaddr, NULL);
}
break;
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
return (req);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
/*%<
* Allocator for read operations. Limited to size 2^16.
*
* Note this doesn't actually allocate anything, it just assigns the
* worker's receive buffer to a socket, and marks it as "in use".
*/
void
isc__nm_alloc_cb(uv_handle_t *handle, size_t size, uv_buf_t *buf) {
isc_nmsocket_t *sock = uv_handle_get_data(handle);
isc__networker_t *worker = NULL;
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(isc__nm_in_netthread());
switch (sock->type) {
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
case isc_nm_udpsocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
REQUIRE(size <= ISC_NETMGR_RECVBUF_SIZE);
size = ISC_NETMGR_RECVBUF_SIZE;
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
break;
Use generic functions for reading and timers in TCP The TCP module has been updated to use the generic functions from netmgr.c instead of its own local copies. This brings the module mostly up to par with the TCPDNS and TLSDNS modules.
2021-03-31 12:14:54 +02:00
case isc_nm_tcpsocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
case isc_nm_tcpdnssocket:
break;
case isc_nm_tlsdnssocket:
/*
* We need to limit the individual chunks to be read, so the
* BIO_write() will always succeed and the consumed before the
* next readcb is called.
*/
if (size >= ISC_NETMGR_TLSBUF_SIZE) {
size = ISC_NETMGR_TLSBUF_SIZE;
}
break;
default:
INSIST(0);
ISC_UNREACHABLE();
}
worker = &sock->mgr->workers[sock->tid];
isc_nm_*connect() always return via callback The isc_nm_*connect() functions were refactored to always return the connection status via the connect callback instead of sometimes returning the hard failure directly (for example, when the socket could not be created, or when the network manager was shutting down). This commit changes the connect functions in all the network manager modules, and also makes the necessary refactoring changes in places where the connect functions are called.
2021-03-31 18:32:32 +02:00
INSIST(!worker->recvbuf_inuse || sock->type == isc_nm_udpsocket);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
buf->base = worker->recvbuf;
buf->len = size;
worker->recvbuf_inuse = true;
}
void
isc__nm_start_reading(isc_nmsocket_t *sock) {
int r;
if (sock->reading) {
return;
}
switch (sock->type) {
case isc_nm_udpsocket:
r = uv_udp_recv_start(&sock->uv_handle.udp, isc__nm_alloc_cb,
isc__nm_udp_read_cb);
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
break;
Use generic functions for reading and timers in TCP The TCP module has been updated to use the generic functions from netmgr.c instead of its own local copies. This brings the module mostly up to par with the TCPDNS and TLSDNS modules.
2021-03-31 12:14:54 +02:00
case isc_nm_tcpsocket:
r = uv_read_start(&sock->uv_handle.stream, isc__nm_alloc_cb,
isc__nm_tcp_read_cb);
break;
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
case isc_nm_tcpdnssocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
r = uv_read_start(&sock->uv_handle.stream, isc__nm_alloc_cb,
isc__nm_tcpdns_read_cb);
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnssocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
r = uv_read_start(&sock->uv_handle.stream, isc__nm_alloc_cb,
isc__nm_tlsdns_read_cb);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
break;
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
default:
INSIST(0);
ISC_UNREACHABLE();
}
RUNTIME_CHECK(r == 0);
sock->reading = true;
}
void
isc__nm_stop_reading(isc_nmsocket_t *sock) {
int r;
if (!sock->reading) {
return;
}
switch (sock->type) {
case isc_nm_udpsocket:
r = uv_udp_recv_stop(&sock->uv_handle.udp);
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
break;
Use generic functions for reading and timers in TCP The TCP module has been updated to use the generic functions from netmgr.c instead of its own local copies. This brings the module mostly up to par with the TCPDNS and TLSDNS modules.
2021-03-31 12:14:54 +02:00
case isc_nm_tcpsocket:
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
case isc_nm_tcpdnssocket:
case isc_nm_tlsdnssocket:
r = uv_read_stop(&sock->uv_handle.stream);
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
break;
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
default:
INSIST(0);
ISC_UNREACHABLE();
}
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
RUNTIME_CHECK(r == 0);
sock->reading = false;
}
bool
Fix and clean up handling of connect callbacks Serveral problems were discovered and fixed after the change in the connection timeout in the previous commits: * In TLSDNS, the connection callback was not called at all under some circumstances when the TCP connection had been established, but the TLS handshake hadn't been completed yet. Additional checks have been put in place so that tls_cycle() will end early when the nmsocket is invalidated by the isc__nm_tlsdns_shutdown() call. * In TCP, TCPDNS and TLSDNS, new connections would be established even when the network manager was shutting down. The new call isc__nm_closing() has been added and is used to bail out early even before uv_tcp_connect() is attempted.
2021-03-31 11:48:41 +02:00
isc__nm_closing(isc_nmsocket_t *sock) {
return (atomic_load(&sock->mgr->closing));
}
bool
isc__nmsocket_closing(isc_nmsocket_t *sock) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return (!isc__nmsocket_active(sock) || atomic_load(&sock->closing) ||
atomic_load(&sock->mgr->closing) ||
(sock->server != NULL && !isc__nmsocket_active(sock->server)));
}
static isc_result_t
processbuffer(isc_nmsocket_t *sock) {
switch (sock->type) {
case isc_nm_tcpdnssocket:
return (isc__nm_tcpdns_processbuffer(sock));
case isc_nm_tlsdnssocket:
Fix typo in processbuffer() - tcpdns vs tlsdns The processbuffer() would call isc__nm_tcpdns_processbuffer() instead of isc__nm_tlsdns_processbuffer() for the isc_nm_tlsdnssocket type of socket.
2021-03-18 21:19:18 +01:00
return (isc__nm_tlsdns_processbuffer(sock));
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
default:
INSIST(0);
ISC_UNREACHABLE();
}
}
/*
* Process a DNS message.
*
* If we only have an incomplete DNS message, we don't touch any
* timers. If we do have a full message, reset the timer.
*
* Stop reading if this is a client socket, or if the server socket
* has been set to sequential mode, or the number of queries we are
* processing simultaneously has reached the clients-per-connection
* limit. In this case we'll be called again by resume_processing()
* later.
*/
void
isc__nm_process_sock_buffer(isc_nmsocket_t *sock) {
for (;;) {
int_fast32_t ah = atomic_load(&sock->ah);
isc_result_t result = processbuffer(sock);
switch (result) {
case ISC_R_NOMORE:
/*
* Don't reset the timer until we have a
* full DNS message.
*/
isc__nm_start_reading(sock);
/*
* Start the timer only if there are no externally used
* active handles, there's always one active handle
* attached internally to sock->recv_handle in
* accept_connection()
*/
if (ah == 1) {
isc__nmsocket_timer_start(sock);
}
return;
case ISC_R_CANCELED:
isc__nmsocket_timer_stop(sock);
isc__nm_stop_reading(sock);
return;
case ISC_R_SUCCESS:
/*
* Stop the timer on the successful message read, this
* also allows to restart the timer when we have no more
* data.
*/
isc__nmsocket_timer_stop(sock);
if (atomic_load(&sock->client) ||
atomic_load(&sock->sequential) ||
ah >= STREAM_CLIENTS_PER_CONN)
{
isc__nm_stop_reading(sock);
return;
}
break;
default:
INSIST(0);
}
}
}
void
isc__nm_resume_processing(void *arg) {
isc_nmsocket_t *sock = (isc_nmsocket_t *)arg;
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(sock->tid == isc_nm_tid());
REQUIRE(!atomic_load(&sock->client));
Fix and clean up handling of connect callbacks Serveral problems were discovered and fixed after the change in the connection timeout in the previous commits: * In TLSDNS, the connection callback was not called at all under some circumstances when the TCP connection had been established, but the TLS handshake hadn't been completed yet. Additional checks have been put in place so that tls_cycle() will end early when the nmsocket is invalidated by the isc__nm_tlsdns_shutdown() call. * In TCP, TCPDNS and TLSDNS, new connections would be established even when the network manager was shutting down. The new call isc__nm_closing() has been added and is used to bail out early even before uv_tcp_connect() is attempted.
2021-03-31 11:48:41 +02:00
if (isc__nmsocket_closing(sock)) {
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return;
}
isc__nm_process_sock_buffer(sock);
}
void
isc_nmhandle_cleartimeout(isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
switch (handle->sock->type) {
case isc_nm_httpsocket:
isc__nm_http_cleartimeout(handle);
return;
case isc_nm_tlssocket:
isc__nm_tls_cleartimeout(handle);
return;
default:
handle->sock->read_timeout = 0;
if (uv_is_active((uv_handle_t *)&handle->sock->timer)) {
isc__nmsocket_timer_stop(handle->sock);
}
}
}
void
isc_nmhandle_settimeout(isc_nmhandle_t *handle, uint32_t timeout) {
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
switch (handle->sock->type) {
case isc_nm_httpsocket:
isc__nm_http_settimeout(handle, timeout);
return;
case isc_nm_tlssocket:
isc__nm_tls_settimeout(handle, timeout);
return;
default:
handle->sock->read_timeout = timeout;
Make it possible to recover from read timeouts Previously, when the client timed out on read, the client socket would be automatically closed and destroyed when the nmhandle was detached. This commit changes the logic so that it's possible for the callback to recover from the ISC_R_TIMEDOUT event by restarting the timer. This is done by calling isc_nmhandle_settimeout(), which prevents the timeout handling code from destroying the socket; instead, it continues to wait for data. One specific use case for multiple timeouts is serve-stale - the client socket could be created with shorter timeout (as specified with stale-answer-client-timeout), so we can serve the requestor with stale answer, but keep the original query running for a longer time.
2021-03-29 10:52:05 +02:00
isc__nmsocket_timer_restart(handle->sock);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
}
add isc_nmhandle_settimeout() function this function sets the read timeout for the socket associated with a netmgr handle and, if the timer is running, resets it. for TCPDNS sockets it also sets the read timeout and resets the timer on the outer TCP socket.
2020-11-02 19:58:05 -08:00
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
void *
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_getextra(isc_nmhandle_t *handle) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
return (handle->extra);
}
isc_sockaddr_t
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_peeraddr(isc_nmhandle_t *handle) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
return (handle->peer);
}
isc_sockaddr_t
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_localaddr(isc_nmhandle_t *handle) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
return (handle->local);
}
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
isc_nm_t *
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_netmgr(isc_nmhandle_t *handle) {
netmgr: make TCP timeouts configurable - restore support for tcp-initial-timeout, tcp-idle-timeout, tcp-keepalive-timeout and tcp-advertised-timeout configuration options, which were ineffective previously.
2019-11-20 22:33:35 +01:00
REQUIRE(VALID_NMHANDLE(handle));
REQUIRE(VALID_NMSOCK(handle->sock));
return (handle->sock->mgr);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_uvreq_t *
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nm_uvreq_get(isc_nm_t *mgr, isc_nmsocket_t *sock FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_uvreq_t *req = NULL;
REQUIRE(VALID_NM(mgr));
REQUIRE(VALID_NMSOCK(sock));
Fix the data race when read-writing sock->active by using cmpxchg
2020-10-22 10:07:56 +02:00
if (sock != NULL && isc__nmsocket_active(sock)) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
/* Try to reuse one */
req = isc_astack_pop(sock->inactivereqs);
}
if (req == NULL) {
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
req = isc_mempool_get(mgr->reqpool);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
*req = (isc__nm_uvreq_t){ .magic = 0 };
netmgr: server-side TLS support Add server-side TLS support to netmgr - that includes moving some of the isc_nm_ functions from tcp.c to a wrapper in netmgr.c calling a proper tcp or tls function, and a new isc_nm_listentls() function. Add DoT support to tcpdns - isc_nm_listentlsdns().
2020-05-13 17:37:51 +02:00
ISC_LINK_INIT(req, link);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
req->uv_req.req.data = req;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_attach(sock, &req->sock FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
req->magic = UVREQ_MAGIC;
return (req);
}
void
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nm_uvreq_put(isc__nm_uvreq_t **req0, isc_nmsocket_t *sock FLARG) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
isc__nm_uvreq_t *req = NULL;
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nmhandle_t *handle = NULL;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(req0 != NULL);
REQUIRE(VALID_UVREQ(*req0));
req = *req0;
*req0 = NULL;
INSIST(sock == req->sock);
req->magic = 0;
/*
* We need to save this first to make sure that handle,
* sock, and the netmgr won't all disappear.
*/
handle = req->handle;
req->handle = NULL;
Fix the data race when read-writing sock->active by using cmpxchg
2020-10-22 10:07:56 +02:00
if (!isc__nmsocket_active(sock) ||
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
!isc_astack_trypush(sock->inactivereqs, req)) {
netmgr: performance improvement - use memory pools for ievent and uvreq objects.
2019-11-21 17:08:06 -08:00
isc_mempool_put(sock->mgr->reqpool, req);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
if (handle != NULL) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nmhandle_detach(&handle FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc___nmsocket_detach(&sock FLARG_PASS);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
}
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
void
Use clang-format to reformat the source files
2020-02-12 13:59:18 +01:00
isc_nm_send(isc_nmhandle_t *handle, isc_region_t *region, isc_nm_cb_t cb,
apply the modified style
2020-02-13 14:44:37 -08:00
void *cbarg) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
REQUIRE(VALID_NMHANDLE(handle));
switch (handle->sock->type) {
case isc_nm_udpsocket:
case isc_nm_udplistener:
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc__nm_udp_send(handle, region, cb, cbarg);
break;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
case isc_nm_tcpsocket:
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc__nm_tcp_send(handle, region, cb, cbarg);
break;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
case isc_nm_tcpdnssocket:
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc__nm_tcpdns_send(handle, region, cb, cbarg);
break;
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
isc__nm_tls_send(handle, region, cb, cbarg);
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnssocket:
isc__nm_tlsdns_send(handle, region, cb, cbarg);
break;
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
case isc_nm_httpsocket:
nghttp2-based HTTP layer in netmgr This commit includes work-in-progress implementation of DNS-over-HTTP(S). Server-side code remains mostly untested, and there is only support for POST requests.
2020-10-31 20:42:18 +01:00
isc__nm_http_send(handle, region, cb, cbarg);
break;
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
default:
INSIST(0);
ISC_UNREACHABLE();
}
}
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
void
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
isc_nm_read(isc_nmhandle_t *handle, isc_nm_recv_cb_t cb, void *cbarg) {
REQUIRE(VALID_NMHANDLE(handle));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
/*
* This is always called via callback (from accept or connect), and
* caller must attach to the handle, so the references always need to be
* at least 2.
*/
REQUIRE(isc_refcount_current(&handle->references) >= 2);
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
switch (handle->sock->type) {
add netmgr functions to support outgoing DNS queries - isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection. - isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a timeout argument to ensure connections time out and are correctly cleaned up on failure. - isc_nm_read() now supports UDP; it reads a single datagram and then stops until the next time it's called. - isc_nm_cancelread() now runs asynchronously to prevent assertion failure if reading is interrupted by a non-network thread (e.g. a timeout). - isc_nm_cancelread() can now apply to UDP sockets. - added shim code to support UDP connection in versions of libuv prior to 1.27, when uv_udp_connect() was added all these functions will be used to support outgoing queries in dig, xfrin, dispatch, etc.
2020-09-05 11:07:40 -07:00
case isc_nm_udpsocket:
isc__nm_udp_read(handle, cb, cbarg);
break;
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
case isc_nm_tcpsocket:
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc__nm_tcp_read(handle, cb, cbarg);
break;
add netmgr functions to support outgoing DNS queries - isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection. - isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a timeout argument to ensure connections time out and are correctly cleaned up on failure. - isc_nm_read() now supports UDP; it reads a single datagram and then stops until the next time it's called. - isc_nm_cancelread() now runs asynchronously to prevent assertion failure if reading is interrupted by a non-network thread (e.g. a timeout). - isc_nm_cancelread() can now apply to UDP sockets. - added shim code to support UDP connection in versions of libuv prior to 1.27, when uv_udp_connect() was added all these functions will be used to support outgoing queries in dig, xfrin, dispatch, etc.
2020-09-05 11:07:40 -07:00
case isc_nm_tcpdnssocket:
isc__nm_tcpdns_read(handle, cb, cbarg);
break;
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
isc__nm_tls_read(handle, cb, cbarg);
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnssocket:
isc__nm_tlsdns_read(handle, cb, cbarg);
break;
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
case isc_nm_httpsocket:
isc__nm_http_read(handle, cb, cbarg);
break;
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
default:
INSIST(0);
ISC_UNREACHABLE();
}
}
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
void
isc_nm_cancelread(isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
switch (handle->sock->type) {
add netmgr functions to support outgoing DNS queries - isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection. - isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a timeout argument to ensure connections time out and are correctly cleaned up on failure. - isc_nm_read() now supports UDP; it reads a single datagram and then stops until the next time it's called. - isc_nm_cancelread() now runs asynchronously to prevent assertion failure if reading is interrupted by a non-network thread (e.g. a timeout). - isc_nm_cancelread() can now apply to UDP sockets. - added shim code to support UDP connection in versions of libuv prior to 1.27, when uv_udp_connect() was added all these functions will be used to support outgoing queries in dig, xfrin, dispatch, etc.
2020-09-05 11:07:40 -07:00
case isc_nm_udpsocket:
isc__nm_udp_cancelread(handle);
break;
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
case isc_nm_tcpsocket:
assorted small netmgr-related changes - rename isc_nmsocket_t->tcphandle to statichandle - cancelread functions now take handles instead of sockets - add a 'client' flag in socket objects, currently unused, to indicate whether it is to be used as a client or server socket
2020-06-10 11:32:39 +02:00
isc__nm_tcp_cancelread(handle);
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
break;
add netmgr functions to support outgoing DNS queries - isc_nm_tcpdnsconnect() sets up up an outgoing TCP DNS connection. - isc_nm_tcpconnect(), _udpconnect() and _tcpdnsconnect() now take a timeout argument to ensure connections time out and are correctly cleaned up on failure. - isc_nm_read() now supports UDP; it reads a single datagram and then stops until the next time it's called. - isc_nm_cancelread() now runs asynchronously to prevent assertion failure if reading is interrupted by a non-network thread (e.g. a timeout). - isc_nm_cancelread() can now apply to UDP sockets. - added shim code to support UDP connection in versions of libuv prior to 1.27, when uv_udp_connect() was added all these functions will be used to support outgoing queries in dig, xfrin, dispatch, etc.
2020-09-05 11:07:40 -07:00
case isc_nm_tcpdnssocket:
isc__nm_tcpdns_cancelread(handle);
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnssocket:
isc__nm_tlsdns_cancelread(handle);
break;
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
isc__nm_tls_cancelread(handle);
break;
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
default:
INSIST(0);
ISC_UNREACHABLE();
}
}
Fix the data race when read-writing sock->active by using cmpxchg
2020-10-22 10:07:56 +02:00
void
use handles for isc_nm_pauseread() and isc_nm_resumeread() by having these functions act on netmgr handles instead of socket objects, they can be used in callback functions outside the netgmr.
2020-07-01 16:17:09 -07:00
isc_nm_pauseread(isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
isc_nmsocket_t *sock = handle->sock;
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
switch (sock->type) {
case isc_nm_tcpsocket:
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_tcp_pauseread(handle);
Fix the data race when read-writing sock->active by using cmpxchg
2020-10-22 10:07:56 +02:00
break;
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
isc__nm_tls_pauseread(handle);
break;
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
default:
INSIST(0);
ISC_UNREACHABLE();
}
}
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
void
use handles for isc_nm_pauseread() and isc_nm_resumeread() by having these functions act on netmgr handles instead of socket objects, they can be used in callback functions outside the netgmr.
2020-07-01 16:17:09 -07:00
isc_nm_resumeread(isc_nmhandle_t *handle) {
REQUIRE(VALID_NMHANDLE(handle));
isc_nmsocket_t *sock = handle->sock;
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
switch (sock->type) {
case isc_nm_tcpsocket:
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
isc__nm_tcp_resumeread(handle);
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
break;
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
isc__nm_tls_resumeread(handle);
break;
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
default:
INSIST(0);
ISC_UNREACHABLE();
}
}
void
isc_nm_stoplistening(isc_nmsocket_t *sock) {
REQUIRE(VALID_NMSOCK(sock));
implement isc_nm_cancelread() The isc_nm_cancelread() function cancels reading on a connected socket and calls its read callback function with a 'result' parameter of ISC_R_CANCELED.
2020-06-05 17:32:36 -07:00
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
switch (sock->type) {
case isc_nm_udplistener:
isc__nm_udp_stoplistening(sock);
break;
case isc_nm_tcpdnslistener:
isc__nm_tcpdns_stoplistening(sock);
break;
case isc_nm_tcplistener:
isc__nm_tcp_stoplistening(sock);
break;
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlslistener:
isc__nm_tls_stoplistening(sock);
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnslistener:
isc__nm_tlsdns_stoplistening(sock);
break;
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
case isc_nm_httplistener:
isc__nm_http_stoplistening(sock);
break;
netmgr refactoring: use generic functions when operating on sockets. tcpdns used transport-specific functions to operate on the outer socket. Use generic ones instead, and select the proper call in netmgr.c. Make the missing functions (e.g. isc_nm_read) generic and add type-specific calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.
2020-03-20 11:55:10 +01:00
default:
INSIST(0);
ISC_UNREACHABLE();
}
}
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
void
isc__nm_connectcb(isc_nmsocket_t *sock, isc__nm_uvreq_t *uvreq,
isc_result_t eresult, bool async) {
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
if (!async) {
isc__netievent_connectcb_t ievent = { .sock = sock,
.req = uvreq,
.result = eresult };
isc__nm_async_connectcb(NULL, (isc__netievent_t *)&ievent);
isc_nm_*connect() always return via callback The isc_nm_*connect() functions were refactored to always return the connection status via the connect callback instead of sometimes returning the hard failure directly (for example, when the socket could not be created, or when the network manager was shutting down). This commit changes the connect functions in all the network manager modules, and also makes the necessary refactoring changes in places where the connect functions are called.
2021-03-31 18:32:32 +02:00
} else {
isc__netievent_connectcb_t *ievent =
isc__nm_get_netievent_connectcb(sock->mgr, sock, uvreq,
eresult);
isc__nm_enqueue_ievent(&sock->mgr->workers[sock->tid],
(isc__netievent_t *)ievent);
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
}
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
}
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
void
isc__nm_async_connectcb(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_connectcb_t *ievent = (isc__netievent_connectcb_t *)ev0;
isc_nmsocket_t *sock = ievent->sock;
isc__nm_uvreq_t *uvreq = ievent->req;
isc_result_t eresult = ievent->result;
UNUSED(worker);
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
REQUIRE(ievent->sock->tid == isc_nm_tid());
REQUIRE(uvreq->cb.connect != NULL);
uvreq->cb.connect(uvreq->handle, eresult, uvreq->cbarg);
isc__nm_uvreq_put(&uvreq, sock);
}
void
isc__nm_readcb(isc_nmsocket_t *sock, isc__nm_uvreq_t *uvreq,
isc_result_t eresult) {
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
run read callbacks synchronously on timeout when running read callbacks, if the event result is not ISC_R_SUCCESS, the callback is always run asynchronously. this is a problem on timeout, because there's no chance to reset the timer before the socket has already been destroyed. this commit allows read callbacks to run synchronously for both ISC_R_SUCCESS and ISC_R_TIMEDOUT result codes.
2021-04-20 23:27:51 -07:00
if (eresult == ISC_R_SUCCESS || eresult == ISC_R_TIMEDOUT) {
Avoid netievent allocations when the callbacks can be called directly After turning the users callbacks to be asynchronous, there was a visible performance drop. This commit prevents the unnecessary allocations while keeping the code paths same for both asynchronous and synchronous calls. The same change was done to the isc__nm_udp_{read,send} as those two functions are in the hot path.
2020-12-02 08:54:51 +01:00
isc__netievent_readcb_t ievent = { .sock = sock,
.req = uvreq,
.result = eresult };
isc__nm_async_readcb(NULL, (isc__netievent_t *)&ievent);
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
} else {
Avoid netievent allocations when the callbacks can be called directly After turning the users callbacks to be asynchronous, there was a visible performance drop. This commit prevents the unnecessary allocations while keeping the code paths same for both asynchronous and synchronous calls. The same change was done to the isc__nm_udp_{read,send} as those two functions are in the hot path.
2020-12-02 08:54:51 +01:00
isc__netievent_readcb_t *ievent = isc__nm_get_netievent_readcb(
sock->mgr, sock, uvreq, eresult);
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
isc__nm_enqueue_ievent(&sock->mgr->workers[sock->tid],
(isc__netievent_t *)ievent);
}
}
void
isc__nm_async_readcb(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_readcb_t *ievent = (isc__netievent_readcb_t *)ev0;
isc_nmsocket_t *sock = ievent->sock;
isc__nm_uvreq_t *uvreq = ievent->req;
isc_result_t eresult = ievent->result;
isc_region_t region = { .base = (unsigned char *)uvreq->uvbuf.base,
.length = uvreq->uvbuf.len };
UNUSED(worker);
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
REQUIRE(sock->tid == isc_nm_tid());
uvreq->cb.recv(uvreq->handle, eresult, &region, uvreq->cbarg);
isc__nm_uvreq_put(&uvreq, sock);
}
void
isc__nm_sendcb(isc_nmsocket_t *sock, isc__nm_uvreq_t *uvreq,
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc_result_t eresult, bool async) {
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
if (!async) {
Avoid netievent allocations when the callbacks can be called directly After turning the users callbacks to be asynchronous, there was a visible performance drop. This commit prevents the unnecessary allocations while keeping the code paths same for both asynchronous and synchronous calls. The same change was done to the isc__nm_udp_{read,send} as those two functions are in the hot path.
2020-12-02 08:54:51 +01:00
isc__netievent_sendcb_t ievent = { .sock = sock,
.req = uvreq,
.result = eresult };
isc__nm_async_sendcb(NULL, (isc__netievent_t *)&ievent);
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
return;
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
}
Fix TCPDNS and TLSDNS timers After the TCPDNS refactoring the initial and idle timers were broken and only the tcp-initial-timeout was always applied on the whole TCP connection. This broke any TCP connection that took longer than tcp-initial-timeout, most often this would affect large zone AXFRs. This commit changes the timeout logic in this way: * On TCP connection accept the tcp-initial-timeout is applied and the timer is started * When we are processing and/or sending any DNS message the timer is stopped * When we stop processing all DNS messages, the tcp-idle-timeout is applied and the timer is started again
2021-03-16 09:03:02 +01:00
isc__netievent_sendcb_t *ievent =
isc__nm_get_netievent_sendcb(sock->mgr, sock, uvreq, eresult);
isc__nm_enqueue_ievent(&sock->mgr->workers[sock->tid],
(isc__netievent_t *)ievent);
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
}
void
isc__nm_async_sendcb(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_sendcb_t *ievent = (isc__netievent_sendcb_t *)ev0;
isc_nmsocket_t *sock = ievent->sock;
isc__nm_uvreq_t *uvreq = ievent->req;
isc_result_t eresult = ievent->result;
UNUSED(worker);
REQUIRE(VALID_NMSOCK(sock));
REQUIRE(VALID_UVREQ(uvreq));
REQUIRE(VALID_NMHANDLE(uvreq->handle));
REQUIRE(sock->tid == isc_nm_tid());
uvreq->cb.send(uvreq->handle, eresult, uvreq->cbarg);
isc__nm_uvreq_put(&uvreq, sock);
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
static void
isc__nm_async_close(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_close_t *ievent = (isc__netievent_close_t *)ev0;
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
isc_nmsocket_t *sock = ievent->sock;
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
REQUIRE(VALID_NMSOCK(ievent->sock));
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
REQUIRE(sock->tid == isc_nm_tid());
REQUIRE(sock->closehandle_cb != NULL);
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
UNUSED(worker);
Turn all the callback to be always asynchronous When calling the high level netmgr functions, the callback would be sometimes called synchronously if we catch the failure directly, or asynchronously if it happens later. The synchronous call to the callback could create deadlocks as the caller would not expect the failed callback to be executed directly.
2020-11-11 10:46:33 +01:00
ievent->sock->closehandle_cb(sock);
netmgr: TCP improvements - add timeout support for TCP and TCPDNS connections to protect against slowloris style attacks. currently, all timeouts are hard-coded. - rework and simplify the TCPDNS state machine.
2019-11-19 11:56:00 +01:00
}
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
void
isc__nm_async_detach(isc__networker_t *worker, isc__netievent_t *ev0) {
isc__netievent_detach_t *ievent = (isc__netievent_detach_t *)ev0;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
FLARG_IEVENT(ievent);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
REQUIRE(VALID_NMSOCK(ievent->sock));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
REQUIRE(VALID_NMHANDLE(ievent->handle));
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
REQUIRE(ievent->sock->tid == isc_nm_tid());
UNUSED(worker);
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
nmhandle_detach_cb(&ievent->handle FLARG_PASS);
Complete the isc_nmhandle_detach() in the worker thread. isc_nmhandle_detach() needs to complete in the same thread as shutdown_walk_cb() to avoid a race. Clear the caller's pointer then pass control to the worker if necessary. WARNING: ThreadSanitizer: data race Write of size 8 at 0x000000000001 by thread T1: #0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15 #1 control_command bin/named/controlconf.c:388:3 #2 dispatch lib/isc/task.c:1152:7 #3 run lib/isc/task.c:1344:2 Previous read of size 8 at 0x000000000001 by thread T2: #0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33 #1 recv_data lib/isccc/ccmsg.c:109:2 #2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4 #3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3 #4 uv_walk <null> #5 process_queue lib/isc/netmgr/netmgr.c:659:4 #6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10 #7 process_queues lib/isc/netmgr/netmgr.c:590:8 #8 async_cb lib/isc/netmgr/netmgr.c:548:2 #9 <null> <null>
2020-10-12 17:51:09 +11:00
}
Fix and clean up handling of connect callbacks Serveral problems were discovered and fixed after the change in the connection timeout in the previous commits: * In TLSDNS, the connection callback was not called at all under some circumstances when the TCP connection had been established, but the TLS handshake hadn't been completed yet. Additional checks have been put in place so that tls_cycle() will end early when the nmsocket is invalidated by the isc__nm_tlsdns_shutdown() call. * In TCP, TCPDNS and TLSDNS, new connections would be established even when the network manager was shutting down. The new call isc__nm_closing() has been added and is used to bail out early even before uv_tcp_connect() is attempted.
2021-03-31 11:48:41 +02:00
void
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
isc__nmsocket_shutdown(isc_nmsocket_t *sock) {
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
REQUIRE(VALID_NMSOCK(sock));
switch (sock->type) {
case isc_nm_udpsocket:
Fix netmgr read/connect timeout issues - don't bother closing sockets that are already closing. - UDP read timeout timer was not stopped after reading. - improve handling of TCP connection failures.
2020-10-26 14:19:37 +01:00
isc__nm_udp_shutdown(sock);
Add isc__nm_udp_shutdown() function This function will be called during isc_nm_closedown() to ensure that all UDP sockets are closed and detached.
2020-10-26 12:30:54 +01:00
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tcpsocket:
Fix netmgr read/connect timeout issues - don't bother closing sockets that are already closing. - UDP read timeout timer was not stopped after reading. - improve handling of TCP connection failures.
2020-10-26 14:19:37 +01:00
isc__nm_tcp_shutdown(sock);
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tcpdnssocket:
isc__nm_tcpdns_shutdown(sock);
break;
case isc_nm_tlsdnssocket:
Refactor TLSDNS module to work with libuv/ssl directly * Following the example set in 634bdfb16d8, the tlsdns netmgr module now uses libuv and SSL primitives directly, rather than opening a TLS socket which opens a TCP socket, as the previous model was difficult to debug. Closes #2335. * Remove the netmgr tls layer (we will have to re-add it for DoH) * Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc library; move the OpenSSL initialization/deinitialization from dstapi needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}() * Add couple of new shims needed for OpenSSL 1.0.x * When LibreSSL is used, require at least version 2.7.0 that has the best OpenSSL 1.1.x compatibility and auto init/deinit * Enforce OpenSSL 1.1.x usage on Windows * Added a TLSDNS unit test and implemented a simple TLSDNS echo server and client.
2020-12-17 11:40:29 +01:00
isc__nm_tlsdns_shutdown(sock);
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
break;
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_udplistener:
case isc_nm_tcplistener:
case isc_nm_tcpdnslistener:
case isc_nm_tlsdnslistener:
return;
default:
INSIST(0);
ISC_UNREACHABLE();
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
}
}
Make it possible to recover from connect timeouts Similarly to the read timeout, it's now possible to recover from ISC_R_TIMEDOUT event by restarting the timer from the connect callback. The change here also fixes platforms that missing the socket() options to set the TCP connection timeout, by moving the timeout code into user space. On platforms that support setting the connect timeout via a socket option, the timeout has been hardcoded to 2 minutes (the maximum value of tcp-initial-timeout).
2021-03-30 09:25:09 +02:00
static void
shutdown_walk_cb(uv_handle_t *handle, void *arg) {
isc_nmsocket_t *sock = uv_handle_get_data(handle);
UNUSED(arg);
if (uv_is_closing(handle)) {
return;
}
switch (handle->type) {
case UV_UDP:
case UV_TCP:
break;
default:
return;
}
isc__nmsocket_shutdown(sock);
}
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_async_shutdown(isc__networker_t *worker, isc__netievent_t *ev0) {
shorten some names reduce line breaks and general unwieldiness by changing some function, type, and parameter names.
2019-12-09 12:24:46 -08:00
UNUSED(ev0);
netmgr: actively close all sockets when shutting down server without this change, named could sometimes lag for a while on shutdown while it waited for open TCP connections to time out.
2019-11-22 14:13:19 +01:00
uv_walk(&worker->loop, shutdown_walk_cb, NULL);
}
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
bool
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_acquire_interlocked(isc_nm_t *mgr) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
LOCK(&mgr->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
bool success = atomic_compare_exchange_strong(
&mgr->interlocked, &(int){ ISC_NETMGR_NON_INTERLOCKED },
isc_nm_tid());
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
UNLOCK(&mgr->lock);
return (success);
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_drop_interlocked(isc_nm_t *mgr) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
LOCK(&mgr->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
int tid = atomic_exchange(&mgr->interlocked,
ISC_NETMGR_NON_INTERLOCKED);
INSIST(tid != ISC_NETMGR_NON_INTERLOCKED);
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
BROADCAST(&mgr->wkstatecond);
UNLOCK(&mgr->lock);
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_acquire_interlocked_force(isc_nm_t *mgr) {
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
LOCK(&mgr->lock);
Refactor taskmgr to run on top of netmgr This commit changes the taskmgr to run the individual tasks on the netmgr internal workers. While an effort has been put into keeping the taskmgr interface intact, couple of changes have been made: * The taskmgr has no concept of universal privileged mode - rather the tasks are either privileged or unprivileged (normal). The privileged tasks are run as a first thing when the netmgr is unpaused. There are now four different queues in in the netmgr: 1. priority queue - netievent on the priority queue are run even when the taskmgr enter exclusive mode and netmgr is paused. This is needed to properly start listening on the interfaces, free resources and resume. 2. privileged task queue - only privileged tasks are queued here and this is the first queue that gets processed when network manager is unpaused using isc_nm_resume(). All netmgr workers need to clean the privileged task queue before they all proceed normal operation. Both task queues are processed when the workers are finished. 3. task queue - only (traditional) task are scheduled here and this queue along with privileged task queues are process when the netmgr workers are finishing. This is needed to process the task shutdown events. 4. normal queue - this is the queue with netmgr events, e.g. reading, sending, callbacks and pretty much everything is processed here. * The isc_taskmgr_create() now requires initialized netmgr (isc_nm_t) object. * The isc_nm_destroy() function now waits for indefinite time, but it will print out the active objects when in tracing mode (-DNETMGR_TRACE=1 and -DNETMGR_TRACE_VERBOSE=1), the netmgr has been made a little bit more asynchronous and it might take longer time to shutdown all the active networking connections. * Previously, the isc_nm_stoplistening() was a synchronous operation. This has been changed and the isc_nm_stoplistening() just schedules the child sockets to stop listening and exits. This was needed to prevent a deadlock as the the (traditional) tasks are now executed on the netmgr threads. * The socket selection logic in isc__nm_udp_send() was flawed, but fortunatelly, it was broken, so we never hit the problem where we created uvreq_t on a socket from nmhandle_t, but then a different socket could be picked up and then we were trying to run the send callback on a socket that had different threadid than currently running.
2021-04-09 11:31:19 +02:00
while (!atomic_compare_exchange_strong(
&mgr->interlocked, &(int){ ISC_NETMGR_NON_INTERLOCKED },
isc_nm_tid()))
apply the modified style
2020-02-13 14:44:37 -08:00
{
netmgr: libuv-based network manager This is a replacement for the existing isc_socket and isc_socketmgr implementation. It uses libuv for asynchronous network communication; "networker" objects will be distributed across worker threads reading incoming packets and sending them for processing. UDP listener sockets automatically create an array of "child" sockets so each worker can listen separately. TCP sockets are shared amongst worker threads. A TCPDNS socket is a wrapper around a TCP socket, which handles the the two-byte length field at the beginning of DNS messages over TCP. (Other wrapper socket types can be implemented in the future to handle DNS over TLS, DNS over HTTPS, etc.)
2019-11-05 13:55:54 -08:00
WAIT(&mgr->wkstatecond, &mgr->lock);
}
UNLOCK(&mgr->lock);
}
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc_nm_setstats(isc_nm_t *mgr, isc_stats_t *stats) {
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
REQUIRE(VALID_NM(mgr));
REQUIRE(mgr->stats == NULL);
REQUIRE(isc_stats_ncounters(stats) == isc_sockstatscounter_max);
isc_stats_attach(stats, &mgr->stats);
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_incstats(isc_nm_t *mgr, isc_statscounter_t counterid) {
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
REQUIRE(VALID_NM(mgr));
REQUIRE(counterid != -1);
if (mgr->stats != NULL) {
isc_stats_increment(mgr->stats, counterid);
}
}
void
apply the modified style
2020-02-13 14:44:37 -08:00
isc__nm_decstats(isc_nm_t *mgr, isc_statscounter_t counterid) {
associate socket stats counters with netmgr socket objects - the socket stat counters have been moved from socket.h to stats.h. - isc_nm_t now attaches to the same stats counter group as isc_socketmgr_t, so that both managers can increment the same set of statistics - isc__nmsocket_init() now takes an interface as a paramter so that the address family can be determined when initializing the socket. - based on the address family and socket type, a group of statistics counters will be associated with the socket - for example, UDP4Active with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets. note that no counters are currently associated with TCPDNS sockets; those stats will be handled by the underlying TCP socket. - the counters are not actually used by netmgr sockets yet; counter increment and decrement calls will be added in a later commit.
2020-01-05 01:02:12 -08:00
REQUIRE(VALID_NM(mgr));
REQUIRE(counterid != -1);
if (mgr->stats != NULL) {
isc_stats_decrement(mgr->stats, counterid);
}
}
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
netmgr: Don't crash if socket() returns an error in udpconnect socket() call can return an error - e.g. EMFILE, so we need to handle this nicely and not crash. Additionally wrap the socket() call inside a platform independent helper function as the Socket data type on Windows is unsigned integer: > This means, for example, that checking for errors when the socket and > accept functions return should not be done by comparing the return > value with –1, or seeing if the value is negative (both common and > legal approaches in UNIX). Instead, an application should use the > manifest constant INVALID_SOCKET as defined in the Winsock2.h header > file.
2020-11-07 20:48:37 +01:00
isc_result_t
isc__nm_socket(int domain, int type, int protocol, uv_os_sock_t *sockp) {
#ifdef WIN32
SOCKET sock;
sock = socket(domain, type, protocol);
if (sock == INVALID_SOCKET) {
char strbuf[ISC_STRERRORSIZE];
DWORD socket_errno = WSAGetLastError();
switch (socket_errno) {
case WSAEMFILE:
case WSAENOBUFS:
return (ISC_R_NORESOURCES);
case WSAEPROTONOSUPPORT:
case WSAEPFNOSUPPORT:
case WSAEAFNOSUPPORT:
return (ISC_R_FAMILYNOSUPPORT);
default:
strerror_r(socket_errno, strbuf, sizeof(strbuf));
Extend log message for unexpected socket() errors Make sure the error code is included in the message logged for unexpected socket creation errors in order to facilitate troubleshooting on Windows.
2020-12-02 22:36:23 +01:00
UNEXPECTED_ERROR(
__FILE__, __LINE__,
"socket() failed with error code %lu: %s",
socket_errno, strbuf);
netmgr: Don't crash if socket() returns an error in udpconnect socket() call can return an error - e.g. EMFILE, so we need to handle this nicely and not crash. Additionally wrap the socket() call inside a platform independent helper function as the Socket data type on Windows is unsigned integer: > This means, for example, that checking for errors when the socket and > accept functions return should not be done by comparing the return > value with –1, or seeing if the value is negative (both common and > legal approaches in UNIX). Instead, an application should use the > manifest constant INVALID_SOCKET as defined in the Winsock2.h header > file.
2020-11-07 20:48:37 +01:00
return (ISC_R_UNEXPECTED);
}
}
#else
int sock = socket(domain, type, protocol);
if (sock < 0) {
return (isc_errno_toresult(errno));
}
#endif
*sockp = (uv_os_sock_t)sock;
return (ISC_R_SUCCESS);
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
void
isc__nm_closesocket(uv_os_sock_t sock) {
#ifdef WIN32
closesocket(sock);
#else
close(sock);
#endif
}
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#define setsockopt_on(socket, level, name) \
setsockopt(socket, level, name, &(int){ 1 }, sizeof(int))
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
#define setsockopt_off(socket, level, name) \
setsockopt(socket, level, name, &(int){ 1 }, sizeof(int))
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
isc_result_t
Use uv_os_sock_t instead of uv_os_fd_t for sockets On POSIX based systems both uv_os_sock_t and uv_os_fd_t are both typedef to int. That's not true on Windows, where uv_os_sock_t is SOCKET and uv_os_fd_t is HANDLE and they differ in level of indirection.
2020-10-05 12:25:19 +02:00
isc__nm_socket_freebind(uv_os_sock_t fd, sa_family_t sa_family) {
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
/*
* Set the IP_FREEBIND (or equivalent option) on the uv_handle.
*/
#ifdef IP_FREEBIND
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
UNUSED(sa_family);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
if (setsockopt_on(fd, IPPROTO_IP, IP_FREEBIND) == -1) {
return (ISC_R_FAILURE);
}
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
return (ISC_R_SUCCESS);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#elif defined(IP_BINDANY) || defined(IPV6_BINDANY)
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
if (sa_family == AF_INET) {
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#if defined(IP_BINDANY)
if (setsockopt_on(fd, IPPROTO_IP, IP_BINDANY) == -1) {
return (ISC_R_FAILURE);
}
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
return (ISC_R_SUCCESS);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#endif
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
} else if (sa_family == AF_INET6) {
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#if defined(IPV6_BINDANY)
if (setsockopt_on(fd, IPPROTO_IPV6, IPV6_BINDANY) == -1) {
return (ISC_R_FAILURE);
}
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
return (ISC_R_SUCCESS);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#endif
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
}
return (ISC_R_NOTIMPLEMENTED);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#elif defined(SO_BINDANY)
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
UNUSED(sa_family);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
if (setsockopt_on(fd, SOL_SOCKET, SO_BINDANY) == -1) {
return (ISC_R_FAILURE);
}
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
return (ISC_R_SUCCESS);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#else
Refactor isc__nm_socket_freebind() to take fd and sa_family as args The isc__nm_socket_freebind() has been refactored to match other isc__nm_socket_...() helper functions and take uv_os_fd_t and sa_family_t as function arguments.
2020-10-05 11:17:52 +02:00
UNUSED(fd);
UNUSED(sa_family);
return (ISC_R_NOTIMPLEMENTED);
netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned. When a new IPv6 interface/address appears it's first in a tentative state - in which we cannot bind to it, yet it's already being reported by the route socket. Because of that BIND9 is unable to listen on any newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or equivalent option on other OSes) and then retrying bind() call.
2020-07-21 13:29:14 +02:00
#endif
}
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
isc_result_t
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
isc__nm_socket_reuse(uv_os_sock_t fd) {
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
/*
* Generally, the SO_REUSEADDR socket option allows reuse of
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
* local addresses.
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
*
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
* On the BSDs, SO_REUSEPORT implies SO_REUSEADDR but with some
* additional refinements for programs that use multicast.
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
*
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
* On Linux, SO_REUSEPORT has different semantics: it _shares_ the port
* rather than steal it from the current listener, so we don't use it
* here, but rather in isc__nm_socket_reuse_lb().
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
*
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
* On Windows, it also allows a socket to forcibly bind to a port in use
* by another socket.
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
*/
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
#if defined(SO_REUSEPORT) && !defined(__linux__)
if (setsockopt_on(fd, SOL_SOCKET, SO_REUSEPORT) == -1) {
return (ISC_R_FAILURE);
}
return (ISC_R_SUCCESS);
#elif defined(SO_REUSEADDR)
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
if (setsockopt_on(fd, SOL_SOCKET, SO_REUSEADDR) == -1) {
return (ISC_R_FAILURE);
}
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
return (ISC_R_SUCCESS);
#else
UNUSED(fd);
return (ISC_R_NOTIMPLEMENTED);
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
#endif
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
}
isc_result_t
isc__nm_socket_reuse_lb(uv_os_sock_t fd) {
/*
* On FreeBSD 12+, SO_REUSEPORT_LB socket option allows sockets to be
* bound to an identical socket address. For UDP sockets, the use of
* this option can provide better distribution of incoming datagrams to
* multiple processes (or threads) as compared to the traditional
* technique of having multiple processes compete to receive datagrams
* on the same socket.
*
* On Linux, the same thing is achieved simply with SO_REUSEPORT.
*/
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
#if defined(SO_REUSEPORT_LB)
if (setsockopt_on(fd, SOL_SOCKET, SO_REUSEPORT_LB) == -1) {
return (ISC_R_FAILURE);
} else {
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
return (ISC_R_SUCCESS);
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
}
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
#elif defined(SO_REUSEPORT) && defined(__linux__)
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
if (setsockopt_on(fd, SOL_SOCKET, SO_REUSEPORT) == -1) {
return (ISC_R_FAILURE);
} else {
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
return (ISC_R_SUCCESS);
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
}
Split reusing the addr/port and load-balancing socket options The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning on different platform. In this commit, we split the function to set the reuse of address/port and setting the load-balancing into separate functions. The libuv library already have multiplatform support for setting SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address and port, but unfortunately, when used after the load-balancing socket options have been already set, it overrides the previous setting, so we need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT first and then enable the load-balancing socket option.
2020-10-05 13:14:04 +02:00
#else
UNUSED(fd);
return (ISC_R_NOTIMPLEMENTED);
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
#endif
}
isc_result_t
Use uv_os_sock_t instead of uv_os_fd_t for sockets On POSIX based systems both uv_os_sock_t and uv_os_fd_t are both typedef to int. That's not true on Windows, where uv_os_sock_t is SOCKET and uv_os_fd_t is HANDLE and they differ in level of indirection.
2020-10-05 12:25:19 +02:00
isc__nm_socket_incoming_cpu(uv_os_sock_t fd) {
Add SO_REUSEPORT and SO_INCOMING_CPU helper functions The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a separate helper functions.
2020-10-05 10:40:02 +02:00
#ifdef SO_INCOMING_CPU
if (setsockopt_on(fd, SOL_SOCKET, SO_INCOMING_CPU) == -1) {
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#else
UNUSED(fd);
#endif
return (ISC_R_NOTIMPLEMENTED);
}
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
isc_result_t
Use uv_os_sock_t instead of uv_os_fd_t for sockets On POSIX based systems both uv_os_sock_t and uv_os_fd_t are both typedef to int. That's not true on Windows, where uv_os_sock_t is SOCKET and uv_os_fd_t is HANDLE and they differ in level of indirection.
2020-10-05 12:25:19 +02:00
isc__nm_socket_dontfrag(uv_os_sock_t fd, sa_family_t sa_family) {
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
/*
* Set the Don't Fragment flag on IP packets
*/
if (sa_family == AF_INET6) {
#if defined(IPV6_DONTFRAG)
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
if (setsockopt_off(fd, IPPROTO_IPV6, IPV6_DONTFRAG) == -1) {
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#elif defined(IPV6_MTU_DISCOVER) && defined(IP_PMTUDISC_OMIT)
if (setsockopt(fd, IPPROTO_IPV6, IPV6_MTU_DISCOVER,
&(int){ IP_PMTUDISC_OMIT }, sizeof(int)) == -1)
{
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
#elif defined(IPV6_MTU_DISCOVER) && defined(IP_PMTUDISC_DONT)
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
if (setsockopt(fd, IPPROTO_IPV6, IPV6_MTU_DISCOVER,
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
&(int){ IP_PMTUDISC_DONT }, sizeof(int)) == -1)
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
{
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#else
UNUSED(fd);
#endif
} else if (sa_family == AF_INET) {
#if defined(IP_DONTFRAG)
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
if (setsockopt_off(fd, IPPROTO_IP, IP_DONTFRAG) == -1) {
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#elif defined(IP_MTU_DISCOVER) && defined(IP_PMTUDISC_OMIT)
if (setsockopt(fd, IPPROTO_IP, IP_MTU_DISCOVER,
&(int){ IP_PMTUDISC_OMIT }, sizeof(int)) == -1)
{
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
#elif defined(IP_MTU_DISCOVER) && defined(IP_PMTUDISC_DONT)
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
if (setsockopt(fd, IPPROTO_IP, IP_MTU_DISCOVER,
Rollback setting IP_DONTFRAG option on the UDP sockets In DNS Flag Day 2020, the development branch started setting the IP_DONTFRAG option on the UDP sockets. It turned out, that this code was incomplete leading to dropping the outgoing UDP packets. Henceforth this commit rolls back this setting until we have a proper fix that would send back empty response with TC flag set.
2021-02-11 08:37:52 +01:00
&(int){ IP_PMTUDISC_DONT }, sizeof(int)) == -1)
Add helper function to enable DF (don't fragment) flag on UDP sockets This commits add isc__nm_socket_dontfrag() helper functions.
2020-10-05 10:51:40 +02:00
{
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#else
UNUSED(fd);
#endif
} else {
return (ISC_R_FAMILYNOSUPPORT);
}
return (ISC_R_NOTIMPLEMENTED);
}
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
#if defined(_WIN32)
#define TIMEOUT_TYPE DWORD
#define TIMEOUT_DIV 1000
#define TIMEOUT_OPTNAME TCP_MAXRT
#elif defined(TCP_CONNECTIONTIMEOUT)
#define TIMEOUT_TYPE int
#define TIMEOUT_DIV 1000
#define TIMEOUT_OPTNAME TCP_CONNECTIONTIMEOUT
#elif defined(TCP_RXT_CONNDROPTIME)
#define TIMEOUT_TYPE int
#define TIMEOUT_DIV 1000
#define TIMEOUT_OPTNAME TCP_RXT_CONNDROPTIME
#elif defined(TCP_USER_TIMEOUT)
#define TIMEOUT_TYPE unsigned int
#define TIMEOUT_DIV 1
#define TIMEOUT_OPTNAME TCP_USER_TIMEOUT
Add FreeBSD connection timeout socket option On FreeBSD, the option to configure connection timeout is called TCP_KEEPINIT, use it to configure the connection timeout there. This also fixes the dangling socket problems in the unit test, so re-enable them.
2020-12-02 21:54:25 +01:00
#elif defined(TCP_KEEPINIT)
#define TIMEOUT_TYPE int
#define TIMEOUT_DIV 1000
#define TIMEOUT_OPTNAME TCP_KEEPINIT
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
#endif
isc_result_t
isc__nm_socket_connectiontimeout(uv_os_sock_t fd, int timeout_ms) {
#if defined(TIMEOUT_OPTNAME)
TIMEOUT_TYPE timeout = timeout_ms / TIMEOUT_DIV;
if (timeout == 0) {
timeout = 1;
}
if (setsockopt(fd, IPPROTO_TCP, TIMEOUT_OPTNAME, &timeout,
sizeof(timeout)) == -1)
{
return (ISC_R_FAILURE);
}
return (ISC_R_SUCCESS);
#else
UNUSED(fd);
UNUSED(timeout_ms);
return (ISC_R_SUCCESS);
#endif
}
Disable Nagle's algorithm for HTTP/2 connections It is advisable to disable Nagle's algorithm for HTTP/2 connections because multiple HTTP/2 streams could be multiplexed over one transport connection. Thus, delays when delivering small packets could bring down performance for the whole session. HTTP/2 is meant to be used this way.
2021-02-28 19:33:16 +02:00
isc_result_t
isc__nm_socket_tcp_nodelay(uv_os_sock_t fd) {
#ifdef TCP_NODELAY
if (setsockopt_on(fd, IPPROTO_TCP, TCP_NODELAY) == -1) {
return (ISC_R_FAILURE);
} else {
return (ISC_R_SUCCESS);
}
#else
UNUSED(fd);
return (ISC_R_SUCCESS);
#endif
}
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
#ifdef NETMGR_TRACE
/*
* Dump all active sockets in netmgr. We output to stderr
* as the logger might be already shut down.
*/
static const char *
nmsocket_type_totext(isc_nmsocket_type type) {
switch (type) {
case isc_nm_udpsocket:
return ("isc_nm_udpsocket");
case isc_nm_udplistener:
return ("isc_nm_udplistener");
case isc_nm_tcpsocket:
return ("isc_nm_tcpsocket");
case isc_nm_tcplistener:
return ("isc_nm_tcplistener");
case isc_nm_tcpdnslistener:
return ("isc_nm_tcpdnslistener");
case isc_nm_tcpdnssocket:
return ("isc_nm_tcpdnssocket");
Resurrect old TLS code This commit resurrects the old TLS code from 8f73c70d23e26954165fd44ce5617a95f112bcff. It also includes numerous stability fixes and support for isc_nm_cancelread() for the TLS layer. The code was resurrected to be used for DoH.
2021-01-25 17:44:39 +02:00
case isc_nm_tlssocket:
return ("isc_nm_tlssocket");
case isc_nm_tlslistener:
return ("isc_nm_tlslistener");
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
case isc_nm_tlsdnslistener:
return ("isc_nm_tlsdnslistener");
case isc_nm_tlsdnssocket:
return ("isc_nm_tlsdnssocket");
Initial support for DNS-over-HTTP(S) This commit completes the support for DNS-over-HTTP(S) built on top of nghttp2 and plugs it into the BIND. Support for both GET and POST requests is present, as required by RFC8484. Both encrypted (via TLS) and unencrypted HTTP/2 connections are supported. The latter are mostly there for debugging/troubleshooting purposes and for the means of encryption offloading to third-party software (as might be desirable in some environments to simplify TLS certificates management).
2020-12-07 14:19:10 +02:00
case isc_nm_httplistener:
return ("isc_nm_httplistener");
refactor outgoing HTTP connection support - style, cleanup, and removal of unnecessary code. - combined isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint() into one function, renamed isc_http_endpoint(). - moved isc_nm_http_connect_send_request() into doh_test.c as a helper function; remove it from the public API. - renamed isc_http2 and isc_nm_http2 types and functions to just isc_http and isc_nm_http, for consistency with other existing names. - shortened a number of long names. - the caller is now responsible for determining the peer address. in isc_nm_httpconnect(); this eliminates the need to parse the URI and the dependency on an external resolver. - the caller is also now responsible for creating the SSL client context, for consistency with isc_nm_tlsdnsconnect(). - added setter functions for HTTP/2 ALPN. instead of setting up ALPN in isc_tlsctx_createclient(), we now have a function isc_tlsctx_enable_http2client_alpn() that can be run from isc_nm_httpconnect(). - refactored isc_nm_httprequest() into separate read and send functions. isc_nm_send() or isc_nm_read() is called on an http socket, it will be stored until a corresponding isc_nm_read() or _send() arrives; when we have both halves of the pair the HTTP request will be initiated. - isc_nm_httprequest() is renamed isc__nm_http_request() for use as an internal helper function by the DoH unit test. (eventually doh_test should be rewritten to use read and send, and this function should be removed.) - added implementations of isc__nm_tls_settimeout() and isc__nm_http_settimeout(). - increased NGHTTP2 header block length for client connections to 128K. - use isc_mem_t for internal memory allocations inside nghttp2, to help track memory leaks. - send "Cache-Control" header in requests and responses. (note: currently we try to bypass HTTP caching proxies, but ideally we should interact with them: https://tools.ietf.org/html/rfc8484#section-5.1)
2021-02-03 16:59:49 -08:00
case isc_nm_httpsocket:
return ("isc_nm_httpsocket");
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
default:
INSIST(0);
ISC_UNREACHABLE();
}
}
static void
nmhandle_dump(isc_nmhandle_t *handle) {
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
fprintf(stderr, "Active handle %p, refs %" PRIuFAST32 "\n", handle,
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
isc_refcount_current(&handle->references));
fprintf(stderr, "Created by:\n");
Add support for generating backtraces on Windows This commit adds support for generating backtraces on Windows and refactors the isc_backtrace API to match the Linux/BSD API (without the isc_ prefix) * isc_backtrace_gettrace() was renamed to isc_backtrace(), the third argument was removed and the return type was changed to int * isc_backtrace_symbols() was added * isc_backtrace_symbols_fd() was added and used as appropriate
2021-04-27 16:20:03 +02:00
isc_backtrace_symbols_fd(handle->backtrace, handle->backtrace_size,
STDERR_FILENO);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
fprintf(stderr, "\n\n");
}
static void
nmsocket_dump(isc_nmsocket_t *sock) {
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc_nmhandle_t *handle = NULL;
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
LOCK(&sock->lock);
fprintf(stderr, "\n=================\n");
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
fprintf(stderr, "Active %s socket %p, type %s, refs %" PRIuFAST32 "\n",
atomic_load(&sock->client) ? "client" : "server", sock,
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
nmsocket_type_totext(sock->type),
isc_refcount_current(&sock->references));
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
fprintf(stderr,
"Parent %p, listener %p, server %p, statichandle = %p\n",
sock->parent, sock->listener, sock->server, sock->statichandle);
Fix compilation with NETMGR_TRACE(_VERBOSE) enabled on non-Linux When NETMGR_TRACE(_VERBOSE) is enabled, the build would fail on some non-Linux non-glibc platforms because: * Use <stdint.h> print macros because uint_fast32_t is not always unsigned long * The header <execinfo.h> is not available on non-glibc, thus commit adds dummy backtrace() and backtrace_symbols_fd() functions for platforms without HAVE_BACKTRACE
2021-01-29 13:00:46 +01:00
fprintf(stderr, "Flags:%s%s%s%s%s\n",
atomic_load(&sock->active) ? " active" : "",
atomic_load(&sock->closing) ? " closing" : "",
atomic_load(&sock->destroying) ? " destroying" : "",
atomic_load(&sock->connecting) ? " connecting" : "",
Refactor netmgr and add more unit tests This is a part of the works that intends to make the netmgr stable, testable, maintainable and tested. It contains a numerous changes to the netmgr code and unfortunately, it was not possible to split this into smaller chunks as the work here needs to be committed as a complete works. NOTE: There's a quite a lot of duplicated code between udp.c, tcp.c and tcpdns.c and it should be a subject to refactoring in the future. The changes that are included in this commit are listed here (extensively, but not exclusively): * The netmgr_test unit test was split into individual tests (udp_test, tcp_test, tcpdns_test and newly added tcp_quota_test) * The udp_test and tcp_test has been extended to allow programatic failures from the libuv API. Unfortunately, we can't use cmocka mock() and will_return(), so we emulate the behaviour with #define and including the netmgr/{udp,tcp}.c source file directly. * The netievents that we put on the nm queue have variable number of members, out of these the isc_nmsocket_t and isc_nmhandle_t always needs to be attached before enqueueing the netievent_<foo> and detached after we have called the isc_nm_async_<foo> to ensure that the socket (handle) doesn't disappear between scheduling the event and actually executing the event. * Cancelling the in-flight TCP connection using libuv requires to call uv_close() on the original uv_tcp_t handle which just breaks too many assumptions we have in the netmgr code. Instead of using uv_timer for TCP connection timeouts, we use platform specific socket option. * Fix the synchronization between {nm,async}_{listentcp,tcpconnect} When isc_nm_listentcp() or isc_nm_tcpconnect() is called it was waiting for socket to either end up with error (that path was fine) or to be listening or connected using condition variable and mutex. Several things could happen: 0. everything is ok 1. the waiting thread would miss the SIGNAL() - because the enqueued event would be processed faster than we could start WAIT()ing. In case the operation would end up with error, it would be ok, as the error variable would be unchanged. 2. the waiting thread miss the sock->{connected,listening} = `true` would be set to `false` in the tcp_{listen,connect}close_cb() as the connection would be so short lived that the socket would be closed before we could even start WAIT()ing * The tcpdns has been converted to using libuv directly. Previously, the tcpdns protocol used tcp protocol from netmgr, this proved to be very complicated to understand, fix and make changes to. The new tcpdns protocol is modeled in a similar way how tcp netmgr protocol. Closes: #2194, #2283, #2318, #2266, #2034, #1920 * The tcp and tcpdns is now not using isc_uv_import/isc_uv_export to pass accepted TCP sockets between netthreads, but instead (similar to UDP) uses per netthread uv_loop listener. This greatly reduces the complexity as the socket is always run in the associated nm and uv loops, and we are also not touching the libuv internals. There's an unfortunate side effect though, the new code requires support for load-balanced sockets from the operating system for both UDP and TCP (see #2137). If the operating system doesn't support the load balanced sockets (either SO_REUSEPORT on Linux or SO_REUSEPORT_LB on FreeBSD 12+), the number of netthreads is limited to 1. * The netmgr has now two debugging #ifdefs: 1. Already existing NETMGR_TRACE prints any dangling nmsockets and nmhandles before triggering assertion failure. This options would reduce performance when enabled, but in theory, it could be enabled on low-performance systems. 2. New NETMGR_TRACE_VERBOSE option has been added that enables extensive netmgr logging that allows the software engineer to precisely track any attach/detach operations on the nmsockets and nmhandles. This is not suitable for any kind of production machine, only for debugging. * The tlsdns netmgr protocol has been split from the tcpdns and it still uses the old method of stacking the netmgr boxes on top of each other. We will have to refactor the tlsdns netmgr protocol to use the same approach - build the stack using only libuv and openssl. * Limit but not assert the tcp buffer size in tcp_alloc_cb Closes: #2061
2020-11-12 10:32:18 +01:00
sock->accepting ? " accepting" : "");
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
fprintf(stderr, "Created by:\n");
Add support for generating backtraces on Windows This commit adds support for generating backtraces on Windows and refactors the isc_backtrace API to match the Linux/BSD API (without the isc_ prefix) * isc_backtrace_gettrace() was renamed to isc_backtrace(), the third argument was removed and the return type was changed to int * isc_backtrace_symbols() was added * isc_backtrace_symbols_fd() was added and used as appropriate
2021-04-27 16:20:03 +02:00
isc_backtrace_symbols_fd(sock->backtrace, sock->backtrace_size,
STDERR_FILENO);
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
fprintf(stderr, "\n");
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
for (handle = ISC_LIST_HEAD(sock->active_handles); handle != NULL;
handle = ISC_LIST_NEXT(handle, active_link))
{
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
static bool first = true;
if (first) {
fprintf(stderr, "Active handles:\n");
first = false;
}
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
nmhandle_dump(handle);
}
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
fprintf(stderr, "\n");
UNLOCK(&sock->lock);
}
void
isc__nm_dump_active(isc_nm_t *nm) {
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
isc_nmsocket_t *sock = NULL;
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
REQUIRE(VALID_NM(nm));
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
LOCK(&nm->lock);
for (sock = ISC_LIST_HEAD(nm->active_sockets); sock != NULL;
sock = ISC_LIST_NEXT(sock, active_link))
{
Fix the isc_nm_closedown() to actually close the pending connections 1. The isc__nm_tcp_send() and isc__nm_tcp_read() was not checking whether the socket was still alive and scheduling reads/sends on closed socket. 2. The isc_nm_read(), isc_nm_send() and isc_nm_resumeread() have been changed to always return the error conditions via the callbacks, so they always succeed. This applies to all protocols (UDP, TCP and TCPDNS).
2020-10-21 12:52:09 +02:00
static bool first = true;
if (first) {
fprintf(stderr, "Outstanding sockets\n");
first = false;
}
tracing of active sockets and handles If NETMGR_TRACE is defined, we now maintain a list of active sockets in the netmgr object and a list of active handles in each socket object; by walking the list and printing `backtrace` in a debugger we can see where they were created, to assist in in debugging of reference counting errors. On shutdown, if netmgr finds there are still active sockets after waiting, isc__nm_dump_active() will be called to log the list of active sockets and their underlying handles, along with some details about them.
2020-09-02 17:57:44 +02:00
nmsocket_dump(sock);
}
UNLOCK(&nm->lock);
}
#endif
Reference in New Issue Copy Permalink