mir/bind
2
0
Fork 0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-22 18:19:42 +00:00
Code Issues Releases Wiki Activity
bind/lib/dns/dispatch.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2317 lines
56 KiB
C
Raw Normal View History

checkpoint
1999-06-16 01:32:31 +00:00
/*
update copyright notice
2011-02-03 12:18:12 +00:00
* Copyright (C) Internet Systems Consortium, Inc. ("ISC")
checkpoint
1999-06-16 01:32:31 +00:00
*
* SPDX-License-Identifier: MPL-2.0
Update the copyright information in all files in the repository This commit converts the license handling to adhere to the REUSE specification. It specifically: 1. Adds used licnses to LICENSES/ directory 2. Add "isc" template for adding the copyright boilerplate 3. Changes all source files to include copyright and SPDX license header, this includes all the C sources, documentation, zone files, configuration files. There are notes in the doc/dev/copyrights file on how to add correct headers to the new files. 4. Handle the rest that can't be modified via .reuse/dep5 file. The binary (or otherwise unmodifiable) files could have license places next to them in <foo>.license file, but this would lead to cluttered repository and most of the files handled in the .reuse/dep5 file are system test files.
2021-06-03 08:37:05 +02:00
*
checkpoint
1999-06-16 01:32:31 +00:00
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, you can obtain one at https://mozilla.org/MPL/2.0/.
Update license headers to not include years in copyright in all applicable files
2018-02-23 09:53:12 +01:00
*
checkpoint
1999-06-16 01:32:31 +00:00
* See the COPYRIGHT file distributed with this work for additional
checkpoint
1999-07-08 22:12:37 +00:00
* information regarding copyright ownership.
checkpoint
1999-06-16 01:32:31 +00:00
*/
1851. [doc] Doxygen comment markup. [RT #11398]
2005-04-27 04:57:32 +00:00
/*! \file */
add RCS id string
2000-06-22 22:00:42 +00:00
Refactor the network manager to be a singleton There is only a single network manager running on top of the loop manager (except for tests). Refactor the network manager to be a singleton (a single instance) and change the unit tests, so that the shorter read timeouts apply only to a specific handle, not the whole extra 'connect_nm' network manager instance.
2025-07-14 17:12:35 +02:00
/*
* FIXME: Might need dns_dispatch_shuttingdown()
*/
Replace custom isc_u?intNN_t types with C99 u?intNN_t types
2018-03-28 14:19:37 +02:00
#include <inttypes.h>
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
#include <stdbool.h>
checkpoint
1999-06-16 01:32:31 +00:00
#include <stdlib.h>
Use the new sorting rules to regroup #include headers
2020-03-09 16:17:26 +01:00
#include <sys/types.h>
2203. [security] Query id generation was cryptographically weak. [RT # 16915]
2007-06-26 02:52:15 +00:00
#include <unistd.h>
checkpoint
1999-06-16 01:32:31 +00:00
Always call the TCP dispatch connected callbacks asynchronously The TCP dispatch connected callbacks could be called synchronously which in turn could destroy xfrin before we return from dns_xfrin_create(). Delay the calling the callback called from tcp_dispatch_connect() by calling it always asynchronously.
2024-03-04 12:58:56 +01:00
#include <isc/async.h>
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
#include <isc/hash.h>
#include <isc/hashmap.h>
Convert all categories and modules into static lists Remove the complicated mechanism that could be (in theory) used by external libraries to register new categories and modules with statically defined lists in <isc/log.h>. This is similar to what we have done for <isc/result.h> result codes. All the libraries are now internal to BIND 9, so we don't need to provide a mechanism to register extra categories and modules.
2024-08-14 13:25:50 +02:00
#include <isc/log.h>
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
#include <isc/loop.h>
checkpoint
1999-06-18 02:01:42 +00:00
#include <isc/mem.h>
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
#include <isc/mutex.h>
Use system ephemeral ports for default portset In dispatch, use system ephemeral ports for default portset instead of hardcoded <1024,65535> range.
2021-10-04 14:11:57 +02:00
#include <isc/net.h>
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
#include <isc/netmgr.h>
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
#include <isc/portset.h>
#include <isc/random.h>
2537. [func] Added more statistics counters including those on socket I/O events and query RTT histograms. [RT #18802]
2009-01-27 22:30:00 +00:00
#include <isc/stats.h>
needs string.h for memset() prototype.
2000-05-13 21:57:02 +00:00
#include <isc/string.h>
Pin the dns_dispatch to threads when reusing Previously, dns_dispatch_gettcp() could pick a TCP connection created by different thread - this breaks our contractual promise to DNS dispatch by using the TCP connection on a different thread than it was created. Add .tid member to the dns_dispatch_t struct and skip the dispatches from other threads when looking up a TCP dispatch that we can reuse in dns_request. NOTE: This is going to be properly refactored, but this change could be also backported to 9.18 for better stability and thread-affinity.
2023-01-04 19:21:14 +01:00
#include <isc/tid.h>
replace gettimeofday with isc_time_now
2007-06-26 06:02:37 +00:00
#include <isc/time.h>
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
#include <isc/tls.h>
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
#include <isc/urcu.h>
move util.h to <isc/util.h>
1999-12-16 22:24:22 +00:00
#include <isc/util.h>
checkpoint
1999-06-16 01:32:31 +00:00
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
#include <dns/acl.h>
checkpoint
1999-06-16 01:32:31 +00:00
#include <dns/dispatch.h>
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
#include <dns/message.h>
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
#include <dns/stats.h>
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
#include <dns/transport.h>
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
#include <dns/types.h>
469. [bug] "query-source address * port 53;" now works.
2000-09-18 04:50:05 +00:00
typedef ISC_LIST(dns_dispentry_t) dns_displist_t;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
struct dns_dispatchmgr {
/* Unlocked. */
unsigned int magic;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_refcount_t references;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
isc_mem_t *mctx;
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
dns_acl_t *blackhole;
2537. [func] Added more statistics counters including those on socket I/O events and query RTT histograms. [RT #18802]
2009-01-27 22:30:00 +00:00
isc_stats_t *stats;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
uint32_t nloops;
struct cds_lfht **tcps;
Move the UDP buffer pool to the dispatchmgr. UDP parameters are now set by the first call to dns_dispatch_getudp(), subsecquent calls can raise the maximum number of buffers. dns_dispatchmgr_setudp() could be made public. Tidy dispatch_allocate(). entropy and mctx are now gone from dns_qid_t.
2000-09-19 06:59:28 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
struct cds_lfht *qids;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
in_port_t *v4ports; /*%< available ports for IPv4 */
unsigned int nv4ports; /*%< # of available ports for IPv4 */
in_port_t *v6ports; /*%< available ports for IPv4 */
unsigned int nv6ports; /*%< # of available ports for IPv4 */
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
};
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
typedef enum {
DNS_DISPATCHSTATE_NONE = 0UL,
DNS_DISPATCHSTATE_CONNECTING,
DNS_DISPATCHSTATE_CONNECTED,
DNS_DISPATCHSTATE_CANCELED,
} dns_dispatchstate_t;
checkpoint
1999-07-06 19:32:40 +00:00
struct dns_dispentry {
checkpoint
1999-06-18 02:01:42 +00:00
unsigned int magic;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_refcount_t references;
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
isc_mem_t *mctx;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
dns_dispatch_t *disp;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
isc_loop_t *loop;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_nmhandle_t *handle; /*%< netmgr handle for UDP connection */
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatchstate_t state;
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
dns_transport_t *transport;
isc_tlsctx_cache_t *tlsctx_cache;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
unsigned int retries;
Separate the connect and the read timeouts in dispatch The network manager layer has two different timers with their own timeout values for TCP connections: connect timeout and read timeout. Separate the connect and the read TCP timeouts in the dispatch module too.
2024-10-30 09:23:33 +00:00
unsigned int connect_timeout;
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
unsigned int timeout;
isc_time_t start;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_sockaddr_t local;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_sockaddr_t peer;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
in_port_t port;
dns_messageid_t id;
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
dispatch_cb_t connected;
dispatch_cb_t sent;
dispatch_cb_t response;
checkpoint
1999-06-16 01:32:31 +00:00
void *arg;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
bool reading;
isc_result_t result;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
ISC_LINK(dns_dispentry_t) alink;
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
ISC_LINK(dns_dispentry_t) plink;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
ISC_LINK(dns_dispentry_t) rlink;
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
struct cds_lfht_node ht_node;
struct rcu_head rcu_head;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
};
checkpoint
1999-06-16 01:32:31 +00:00
struct dns_dispatch {
/* Unlocked. */
Pin the dns_dispatch to threads when reusing Previously, dns_dispatch_gettcp() could pick a TCP connection created by different thread - this breaks our contractual promise to DNS dispatch by using the TCP connection on a different thread than it was created. Add .tid member to the dns_dispatch_t struct and skip the dispatches from other threads when looking up a TCP dispatch that we can reuse in dns_request. NOTE: This is going to be properly refactored, but this change could be also backported to 9.18 for better stability and thread-affinity.
2023-01-04 19:21:14 +01:00
unsigned int magic; /*%< magic */
Convert the isc/tid.h to use own signed integer isc_tid_t type Change the internal type used for isc_tid unit to isc_tid_t to hide the specific integer type being used for the 'tid'. Internally, the signed integer type is being used. This allows us to have negatively indexed arrays that works both for threads with assigned tid and the threads with unassigned tid. This should be used only in specific situations.
2025-06-04 17:54:20 +02:00
isc_tid_t tid;
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
isc_socktype_t socktype;
isc_refcount_t references;
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
isc_mem_t *mctx;
Fix TCP dispatches and transport Dispatch needs to know the transport that is being used over the TCP connection to correctly allow for it to be reused. Add a transport parameter to dns_dispatch_createtcp and dns_dispatch_gettcp and use it when selecting a TCP socket for reuse.
2024-10-15 16:09:48 +11:00
dns_dispatchmgr_t *mgr; /*%< dispatch manager */
isc_nmhandle_t *handle; /*%< netmgr handle for TCP connection */
isc_sockaddr_t local; /*%< local address */
isc_sockaddr_t peer; /*%< peer address (TCP) */
dns_transport_t *transport; /*%< TCP transport parameters */
Improve locking performance in dispatch.c 3316. [tuning] Improved locking performance when recursing. [RT #28836] - Use one lock per memory pool instead of associating them all with a single 'pool_lock' in the dispatch manager. - Reduce the critical sections for qid->lock in get_dispsocket(), deref_portentry() and dns_dispatch_addresponse2(). - Added a memory pool for socket events in dns_dispatch_t. - Add an isc_socketevent_t member in the resquery_t object, and use it with isc_socket_sendto2() instead of using isc_socket_sendto() - Tuned the memory pools in dispatch.c for better performance under load
2012-04-28 14:52:28 -07:00
Add option to mark TCP dispatch as unshared The current dispatch code could reuse the TCP connection when dns_dispatch_gettcp() would be used first. This is problematic as the dns_resolver doesn't use TCP connection sharing, but dns_request could get the TCP stream that was created outside of the dns_request. Add new DNS_DISPATCHOPT_UNSHARED option to dns_dispatch_createtcp() that would prevent the TCP stream to be reused. Use that option in the dns_resolver call to dns_dispatch_createtcp() to prevent dns_request from reusing the TCP connections created by dns_resolver. Additionally, the dns_xfrin unit added TCP connection sharing for incoming transfers. While interleaving *xfr streams on a TCP connection should work this should be a deliberate change and be property of the server that can be controlled. Additionally some level of parallel TCP streams is desirable. Revert to the old behaviour by removing the dns_dispatch_gettcp() calls from dns_xfrin and use the new option to prevent from sharing the transfer streams with dns_request.
2023-10-20 08:14:27 +02:00
dns_dispatchopt_t options;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatchstate_t state;
bool reading;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
dns_displist_t pending;
dns_displist_t active;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
uint_fast32_t requests; /*%< how many requests we have */
Tear down the TCP connection on too many unexpected DNS messages When the outgoing TCP dispatch times-out active response, we might still receive the answer during the lifetime of the connection. Previously, we would just ignore any non-matching DNS answers, which would allow the server to feed us with otherwise valid DNS answer and keep the connection open. Add a counter for timed-out DNS queries over TCP and tear down the whole TCP connection if we receive unexpected number of DNS answers.
2021-11-29 09:59:33 +01:00
unsigned int timedout;
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
struct cds_lfht_node ht_node;
struct rcu_head rcu_head;
checkpoint
1999-06-16 01:32:31 +00:00
};
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
#define RESPONSE_MAGIC ISC_MAGIC('D', 'r', 's', 'p')
#define VALID_RESPONSE(e) ISC_MAGIC_VALID((e), RESPONSE_MAGIC)
checkpoint
1999-06-18 02:01:42 +00:00
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
#define DISPATCH_MAGIC ISC_MAGIC('D', 'i', 's', 'p')
#define VALID_DISPATCH(e) ISC_MAGIC_VALID((e), DISPATCH_MAGIC)
checkpoint
1999-06-18 02:01:42 +00:00
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
#define DNS_DISPATCHMGR_MAGIC ISC_MAGIC('D', 'M', 'g', 'r')
#define VALID_DISPATCHMGR(e) ISC_MAGIC_VALID((e), DNS_DISPATCHMGR_MAGIC)
checkpoint
1999-06-18 02:01:42 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
#define dns_dispentry_ref(ptr) \
dns_dispentry__ref(ptr, __func__, __FILE__, __LINE__)
#define dns_dispentry_unref(ptr) \
dns_dispentry__unref(ptr, __func__, __FILE__, __LINE__)
#define dns_dispentry_attach(ptr, ptrp) \
dns_dispentry__attach(ptr, ptrp, __func__, __FILE__, __LINE__)
#define dns_dispentry_detach(ptrp) \
dns_dispentry__detach(ptrp, __func__, __FILE__, __LINE__)
ISC_REFCOUNT_TRACE_DECL(dns_dispentry);
#else
ISC_REFCOUNT_DECL(dns_dispentry);
#endif
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
/*
* The number of attempts to find unique <addr, port, query_id> combination
*/
#define QID_MAX_TRIES 64
/*
* Initial and minimum QID table sizes.
*/
#define QIDS_INIT_SIZE (1 << 4) /* Must be power of 2 */
#define QIDS_MIN_SIZE (1 << 4) /* Must be power of 2 */
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
/*
needs string.h for memset() prototype.
2000-05-13 21:57:02 +00:00
* Statics.
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
*/
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
static void
dispatchmgr_destroy(dns_dispatchmgr_t *mgr);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
static void
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
udp_recv(isc_nmhandle_t *handle, isc_result_t eresult, isc_region_t *region,
void *arg);
Use an LFSR to generate unpredictable IDs.
1999-12-15 03:11:00 +00:00
static void
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
tcp_recv(isc_nmhandle_t *handle, isc_result_t eresult, isc_region_t *region,
void *arg);
1687. [bug] Race condition in dispatch. [RT #10272]
2004-07-21 00:48:19 +00:00
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_cancel(dns_dispentry_t *resp, isc_result_t result);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
static isc_result_t
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
dispatch_createudp(dns_dispatchmgr_t *mgr, const isc_sockaddr_t *localaddr,
Convert the isc/tid.h to use own signed integer isc_tid_t type Change the internal type used for isc_tid unit to isc_tid_t to hide the specific integer type being used for the 'tid'. Internally, the signed integer type is being used. This allows us to have negatively indexed arrays that works both for threads with assigned tid and the threads with unassigned tid. This should be used only in specific situations.
2025-06-04 17:54:20 +02:00
isc_tid_t tid, dns_dispatch_t **dispp);
Dispatch API simplification - Many dispatch attributes can be set implicitly instead of being passed in. we can infer whether to set DNS_DISPATCHATTR_TCP or _UDP from whether we're calling dns_dispatch_createtcp() or _createudp(). we can also infer DNS_DISPATCHATTR_IPV4 or _IPV6 from the addresses or the socket that were passed in. - We no longer use dup'd sockets in UDP dispatches, so the 'dup_socket' parameter has been removed from dns_dispatch_createudp(), along with the code implementing it. also removed isc_socket_dup() since it no longer has any callers. - The 'buffersize' parameter was ignored and has now been removed; buffersize is now fixed at 4096. - Maxbuffers and maxrequests don't need to be passed in on every call to dns_dispatch_createtcp() and _createudp(). In all current uses, the value for mgr->maxbuffers will either be raised once from its default of 20000 to 32768, or else left alone. (passing in a value lower than 20000 does not lower it.) there isn't enough difference between these values for there to be any need to configure this. The value for disp->maxrequests controls both the quota of concurrent requests for a dispatch and also the size of the dispatch socket memory pool. it's not clear that this quota is necessary at all. the memory pool size currently starts at 32768, but is sometimes lowered to 4096, which is definitely unnecessary. This commit sets both values permanently to 32768. - Previously TCP dispatches allocated their own separate QID table, which didn't incorporate a port table. this commit removes per-dispatch QID tables and shares the same table between all dispatches. since dispatches are created for each TCP socket, this may speed up the dispatch allocation process. there may be a slight increase in lock contention since all dispatches are sharing a single QID table, but since TCP sockets are used less often than UDP sockets (which were already sharing a QID table), it should not be a substantial change. - The dispatch port table was being used to determine whether a port was already in use; if so, then a UDP socket would be bound with REUSEADDR. this commit removes the port table, and always binds UDP sockets that way.
2020-12-17 00:43:00 -08:00
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
udp_startrecv(isc_nmhandle_t *handle, dns_dispentry_t *resp);
static void
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
udp_dispatch_connect(dns_dispatch_t *disp, dns_dispentry_t *resp);
static void
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
tcp_startrecv(dns_dispatch_t *disp, dns_dispentry_t *resp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
tcp_dispatch_getnext(dns_dispatch_t *disp, dns_dispentry_t *resp,
Remove dispatch timeout INT16_MAX limitation In some places there was a limitation of the maximum timeout value of INT16_MAX, which is only about 32 seconds. Refactor the code to remove the limitation.
2024-10-30 13:31:34 +00:00
int64_t timeout);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
Remove dispatch timeout INT16_MAX limitation In some places there was a limitation of the maximum timeout value of INT16_MAX, which is only about 32 seconds. Refactor the code to remove the limitation.
2024-10-30 13:31:34 +00:00
udp_dispatch_getnext(dns_dispentry_t *resp, int64_t timeout);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static const char *
socktype2str(dns_dispentry_t *resp) {
dns_transport_type_t transport_type = DNS_TRANSPORT_UDP;
dns_dispatch_t *disp = resp->disp;
if (disp->socktype == isc_socktype_tcp) {
if (resp->transport != NULL) {
transport_type =
dns_transport_get_type(resp->transport);
} else {
transport_type = DNS_TRANSPORT_TCP;
}
}
switch (transport_type) {
case DNS_TRANSPORT_UDP:
return "UDP";
case DNS_TRANSPORT_TCP:
return "TCP";
case DNS_TRANSPORT_TLS:
return "TLS";
case DNS_TRANSPORT_HTTP:
return "HTTP";
default:
return "<unexpected>";
}
}
static const char *
state2str(dns_dispatchstate_t state) {
switch (state) {
case DNS_DISPATCHSTATE_NONE:
return "none";
case DNS_DISPATCHSTATE_CONNECTING:
return "connecting";
case DNS_DISPATCHSTATE_CONNECTED:
return "connected";
case DNS_DISPATCHSTATE_CANCELED:
return "canceled";
default:
return "<unexpected>";
}
}
format string bugs and improved format string checking [RT #1578]
2001-08-08 22:54:55 +00:00
static void
mgr_log(dns_dispatchmgr_t *mgr, int level, const char *fmt, ...)
ISC_FORMAT_PRINTF(3, 4);
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
static void
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
mgr_log(dns_dispatchmgr_t *mgr, int level, const char *fmt, ...) {
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
char msgbuf[2048];
va_list ap;
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
if (!isc_log_wouldlog(level)) {
Added calls to isc_log_wouldlog() in additional logging functions.
2000-07-17 23:25:35 +00:00
return;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Added calls to isc_log_wouldlog() in additional logging functions.
2000-07-17 23:25:35 +00:00
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
va_start(ap, fmt);
vsnprintf(msgbuf, sizeof(msgbuf), fmt, ap);
va_end(ap);
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
isc_log_write(DNS_LOGCATEGORY_DISPATCH, DNS_LOGMODULE_DISPATCH, level,
"dispatchmgr %p: %s", mgr, msgbuf);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
isc_statscounter_t
2009-01-31 00:37:04 +00:00
inc_stats(dns_dispatchmgr_t *mgr, isc_statscounter_t counter) {
2541. [bug] Conditionally update dispatch manager statistics. [RT #19247]
2009-01-31 00:10:24 +00:00
if (mgr->stats != NULL) {
isc_stats_increment(mgr->stats, counter);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
2541. [bug] Conditionally update dispatch manager statistics. [RT #19247]
2009-01-31 00:10:24 +00:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
dec_stats(dns_dispatchmgr_t *mgr, isc_statscounter_t counter) {
if (mgr->stats != NULL) {
isc_stats_decrement(mgr->stats, counter);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
}
format string bugs and improved format string checking [RT #1578]
2001-08-08 22:54:55 +00:00
static void
dispatch_log(dns_dispatch_t *disp, int level, const char *fmt, ...)
ISC_FORMAT_PRINTF(3, 4);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
static void
dispatch_log(dns_dispatch_t *disp, int level, const char *fmt, ...) {
char msgbuf[2048];
va_list ap;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
int r;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
if (!isc_log_wouldlog(level)) {
use isc_log_wouldlog()
2000-07-13 01:16:22 +00:00
return;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Trailing whitespace trimmed. Perhaps running "perl util/spacewhack.pl in your own CVS tree will help minimize CVS conflicts. Maybe not. Blame Graff for getting me to trim all trailing whitespace.
2000-08-01 01:33:37 +00:00
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
va_start(ap, fmt);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
r = vsnprintf(msgbuf, sizeof(msgbuf), fmt, ap);
if (r < 0) {
msgbuf[0] = '\0';
} else if ((unsigned int)r >= sizeof(msgbuf)) {
/* Truncated */
msgbuf[sizeof(msgbuf) - 1] = '\0';
}
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
va_end(ap);
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
isc_log_write(DNS_LOGCATEGORY_DISPATCH, DNS_LOGMODULE_DISPATCH, level,
"dispatch %p: %s", disp, msgbuf);
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
dispentry_log(dns_dispentry_t *resp, int level, const char *fmt, ...)
ISC_FORMAT_PRINTF(3, 4);
static void
dispentry_log(dns_dispentry_t *resp, int level, const char *fmt, ...) {
char msgbuf[2048];
va_list ap;
int r;
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
if (!isc_log_wouldlog(level)) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
return;
}
va_start(ap, fmt);
r = vsnprintf(msgbuf, sizeof(msgbuf), fmt, ap);
if (r < 0) {
msgbuf[0] = '\0';
} else if ((unsigned int)r >= sizeof(msgbuf)) {
/* Truncated */
msgbuf[sizeof(msgbuf) - 1] = '\0';
}
va_end(ap);
dispatch_log(resp->disp, level, "%s response %p: %s",
socktype2str(resp), resp, msgbuf);
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
/*%
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
* Choose a random port number for a dispatch entry.
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
*/
static isc_result_t
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
setup_socket(dns_dispatch_t *disp, dns_dispentry_t *resp,
const isc_sockaddr_t *dest, in_port_t *portp) {
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
dns_dispatchmgr_t *mgr = disp->mgr;
unsigned int nports;
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
in_port_t *ports = NULL;
use configured source ports for UDP requests the optional 'port' option, when used with notify-source, transfer-source, etc, is used to set up UDP dispatches with a particular source port, but when the actual UDP connection was established the port would be overridden with a random one. this has been fixed. (configuring source ports is deprecated in 9.20 and slated for removal in 9.22, but should still work correctly until then.)
2023-02-02 12:16:49 -08:00
in_port_t port = *portp;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
if (resp->retries++ > 5) {
return ISC_R_FAILURE;
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
if (isc_sockaddr_pf(&disp->local) == AF_INET) {
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
nports = mgr->nv4ports;
ports = mgr->v4ports;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
} else {
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
nports = mgr->nv6ports;
ports = mgr->v6ports;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
}
if (nports == 0) {
return ISC_R_ADDRNOTAVAIL;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
resp->local = disp->local;
resp->peer = *dest;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
use configured source ports for UDP requests the optional 'port' option, when used with notify-source, transfer-source, etc, is used to set up UDP dispatches with a particular source port, but when the actual UDP connection was established the port would be overridden with a random one. this has been fixed. (configuring source ports is deprecated in 9.20 and slated for removal in 9.22, but should still work correctly until then.)
2023-02-02 12:16:49 -08:00
if (port == 0) {
port = ports[isc_random_uniform(nports)];
isc_sockaddr_setport(&resp->local, port);
*portp = port;
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
resp->port = port;
Dispatch API simplification - Many dispatch attributes can be set implicitly instead of being passed in. we can infer whether to set DNS_DISPATCHATTR_TCP or _UDP from whether we're calling dns_dispatch_createtcp() or _createudp(). we can also infer DNS_DISPATCHATTR_IPV4 or _IPV6 from the addresses or the socket that were passed in. - We no longer use dup'd sockets in UDP dispatches, so the 'dup_socket' parameter has been removed from dns_dispatch_createudp(), along with the code implementing it. also removed isc_socket_dup() since it no longer has any callers. - The 'buffersize' parameter was ignored and has now been removed; buffersize is now fixed at 4096. - Maxbuffers and maxrequests don't need to be passed in on every call to dns_dispatch_createtcp() and _createudp(). In all current uses, the value for mgr->maxbuffers will either be raised once from its default of 20000 to 32768, or else left alone. (passing in a value lower than 20000 does not lower it.) there isn't enough difference between these values for there to be any need to configure this. The value for disp->maxrequests controls both the quota of concurrent requests for a dispatch and also the size of the dispatch socket memory pool. it's not clear that this quota is necessary at all. the memory pool size currently starts at 32768, but is sometimes lowered to 4096, which is definitely unnecessary. This commit sets both values permanently to 32768. - Previously TCP dispatches allocated their own separate QID table, which didn't incorporate a port table. this commit removes per-dispatch QID tables and shares the same table between all dispatches. since dispatches are created for each TCP socket, this may speed up the dispatch allocation process. there may be a slight increase in lock contention since all dispatches are sharing a single QID table, but since TCP sockets are used less often than UDP sockets (which were already sharing a QID table), it should not be a substantial change. - The dispatch port table was being used to determine whether a port was already in use; if so, then a UDP socket would be bound with REUSEADDR. this commit removes the port table, and always binds UDP sockets that way.
2020-12-17 00:43:00 -08:00
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
return ISC_R_SUCCESS;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
static uint32_t
qid_hash(const dns_dispentry_t *dispentry) {
Use the new isc_sockaddr_hash_ex() to fix QID table hashing The QID table hashing used a custom merging of the sockaddr, port and id into a single hashvalue. Normalize the QID table hashing function to use isc_hash32 API for all the values.
2023-09-18 10:02:37 +02:00
isc_hash32_t hash;
isc_hash32_init(&hash);
isc_sockaddr_hash_ex(&hash, &dispentry->peer, true);
isc_hash32_hash(&hash, &dispentry->id, sizeof(dispentry->id), true);
isc_hash32_hash(&hash, &dispentry->port, sizeof(dispentry->port), true);
return isc_hash32_finalize(&hash);
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
}
add another REQUIRE() where it might help
1999-07-09 20:34:26 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
static int
qid_match(struct cds_lfht_node *node, const void *key0) {
const dns_dispentry_t *dispentry =
caa_container_of(node, dns_dispentry_t, ht_node);
const dns_dispentry_t *key = key0;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
return dispentry->id == key->id && dispentry->port == key->port &&
isc_sockaddr_equal(&dispentry->peer, &key->peer);
}
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
static void
dispentry_destroy_rcu(struct rcu_head *rcu_head) {
dns_dispentry_t *resp = caa_container_of(rcu_head, dns_dispentry_t,
rcu_head);
isc_mem_putanddetach(&resp->mctx, resp, sizeof(*resp));
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_destroy(dns_dispentry_t *resp) {
dns_dispatch_t *disp = resp->disp;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/*
* We need to call this from here in case there's an external event that
* shuts down our dispatch (like ISC_R_SHUTTINGDOWN).
*/
dispentry_cancel(resp, ISC_R_CANCELED);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(disp->requests > 0);
disp->requests--;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
resp->magic = 0;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(!ISC_LINK_LINKED(resp, plink));
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
INSIST(!ISC_LINK_LINKED(resp, alink));
INSIST(!ISC_LINK_LINKED(resp, rlink));
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "destroying");
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (resp->handle != NULL) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
"detaching handle %p from %p", resp->handle,
&resp->handle);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_nmhandle_detach(&resp->handle);
}
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
if (resp->tlsctx_cache != NULL) {
isc_tlsctx_cache_detach(&resp->tlsctx_cache);
}
if (resp->transport != NULL) {
dns_transport_detach(&resp->transport);
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_detach(&disp); /* DISPATCH001 */
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
call_rcu(&resp->rcu_head, dispentry_destroy_rcu);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
ISC_REFCOUNT_TRACE_IMPL(dns_dispentry, dispentry_destroy);
#else
ISC_REFCOUNT_IMPL(dns_dispentry, dispentry_destroy);
#endif
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
/*
* How long in milliseconds has it been since this dispentry
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
* started reading?
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
*/
static unsigned int
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
dispentry_runtime(dns_dispentry_t *resp, const isc_time_t *now) {
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
if (isc_time_isepoch(&resp->start)) {
return 0;
}
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
return isc_time_microdiff(now, &resp->start) / 1000;
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
}
checkpoint
1999-07-08 02:50:00 +00:00
/*
* General flow:
*
don't retry UDP receives after they return an error - an error return from the socket library means the library has already decided that the error is not retryable and likely to throw us into an infinite loop if retried
2001-05-14 23:10:19 +00:00
* If I/O result == CANCELED or error, free the buffer.
checkpoint
1999-07-08 02:50:00 +00:00
*
fixed a bug that named crashes with an assertion failure on exit when sharing the same port for listening and querying, and changing listening addresses several times. [RT# 3509] additionally, + limited the canceled socket tasks in dispatch.c + made dns_dispatch_changeattributes() care about the NOLISTEN mask + described side effects of dns_dispatch_changeattributes() in its description comment
2002-09-04 02:26:13 +00:00
* If query, free the buffer, restart.
checkpoint
1999-07-08 02:50:00 +00:00
*
* If response:
* Allocate event, fill in details.
* If cannot allocate, free buffer, restart.
* find target. If not found, free buffer, restart.
* if event queue is not empty, queue. else, send.
* restart.
*/
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
static void
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
udp_recv(isc_nmhandle_t *handle, isc_result_t eresult, isc_region_t *region,
void *arg) {
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dns_dispentry_t *resp = (dns_dispentry_t *)arg;
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
dns_dispatch_t *disp = NULL;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
dns_messageid_t id;
fix many bugs.
1999-07-22 01:34:31 +00:00
isc_result_t dres;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
isc_buffer_t source;
unsigned int flags;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_sockaddr_t peer;
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
isc_netaddr_t netaddr;
Remove dispatch timeout INT16_MAX limitation In some places there was a limitation of the maximum timeout value of INT16_MAX, which is only about 32 seconds. Refactor the code to remove the limitation.
2024-10-30 13:31:34 +00:00
int match;
int64_t timeout = 0;
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
bool respond = true;
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
isc_time_t now;
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
REQUIRE(VALID_RESPONSE(resp));
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
REQUIRE(VALID_DISPATCH(resp->disp));
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
disp = resp->disp;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(resp->reading);
resp->reading = false;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (resp->state == DNS_DISPATCHSTATE_CANCELED) {
/*
* Nobody is interested in the callback if the response
* has been canceled already. Detach from the response
* and the handle.
*/
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
respond = false;
Handle ISC_R_SUCCESS on a deactivated response in udp_recv() There is a possibility for `udp_recv()` to be called with `eresult` being `ISC_R_SUCCESS`, but nevertheless with already deactivated `resp`, which can happen when the request has been canceled in the meantime.
2022-04-27 12:29:50 +00:00
eresult = ISC_R_CANCELED;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
}
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
"read callback:%s, requests %" PRIuFAST32,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(eresult), disp->requests);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
if (eresult != ISC_R_SUCCESS) {
add missing exit checks to udp_recv and tcp_recv
2000-02-02 23:29:47 +00:00
/*
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
* This is most likely a network error on a connected
* socket, a timeout, or the query has been canceled.
* It makes no sense to check the address or parse the
* packet, but we can return the error to the caller.
add missing exit checks to udp_recv and tcp_recv
2000-02-02 23:29:47 +00:00
*/
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto done;
add missing exit checks to udp_recv and tcp_recv
2000-02-02 23:29:47 +00:00
}
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
peer = isc_nmhandle_peeraddr(handle);
isc_netaddr_fromsockaddr(&netaddr, &peer);
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
/*
* If this is from a blackholed address, drop it.
*/
there is no need for each dispatch to have a separate reference to the blackhole list; they can use the blackhole list of the dispatch manager directly. This eliminates some unsafe unlocked accesses to the reference count of the blackhole list dns_acl_t object
2001-02-08 18:25:09 +00:00
if (disp->mgr->blackhole != NULL &&
remove the experimental authoritative ECS support from named - mark the 'geoip-use-ecs' option obsolete; warn when it is used in named.conf - prohibit 'ecs' ACL tags in named.conf; note that this is a fatal error since simply ignoring the tags could make ACLs behave unpredictably - re-simplify the radix and iptable code - clean up dns_acl_match(), dns_aclelement_match(), dns_acl_allowed() and dns_geoip_match() so they no longer take ecs options - remove the ECS-specific unit and system test cases - remove references to ECS from the ARM
2018-04-26 20:57:41 -07:00
dns_acl_match(&netaddr, NULL, disp->mgr->blackhole, NULL, &match,
2353. [func] Add support for Name Server ID (RFC 5001). 'dig +nsid' requests NSID from server. 'request-nsid yes;' causes recursive server to send NSID requests to upstream servers. Server responds to NSID requests with the string configured by 'server-id' option. [RT #17091]
2008-04-03 02:01:08 +00:00
NULL) == ISC_R_SUCCESS &&
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
match > 0)
{
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
if (isc_log_wouldlog(ISC_LOG_DEBUG(10))) {
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
char netaddrstr[ISC_NETADDR_FORMATSIZE];
isc_netaddr_format(&netaddr, netaddrstr,
sizeof(netaddrstr));
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(10),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"blackholed packet from %s", netaddrstr);
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
}
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto next;
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
}
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
/*
* Peek into the buffer to see what we can see.
*/
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
id = resp->id;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_buffer_init(&source, region->base, region->length);
isc_buffer_add(&source, region->length);
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
dres = dns_message_peekheader(&source, &id, &flags);
s/DNS_R_/ISC_R_/ change for some codes.
2000-04-06 22:03:35 +00:00
if (dres != ISC_R_SUCCESS) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
char netaddrstr[ISC_NETADDR_FORMATSIZE];
isc_netaddr_format(&netaddr, netaddrstr, sizeof(netaddrstr));
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(10),
"got garbage packet from %s", netaddrstr);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto next;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
}
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(92),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"got valid DNS message header, /QR %c, id %u",
((flags & DNS_MESSAGEFLAG_QR) != 0) ? '1' : '0', id);
checkpoint
1999-07-08 22:12:37 +00:00
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
/*
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
* Look at the message flags. If it's a query, ignore it.
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
*/
checkpoint
1999-07-06 19:32:40 +00:00
if ((flags & DNS_MESSAGEFLAG_QR) == 0) {
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto next;
1687. [bug] Race condition in dispatch. [RT #10272]
2004-07-21 00:48:19 +00:00
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
/*
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
* The QID and the address must match the expected ones.
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
*/
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
if (resp->id != id || !isc_sockaddr_equal(&peer, &resp->peer)) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
"response doesn't match");
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
inc_stats(disp->mgr, dns_resstatscounter_mismatch);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto next;
Silence cppcheck 2.2 false positive in udp_recv() cppcheck 2.2 reports the following false positive: lib/dns/dispatch.c:1239:14: warning: Either the condition 'resp==NULL' is redundant or there is possible null pointer dereference: resp. [nullPointerRedundantCheck] if (disp != resp->disp) { ^ lib/dns/dispatch.c:1210:11: note: Assuming that condition 'resp==NULL' is not redundant if (resp == NULL) { ^ lib/dns/dispatch.c:1239:14: note: Null pointer dereference if (disp != resp->disp) { ^ Apparently this version of cppcheck gets confused about conditional "goto" statements because line 1239 can never be reached if 'resp' is NULL. Move a code block to prevent the above false positive from being reported without affecting the processing logic.
2020-11-25 12:45:47 +01:00
}
checkpoint
1999-07-08 22:12:37 +00:00
/*
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
* We have the right resp, so call the caller back.
checkpoint
1999-07-08 22:12:37 +00:00
*/
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto done;
checkpoint
1999-07-08 22:12:37 +00:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
next:
/*
Fix handling of mismatched responses past timeout When a UDP dispatch receives a mismatched response, it checks whether there is still enough time to wait for the correct one to arrive before the timeout fires. If there is not, the result code is set to ISC_R_TIMEDOUT, but it is not subsequently used anywhere as 'response' is set to NULL a few lines earlier. This results in the higher-level read callback (resquery_response() in case of resolver code) not being called. However, shortly afterwards, a few levels up the call chain, isc__nm_udp_read_cb() calls isc__nmsocket_timer_stop() on the dispatch socket, effectively disabling read timeout handling for that socket. Combined with the fact that reading is not restarted in such a case (e.g. by calling dispatch_getnext() from udp_recv()), this leads to the higher-level query structure remaining referenced indefinitely because the dispatch socket it uses will neither be read from nor closed due to a timeout. This in turn causes fetch contexts to linger around indefinitely, which in turn i.a. prevents certain cache nodes (those containing rdatasets used by fetch contexts, like fctx->nameservers) from being cleaned. Fix by making sure the higher-level callback does get invoked with the ISC_R_TIMEDOUT result code when udp_recv() determines there is no more time left to receive the correct UDP response before the timeout fires. This allows the higher-level callback to clean things up, preventing the reference leak described above.
2021-11-23 15:35:39 +01:00
* This is the wrong response. Check whether there is still enough
* time to wait for the correct one to arrive before the timeout fires.
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
*/
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
now = isc_loop_now(resp->loop);
Modify dns_dispatch API to accept zero timeout The dns_dispatch_add() has timeout parameter that could not be 0 (for not timeout). Modify the dns_dispatch implementation to accept a zero timeout for cases where the timeouts are undesirable because they are managed externally.
2024-09-20 15:13:09 +02:00
if (resp->timeout > 0) {
timeout = resp->timeout - dispentry_runtime(resp, &now);
if (timeout <= 0) {
/*
* The time window for receiving the correct response is
* already closed, libuv has just not processed the
* socket timer yet. Invoke the read callback,
* indicating a timeout.
*/
eresult = ISC_R_TIMEDOUT;
goto done;
}
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
}
Fix handling of mismatched responses past timeout When a UDP dispatch receives a mismatched response, it checks whether there is still enough time to wait for the correct one to arrive before the timeout fires. If there is not, the result code is set to ISC_R_TIMEDOUT, but it is not subsequently used anywhere as 'response' is set to NULL a few lines earlier. This results in the higher-level read callback (resquery_response() in case of resolver code) not being called. However, shortly afterwards, a few levels up the call chain, isc__nm_udp_read_cb() calls isc__nmsocket_timer_stop() on the dispatch socket, effectively disabling read timeout handling for that socket. Combined with the fact that reading is not restarted in such a case (e.g. by calling dispatch_getnext() from udp_recv()), this leads to the higher-level query structure remaining referenced indefinitely because the dispatch socket it uses will neither be read from nor closed due to a timeout. This in turn causes fetch contexts to linger around indefinitely, which in turn i.a. prevents certain cache nodes (those containing rdatasets used by fetch contexts, like fctx->nameservers) from being cleaned. Fix by making sure the higher-level callback does get invoked with the ISC_R_TIMEDOUT result code when udp_recv() determines there is no more time left to receive the correct UDP response before the timeout fires. This allows the higher-level callback to clean things up, preventing the reference leak described above.
2021-11-23 15:35:39 +01:00
/*
* Do not invoke the read callback just yet and instead wait for the
* proper response to arrive until the original timeout fires.
*/
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
respond = false;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
udp_dispatch_getnext(resp, timeout);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
done:
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
if (respond) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
"UDP read callback on %p: %s", handle,
isc_result_totext(eresult));
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
resp->response(eresult, region, resp->arg);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_detach(&resp); /* DISPENTRY003 */
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
}
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
static isc_result_t
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_recv_oldest(dns_dispatch_t *disp, dns_dispentry_t **respp) {
dns_dispentry_t *resp = NULL;
resp = ISC_LIST_HEAD(disp->active);
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
if (resp != NULL) {
disp->timedout++;
*respp = resp;
return ISC_R_TIMEDOUT;
}
return ISC_R_NOTFOUND;
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
/*
* NOTE: Must be RCU read locked!
*/
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
static isc_result_t
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
tcp_recv_success(dns_dispatch_t *disp, isc_region_t *region,
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
isc_sockaddr_t *peer, dns_dispentry_t **respp) {
isc_buffer_t source;
dns_messageid_t id;
unsigned int flags;
isc_result_t result = ISC_R_SUCCESS;
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(90),
"TCP read success, length == %d, addr = %p",
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
region->length, region->base);
/*
* Peek into the buffer to see what we can see.
*/
isc_buffer_init(&source, region->base, region->length);
isc_buffer_add(&source, region->length);
result = dns_message_peekheader(&source, &id, &flags);
if (result != ISC_R_SUCCESS) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(10), "got garbage packet");
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
return ISC_R_UNEXPECTED;
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(92),
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
"got valid DNS message header, /QR %c, id %u",
((flags & DNS_MESSAGEFLAG_QR) != 0) ? '1' : '0', id);
/*
* Look at the message flags. If it's a query, ignore it and keep
* reading.
*/
if ((flags & DNS_MESSAGEFLAG_QR) == 0) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(10),
"got DNS query instead of answer");
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
return ISC_R_UNEXPECTED;
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
/*
* We have a valid response; find the associated dispentry object
* and call the caller back.
*/
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
dns_dispentry_t key = {
.id = id,
.peer = *peer,
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
.port = isc_sockaddr_getport(&disp->local),
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
};
struct cds_lfht_iter iter;
cds_lfht_lookup(disp->mgr->qids, qid_hash(&key), qid_match, &key,
&iter);
dns_dispentry_t *resp = cds_lfht_entry(cds_lfht_iter_get_node(&iter),
dns_dispentry_t, ht_node);
Skip TCP dispatch responses that are not ours When matching the TCP dispatch responses, we should skip the responses that do not belong to our TCP connection. This can happen with faulty upstream server that sends invalid QID back to us.
2024-10-02 08:37:48 +02:00
/* Skip responses that are not ours */
if (resp != NULL && resp->disp == disp) {
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
if (!resp->reading) {
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
/*
* We already got a message for this QID and weren't
* expecting any more.
*/
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
result = ISC_R_UNEXPECTED;
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
} else {
*respp = resp;
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
}
Tear down the TCP connection on too many unexpected DNS messages When the outgoing TCP dispatch times-out active response, we might still receive the answer during the lifetime of the connection. Previously, we would just ignore any non-matching DNS answers, which would allow the server to feed us with otherwise valid DNS answer and keep the connection open. Add a counter for timed-out DNS queries over TCP and tear down the whole TCP connection if we receive unexpected number of DNS answers.
2021-11-29 09:59:33 +01:00
} else {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
result = ISC_R_NOTFOUND;
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(90),
"search for response in hashtable: %s",
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
isc_result_totext(result));
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
return result;
}
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_recv_add(dns_displist_t *resps, dns_dispentry_t *resp,
isc_result_t result) {
dns_dispentry_ref(resp); /* DISPENTRY009 */
ISC_LIST_UNLINK(resp->disp->active, resp, alink);
ISC_LIST_APPEND(*resps, resp, rlink);
INSIST(resp->reading);
resp->reading = false;
resp->result = result;
}
static void
tcp_recv_shutdown(dns_dispatch_t *disp, dns_displist_t *resps,
isc_result_t result) {
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
/*
* If there are any active responses, shut them all down.
*/
make all ISC_LIST_FOREACH calls safe previously, ISC_LIST_FOREACH and ISC_LIST_FOREACH_SAFE were two separate macros, with the _SAFE version allowing entries to be unlinked during the loop. ISC_LIST_FOREACH is now also safe, and the separate _SAFE macro has been removed. similarly, the ISC_LIST_FOREACH_REV macro is now safe, and ISC_LIST_FOREACH_REV_SAFE has also been removed.
2025-05-23 13:02:22 -07:00
ISC_LIST_FOREACH(disp->active, resp, alink) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_recv_add(resps, resp, result);
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
Ignore TCP dispatches that have zero references The TCP dispatches are removed from the dispatchmgr->list in the dispatch_destroy() and there's a brief period of time where dns_dispatch_gettcp() can find a dispatch in connected state that's being destroyed. Set the dispatch state to DNS_DISPATCHSTATE_NONE in the TCP connection callback if there are no responses waiting, and ignore TCP dispatches with zero references in dns_dispatch_gettcp().
2022-12-20 06:11:26 +01:00
disp->state = DNS_DISPATCHSTATE_CANCELED;
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_recv_processall(dns_displist_t *resps, isc_region_t *region) {
make all ISC_LIST_FOREACH calls safe previously, ISC_LIST_FOREACH and ISC_LIST_FOREACH_SAFE were two separate macros, with the _SAFE version allowing entries to be unlinked during the loop. ISC_LIST_FOREACH is now also safe, and the separate _SAFE macro has been removed. similarly, the ISC_LIST_FOREACH_REV macro is now safe, and ISC_LIST_FOREACH_REV_SAFE has also been removed.
2025-05-23 13:02:22 -07:00
ISC_LIST_FOREACH(*resps, resp, rlink) {
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
ISC_LIST_UNLINK(*resps, resp, rlink);
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "read callback: %s",
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
isc_result_totext(resp->result));
resp->response(resp->result, region, resp->arg);
dns_dispentry_detach(&resp); /* DISPENTRY009 */
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
}
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
/*
* General flow:
*
fixed a bug that named crashes with an assertion failure on exit when sharing the same port for listening and querying, and changing listening addresses several times. [RT# 3509] additionally, + limited the canceled socket tasks in dispatch.c + made dns_dispatch_changeattributes() care about the NOLISTEN mask + described side effects of dns_dispatch_changeattributes() in its description comment
2002-09-04 02:26:13 +00:00
* If I/O result == CANCELED, EOF, or error, notify everyone as the
* various queues drain.
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
*
* If response:
* Allocate event, fill in details.
fixed a bug that named crashes with an assertion failure on exit when sharing the same port for listening and querying, and changing listening addresses several times. [RT# 3509] additionally, + limited the canceled socket tasks in dispatch.c + made dns_dispatch_changeattributes() care about the NOLISTEN mask + described side effects of dns_dispatch_changeattributes() in its description comment
2002-09-04 02:26:13 +00:00
* If cannot allocate, restart.
* find target. If not found, restart.
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
* if event queue is not empty, queue. else, send.
* restart.
*/
static void
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
tcp_recv(isc_nmhandle_t *handle, isc_result_t result, isc_region_t *region,
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
void *arg) {
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
dns_dispatch_t *disp = (dns_dispatch_t *)arg;
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
dns_dispentry_t *resp = NULL;
1342, [func] Log remote address with TCP dispatch failures.
2002-07-29 01:03:24 +00:00
char buf[ISC_SOCKADDR_FORMATSIZE];
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_sockaddr_t peer;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_displist_t resps = ISC_LIST_INITIALIZER;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
isc_time_t now;
Modify dns_dispatch API to accept zero timeout The dns_dispatch_add() has timeout parameter that could not be 0 (for not timeout). Modify the dns_dispatch implementation to accept a zero timeout for cases where the timeouts are undesirable because they are managed externally.
2024-09-20 15:13:09 +02:00
int timeout = 0;
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
REQUIRE(VALID_DISPATCH(disp));
In tcp_recv(), free the event before calling destroy(), since the event is a statically allocated part of the tcpmsg.
1999-11-16 21:05:09 +00:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(disp->reading);
disp->reading = false;
lock dispatch before reporting state
2019-11-22 10:49:40 +11:00
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(90),
"TCP read:%s:requests %" PRIuFAST32,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(result), disp->requests);
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
peer = isc_nmhandle_peeraddr(handle);
2353. [func] Add support for Name Server ID (RFC 5001). 'dig +nsid' requests NSID from server. 'request-nsid yes;' causes recursive server to send NSID requests to upstream servers. Server responds to NSID requests with the string configured by 'server-id' option. [RT #17091]
2008-04-03 02:01:08 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_lock();
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/*
* Phase 1: Process timeout and success.
*/
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
switch (result) {
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
case ISC_R_TIMEDOUT:
/*
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
* Time out the oldest response in the active queue.
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
*/
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
result = tcp_recv_oldest(disp, &resp);
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
break;
case ISC_R_SUCCESS:
/* We got an answer */
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
result = tcp_recv_success(disp, region, &peer, &resp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
break;
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
default:
break;
}
if (resp != NULL) {
tcp_recv_add(&resps, resp, result);
}
/*
* Phase 2: Look if we timed out before.
*/
if (result == ISC_R_NOTFOUND) {
if (disp->timedout > 0) {
/* There was active query that timed-out before */
disp->timedout--;
} else {
result = ISC_R_UNEXPECTED;
}
}
/*
* Phase 3: Trigger timeouts. It's possible that the responses would
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
* have been timed out out already, but non-matching TCP reads have
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
* prevented this.
*/
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp = ISC_LIST_HEAD(disp->active);
if (resp != NULL) {
now = isc_loop_now(resp->loop);
}
while (resp != NULL) {
dns_dispentry_t *next = ISC_LIST_NEXT(resp, alink);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
Modify dns_dispatch API to accept zero timeout The dns_dispatch_add() has timeout parameter that could not be 0 (for not timeout). Modify the dns_dispatch implementation to accept a zero timeout for cases where the timeouts are undesirable because they are managed externally.
2024-09-20 15:13:09 +02:00
if (resp->timeout > 0) {
timeout = resp->timeout - dispentry_runtime(resp, &now);
if (timeout <= 0) {
tcp_recv_add(&resps, resp, ISC_R_TIMEDOUT);
}
TCP receives should never be retried
2000-09-08 22:02:21 +00:00
}
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp = next;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
/*
* Phase 4: log if we errored out.
*/
switch (result) {
case ISC_R_SUCCESS:
case ISC_R_TIMEDOUT:
case ISC_R_NOTFOUND:
break;
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case ISC_R_SHUTTINGDOWN:
case ISC_R_CANCELED:
case ISC_R_EOF:
case ISC_R_CONNECTIONRESET:
isc_sockaddr_format(&peer, buf, sizeof(buf));
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(90),
"shutting down TCP: %s: %s", buf,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(result));
tcp_recv_shutdown(disp, &resps, result);
break;
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
default:
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
isc_sockaddr_format(&peer, buf, sizeof(buf));
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
dispatch_log(disp, ISC_LOG_ERROR,
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
"shutting down due to TCP "
"receive error: %s: %s",
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
buf, isc_result_totext(result));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_recv_shutdown(disp, &resps, result);
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
break;
Shutdown all TCP connection on invalid DNS message Previously, when invalid DNS message is received over TCP we throw the garbage DNS message away and continued looking for valid DNS message that would match our outgoing queries. This logic makes sense for UDP, because anyone can send DNS message over UDP. Change the logic that the TCP connection is closed when we receive garbage, because the other side is acting malicious.
2021-11-26 09:14:58 +01:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/*
* Phase 5: Resume reading if there are still active responses
*/
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
resp = ISC_LIST_HEAD(disp->active);
if (resp != NULL) {
Modify dns_dispatch API to accept zero timeout The dns_dispatch_add() has timeout parameter that could not be 0 (for not timeout). Modify the dns_dispatch implementation to accept a zero timeout for cases where the timeouts are undesirable because they are managed externally.
2024-09-20 15:13:09 +02:00
if (resp->timeout > 0) {
timeout = resp->timeout - dispentry_runtime(resp, &now);
INSIST(timeout > 0);
}
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
tcp_startrecv(disp, resp);
Modify dns_dispatch API to accept zero timeout The dns_dispatch_add() has timeout parameter that could not be 0 (for not timeout). Modify the dns_dispatch implementation to accept a zero timeout for cases where the timeouts are undesirable because they are managed externally.
2024-09-20 15:13:09 +02:00
if (timeout > 0) {
isc_nmhandle_settimeout(handle, timeout);
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_unlock();
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/*
* Phase 6: Process all scheduled callbacks.
*/
tcp_recv_processall(&resps, region);
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_detach(&disp); /* DISPATCH002 */
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
/*%
* Create a temporary port list to set the initial default set of dispatch
Use system ephemeral ports for default portset In dispatch, use system ephemeral ports for default portset instead of hardcoded <1024,65535> range.
2021-10-04 14:11:57 +02:00
* ephemeral ports. This is almost meaningless as the application will
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
* normally set the ports explicitly, but is provided to fill some minor corner
* cases.
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
*/
static void
Use system ephemeral ports for default portset In dispatch, use system ephemeral ports for default portset instead of hardcoded <1024,65535> range.
2021-10-04 14:11:57 +02:00
create_default_portset(isc_mem_t *mctx, int family, isc_portset_t **portsetp) {
in_port_t low, high;
isc_net_getudpportrange(family, &low, &high);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_portset_create(mctx, portsetp);
Use system ephemeral ports for default portset In dispatch, use system ephemeral ports for default portset instead of hardcoded <1024,65535> range.
2021-10-04 14:11:57 +02:00
isc_portset_addrange(*portsetp, low, high);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
static isc_result_t
setavailports(dns_dispatchmgr_t *mgr, isc_portset_t *v4portset,
isc_portset_t *v6portset) {
in_port_t *v4ports, *v6ports, p = 0;
unsigned int nv4ports, nv6ports, i4 = 0, i6 = 0;
nv4ports = isc_portset_nports(v4portset);
nv6ports = isc_portset_nports(v6portset);
v4ports = NULL;
if (nv4ports != 0) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
v4ports = isc_mem_cget(mgr->mctx, nv4ports, sizeof(in_port_t));
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
v6ports = NULL;
if (nv6ports != 0) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
v6ports = isc_mem_cget(mgr->mctx, nv6ports, sizeof(in_port_t));
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
}
do {
if (isc_portset_isset(v4portset, p)) {
INSIST(i4 < nv4ports);
v4ports[i4++] = p;
}
if (isc_portset_isset(v6portset, p)) {
INSIST(i6 < nv6ports);
v6ports[i6++] = p;
}
} while (p++ < 65535);
INSIST(i4 == nv4ports && i6 == nv6ports);
if (mgr->v4ports != NULL) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
isc_mem_cput(mgr->mctx, mgr->v4ports, mgr->nv4ports,
sizeof(in_port_t));
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
}
mgr->v4ports = v4ports;
mgr->nv4ports = nv4ports;
if (mgr->v6ports != NULL) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
isc_mem_cput(mgr->mctx, mgr->v6ports, mgr->nv6ports,
sizeof(in_port_t));
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
}
mgr->v6ports = v6ports;
mgr->nv6ports = nv6ports;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
return ISC_R_SUCCESS;
}
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
/*
* Publics.
*/
fix many bugs.
1999-07-22 01:34:31 +00:00
isc_result_t
Refactor the network manager to be a singleton There is only a single network manager running on top of the loop manager (except for tests). Refactor the network manager to be a singleton (a single instance) and change the unit tests, so that the shorter read timeouts apply only to a specific handle, not the whole extra 'connect_nm' network manager instance.
2025-07-14 17:12:35 +02:00
dns_dispatchmgr_create(isc_mem_t *mctx, dns_dispatchmgr_t **mgrp) {
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
dns_dispatchmgr_t *mgr = NULL;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
isc_portset_t *v4portset = NULL;
isc_portset_t *v6portset = NULL;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
REQUIRE(mctx != NULL);
REQUIRE(mgrp != NULL && *mgrp == NULL);
mgr = isc_mem_get(mctx, sizeof(dns_dispatchmgr_t));
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
*mgr = (dns_dispatchmgr_t){
.magic = 0,
Change the loopmgr to be singleton All the applications built on top of the loop manager were required to create just a single instance of the loop manager. Refactor the loop manager to not expose this instance to the callers and keep the loop manager object internal to the isc_loop compilation unit. This significantly simplifies a number of data structures and calls to the isc_loop API.
2025-07-14 10:50:21 +02:00
.nloops = isc_loopmgr_nloops(),
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
};
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
fprintf(stderr, "dns_dispatchmgr__init:%s:%s:%d:%p->references = 1\n",
__func__, __FILE__, __LINE__, mgr);
#endif
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_refcount_init(&mgr->references, 1);
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
isc_mem_attach(mctx, &mgr->mctx);
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
mgr->tcps = isc_mem_cget(mgr->mctx, mgr->nloops, sizeof(mgr->tcps[0]));
for (size_t i = 0; i < mgr->nloops; i++) {
mgr->tcps[i] = cds_lfht_new(
2, 2, 0, CDS_LFHT_AUTO_RESIZE | CDS_LFHT_ACCOUNTING,
NULL);
}
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
create_default_portset(mgr->mctx, AF_INET, &v4portset);
create_default_portset(mgr->mctx, AF_INET6, &v6portset);
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
setavailports(mgr, v4portset, v6portset);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
isc_portset_destroy(mgr->mctx, &v4portset);
isc_portset_destroy(mgr->mctx, &v6portset);
mgr->qids = cds_lfht_new(QIDS_INIT_SIZE, QIDS_MIN_SIZE, 0,
CDS_LFHT_AUTO_RESIZE | CDS_LFHT_ACCOUNTING,
NULL);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
mgr->magic = DNS_DISPATCHMGR_MAGIC;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
*mgrp = mgr;
return ISC_R_SUCCESS;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
ISC_REFCOUNT_TRACE_IMPL(dns_dispatchmgr, dispatchmgr_destroy);
#else
ISC_REFCOUNT_IMPL(dns_dispatchmgr, dispatchmgr_destroy);
#endif
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
void
dns_dispatchmgr_setblackhole(dns_dispatchmgr_t *mgr, dns_acl_t *blackhole) {
REQUIRE(VALID_DISPATCHMGR(mgr));
635. [bug] Reloading a server with a configured blackhole list would cause an assertion. [RT #590]
2000-12-26 09:48:41 +00:00
if (mgr->blackhole != NULL) {
dns_acl_detach(&mgr->blackhole);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
dns_acl_attach(blackhole, &mgr->blackhole);
}
removed more needless attaching and detaching of the blackhole ACL
2001-02-09 00:23:16 +00:00
dns_acl_t *
dns_dispatchmgr_getblackhole(dns_dispatchmgr_t *mgr) {
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
REQUIRE(VALID_DISPATCHMGR(mgr));
removed more needless attaching and detaching of the blackhole ACL
2001-02-09 00:23:16 +00:00
return mgr->blackhole;
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
isc_result_t
dns_dispatchmgr_setavailports(dns_dispatchmgr_t *mgr, isc_portset_t *v4portset,
isc_portset_t *v6portset) {
REQUIRE(VALID_DISPATCHMGR(mgr));
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
return setavailports(mgr, v4portset, v6portset);
1442. [func] New fuctions for manipulating port lists: dns_portlist_create(), dns_portlist_add(), dns_portlist_remove(), dns_portlist_match(), dns_portlist_attach() and dns_portlist_detach(). 1441. [func] It is now possible to tell dig to bind to a specific source port. 1440. [func] It is now possible to tell named to avoid using certian source ports (avoid-v4-udp-ports, avoid-v6-udp-ports). developer: marka reviewer: explorer
2003-02-26 05:05:16 +00:00
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
static void
dispatchmgr_destroy(dns_dispatchmgr_t *mgr) {
REQUIRE(VALID_DISPATCHMGR(mgr));
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_refcount_destroy(&mgr->references);
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
mgr->magic = 0;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
RUNTIME_CHECK(!cds_lfht_destroy(mgr->qids, NULL));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
for (size_t i = 0; i < mgr->nloops; i++) {
RUNTIME_CHECK(!cds_lfht_destroy(mgr->tcps[i], NULL));
}
isc_mem_cput(mgr->mctx, mgr->tcps, mgr->nloops, sizeof(mgr->tcps[0]));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (mgr->blackhole != NULL) {
dns_acl_detach(&mgr->blackhole);
}
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (mgr->stats != NULL) {
isc_stats_detach(&mgr->stats);
}
Trailing whitespace trimmed. Perhaps running "perl util/spacewhack.pl in your own CVS tree will help minimize CVS conflicts. Maybe not. Blame Graff for getting me to trim all trailing whitespace.
2000-08-01 01:33:37 +00:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (mgr->v4ports != NULL) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
isc_mem_cput(mgr->mctx, mgr->v4ports, mgr->nv4ports,
sizeof(in_port_t));
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (mgr->v6ports != NULL) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
isc_mem_cput(mgr->mctx, mgr->v6ports, mgr->nv6ports,
sizeof(in_port_t));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
isc_mem_putanddetach(&mgr->mctx, mgr, sizeof(dns_dispatchmgr_t));
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
}
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
void
2537. [func] Added more statistics counters including those on socket I/O events and query RTT histograms. [RT #18802]
2009-01-27 22:30:00 +00:00
dns_dispatchmgr_setstats(dns_dispatchmgr_t *mgr, isc_stats_t *stats) {
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
REQUIRE(VALID_DISPATCHMGR(mgr));
REQUIRE(mgr->stats == NULL);
2537. [func] Added more statistics counters including those on socket I/O events and query RTT histograms. [RT #18802]
2009-01-27 22:30:00 +00:00
isc_stats_attach(stats, &mgr->stats);
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
}
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
/*
* Allocate and set important limits.
*/
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
static void
Convert the isc/tid.h to use own signed integer isc_tid_t type Change the internal type used for isc_tid unit to isc_tid_t to hide the specific integer type being used for the 'tid'. Internally, the signed integer type is being used. This allows us to have negatively indexed arrays that works both for threads with assigned tid and the threads with unassigned tid. This should be used only in specific situations.
2025-06-04 17:54:20 +02:00
dispatch_allocate(dns_dispatchmgr_t *mgr, isc_socktype_t type, isc_tid_t tid,
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
dns_dispatch_t **dispp) {
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
dns_dispatch_t *disp = NULL;
checkpoint
1999-06-18 02:01:42 +00:00
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
REQUIRE(VALID_DISPATCHMGR(mgr));
checkpoint
1999-06-18 23:54:59 +00:00
REQUIRE(dispp != NULL && *dispp == NULL);
checkpoint
1999-06-18 02:01:42 +00:00
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
/*
* Set up the dispatcher, mostly. Don't bother setting some of
* the options that are controlled by tcp vs. udp, etc.
*/
checkpoint
1999-06-18 02:01:42 +00:00
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
disp = isc_mem_get(mgr->mctx, sizeof(*disp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
*disp = (dns_dispatch_t){
.socktype = type,
.active = ISC_LIST_INITIALIZER,
.pending = ISC_LIST_INITIALIZER,
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
.tid = tid,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
.magic = DISPATCH_MAGIC,
};
checkpoint
1999-06-18 02:01:42 +00:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
isc_mem_attach(mgr->mctx, &disp->mctx);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dns_dispatchmgr_attach(mgr, &disp->mgr);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
fprintf(stderr, "dns_dispatch__init:%s:%s:%d:%p->references = 1\n",
__func__, __FILE__, __LINE__, disp);
#endif
isc_refcount_init(&disp->references, 1); /* DISPATCH000 */
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
*dispp = disp;
}
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
struct dispatch_key {
const isc_sockaddr_t *local;
const isc_sockaddr_t *peer;
Fix TCP dispatches and transport Dispatch needs to know the transport that is being used over the TCP connection to correctly allow for it to be reused. Add a transport parameter to dns_dispatch_createtcp and dns_dispatch_gettcp and use it when selecting a TCP socket for reuse.
2024-10-15 16:09:48 +11:00
const dns_transport_t *transport;
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
};
static uint32_t
dispatch_hash(struct dispatch_key *key) {
uint32_t hashval = isc_sockaddr_hash(key->peer, false);
if (key->local) {
hashval ^= isc_sockaddr_hash(key->local, true);
}
return hashval;
}
static int
dispatch_match(struct cds_lfht_node *node, const void *key0) {
dns_dispatch_t *disp = caa_container_of(node, dns_dispatch_t, ht_node);
const struct dispatch_key *key = key0;
Improve reuse of outgoing TCP connections The dns_dispatch_gettcp() function is used for finding an existing TCP connection that can be reused for sending a query from a specified local address to a specified remote address. The logic for matching the provided <local address, remote address> tuple to one of the existing TCP connections is implemented in the dispatch_match() function: - if the examined TCP connection already has a libuv handle assigned, it means the connection has already been established; therefore, compare the provided <local address, remote address> tuple against the corresponding address tuple for the libuv handle associated with the connection, - if the examined TCP connection does not yet have a libuv handle assigned, it means the connection has not yet been established; therefore, compare the provided <local address, remote address> tuple against the corresponding address tuple that the TCP connection was originally created for. This logic limits TCP connection reuse potential as the libuv handle assigned to an existing dispatch object may have a more specific local <address, port> tuple associated with it than the local <address, port> tuple that the dispatch object was originally created for. That's because the local address for outgoing connections can be set to a wildcard <address, port> tuple (indicating that the caller does not care what source <address, port> tuple will be used for establishing the connection, thereby delegating the task of picking it to the operating system) and then get "upgraded" to a specific <address, port> tuple when the socket is bound (and a libuv handle gets associated with it). When another dns_dispatch_gettcp() caller then tries to look for an existing TCP connection to the same peer and passes a wildcard address in the local part of the tuple, the function will not match that request to a previously-established TCP connection (unless isc_nmhandle_localaddr() returns a wildcard address as well). Simplify dispatch_match() so that the libuv handle associated with an existing dispatch object is not examined for the purpose of matching it to the provided <local address, remote address> tuple; instead, always examine the <local address, remote address> tuple that the dispatch object was originally created for. This enables reuse of TCP connections created without providing a specific local socket address while still preventing other connections (created for a specific local socket address) from being inadvertently shared.
2024-12-29 10:22:20 +01:00
return disp->transport == key->transport &&
isc_sockaddr_equal(&disp->peer, key->peer) &&
(key->local == NULL ||
isc_sockaddr_equal(&disp->local, key->local));
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
}
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
isc_result_t
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
dns_dispatch_createtcp(dns_dispatchmgr_t *mgr, const isc_sockaddr_t *localaddr,
Add option to mark TCP dispatch as unshared The current dispatch code could reuse the TCP connection when dns_dispatch_gettcp() would be used first. This is problematic as the dns_resolver doesn't use TCP connection sharing, but dns_request could get the TCP stream that was created outside of the dns_request. Add new DNS_DISPATCHOPT_UNSHARED option to dns_dispatch_createtcp() that would prevent the TCP stream to be reused. Use that option in the dns_resolver call to dns_dispatch_createtcp() to prevent dns_request from reusing the TCP connections created by dns_resolver. Additionally, the dns_xfrin unit added TCP connection sharing for incoming transfers. While interleaving *xfr streams on a TCP connection should work this should be a deliberate change and be property of the server that can be controlled. Additionally some level of parallel TCP streams is desirable. Revert to the old behaviour by removing the dns_dispatch_gettcp() calls from dns_xfrin and use the new option to prevent from sharing the transfer streams with dns_request.
2023-10-20 08:14:27 +02:00
const isc_sockaddr_t *destaddr,
Fix TCP dispatches and transport Dispatch needs to know the transport that is being used over the TCP connection to correctly allow for it to be reused. Add a transport parameter to dns_dispatch_createtcp and dns_dispatch_gettcp and use it when selecting a TCP socket for reuse.
2024-10-15 16:09:48 +11:00
dns_transport_t *transport, dns_dispatchopt_t options,
dns_dispatch_t **dispp) {
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
dns_dispatch_t *disp = NULL;
Convert the isc/tid.h to use own signed integer isc_tid_t type Change the internal type used for isc_tid unit to isc_tid_t to hide the specific integer type being used for the 'tid'. Internally, the signed integer type is being used. This allows us to have negatively indexed arrays that works both for threads with assigned tid and the threads with unassigned tid. This should be used only in specific situations.
2025-06-04 17:54:20 +02:00
isc_tid_t tid = isc_tid();
Move the UDP buffer pool to the dispatchmgr. UDP parameters are now set by the first call to dns_dispatch_getudp(), subsecquent calls can raise the maximum number of buffers. dns_dispatchmgr_setudp() could be made public. Tidy dispatch_allocate(). entropy and mctx are now gone from dns_qid_t.
2000-09-19 06:59:28 +00:00
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
REQUIRE(VALID_DISPATCHMGR(mgr));
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
REQUIRE(destaddr != NULL);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
dispatch_allocate(mgr, isc_socktype_tcp, tid, &disp);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Add option to mark TCP dispatch as unshared The current dispatch code could reuse the TCP connection when dns_dispatch_gettcp() would be used first. This is problematic as the dns_resolver doesn't use TCP connection sharing, but dns_request could get the TCP stream that was created outside of the dns_request. Add new DNS_DISPATCHOPT_UNSHARED option to dns_dispatch_createtcp() that would prevent the TCP stream to be reused. Use that option in the dns_resolver call to dns_dispatch_createtcp() to prevent dns_request from reusing the TCP connections created by dns_resolver. Additionally, the dns_xfrin unit added TCP connection sharing for incoming transfers. While interleaving *xfr streams on a TCP connection should work this should be a deliberate change and be property of the server that can be controlled. Additionally some level of parallel TCP streams is desirable. Revert to the old behaviour by removing the dns_dispatch_gettcp() calls from dns_xfrin and use the new option to prevent from sharing the transfer streams with dns_request.
2023-10-20 08:14:27 +02:00
disp->options = options;
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
disp->peer = *destaddr;
Fix TCP dispatches and transport Dispatch needs to know the transport that is being used over the TCP connection to correctly allow for it to be reused. Add a transport parameter to dns_dispatch_createtcp and dns_dispatch_gettcp and use it when selecting a TCP socket for reuse.
2024-10-15 16:09:48 +11:00
if (transport != NULL) {
dns_transport_attach(transport, &disp->transport);
}
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
if (localaddr != NULL) {
disp->local = *localaddr;
} else {
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
int pf;
pf = isc_sockaddr_pf(destaddr);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_sockaddr_anyofpf(&disp->local, pf);
isc_sockaddr_setport(&disp->local, 0);
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
}
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
/*
* Append it to the dispatcher list.
*/
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
struct dispatch_key key = {
.local = &disp->local,
.peer = &disp->peer,
Fix TCP dispatches and transport Dispatch needs to know the transport that is being used over the TCP connection to correctly allow for it to be reused. Add a transport parameter to dns_dispatch_createtcp and dns_dispatch_gettcp and use it when selecting a TCP socket for reuse.
2024-10-15 16:09:48 +11:00
.transport = transport,
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
};
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
Add option to mark TCP dispatch as unshared The current dispatch code could reuse the TCP connection when dns_dispatch_gettcp() would be used first. This is problematic as the dns_resolver doesn't use TCP connection sharing, but dns_request could get the TCP stream that was created outside of the dns_request. Add new DNS_DISPATCHOPT_UNSHARED option to dns_dispatch_createtcp() that would prevent the TCP stream to be reused. Use that option in the dns_resolver call to dns_dispatch_createtcp() to prevent dns_request from reusing the TCP connections created by dns_resolver. Additionally, the dns_xfrin unit added TCP connection sharing for incoming transfers. While interleaving *xfr streams on a TCP connection should work this should be a deliberate change and be property of the server that can be controlled. Additionally some level of parallel TCP streams is desirable. Revert to the old behaviour by removing the dns_dispatch_gettcp() calls from dns_xfrin and use the new option to prevent from sharing the transfer streams with dns_request.
2023-10-20 08:14:27 +02:00
if ((disp->options & DNS_DISPATCHOPT_UNSHARED) == 0) {
rcu_read_lock();
cds_lfht_add(mgr->tcps[tid], dispatch_hash(&key),
&disp->ht_node);
rcu_read_unlock();
}
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
if (isc_log_wouldlog(90)) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
char addrbuf[ISC_SOCKADDR_FORMATSIZE];
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
Don't pass a NULL pointer to isc_sockaddr_format() The 'localaddr' pointer can be NULL, which causes an assertion failure. Use '&disp->local' instead when printing a debug log message.
2022-12-28 12:10:09 +00:00
isc_sockaddr_format(&disp->local, addrbuf,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
ISC_SOCKADDR_FORMATSIZE);
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
mgr_log(mgr, ISC_LOG_DEBUG(90),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"dns_dispatch_createtcp: created TCP dispatch %p for "
"%s",
disp, addrbuf);
}
*dispp = disp;
return ISC_R_SUCCESS;
}
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
3939. [func] Improve UPDATE forwarding performance by allowing TCP connections to be shared. [RT #37039]
2014-09-04 10:37:45 +10:00
isc_result_t
4546. [func] Extend the use of const declarations. [RT #43379]
2016-12-30 15:45:08 +11:00
dns_dispatch_gettcp(dns_dispatchmgr_t *mgr, const isc_sockaddr_t *destaddr,
Fix TCP dispatches and transport Dispatch needs to know the transport that is being used over the TCP connection to correctly allow for it to be reused. Add a transport parameter to dns_dispatch_createtcp and dns_dispatch_gettcp and use it when selecting a TCP socket for reuse.
2024-10-15 16:09:48 +11:00
const isc_sockaddr_t *localaddr, dns_transport_t *transport,
dns_dispatch_t **dispp) {
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
dns_dispatch_t *disp_connected = NULL;
dns_dispatch_t *disp_fallback = NULL;
isc_result_t result = ISC_R_NOTFOUND;
Convert the isc/tid.h to use own signed integer isc_tid_t type Change the internal type used for isc_tid unit to isc_tid_t to hide the specific integer type being used for the 'tid'. Internally, the signed integer type is being used. This allows us to have negatively indexed arrays that works both for threads with assigned tid and the threads with unassigned tid. This should be used only in specific situations.
2025-06-04 17:54:20 +02:00
isc_tid_t tid = isc_tid();
[master] allow shared TCP sockets when connecting 4041. [func] TCP sockets can now be shared while connecting. (This will be used to enable client-side support of pipelined queries.) [RT #38231]
2015-01-20 17:22:31 -08:00
REQUIRE(VALID_DISPATCHMGR(mgr));
REQUIRE(destaddr != NULL);
REQUIRE(dispp != NULL && *dispp == NULL);
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
struct dispatch_key key = {
.local = localaddr,
.peer = destaddr,
Fix TCP dispatches and transport Dispatch needs to know the transport that is being used over the TCP connection to correctly allow for it to be reused. Add a transport parameter to dns_dispatch_createtcp and dns_dispatch_gettcp and use it when selecting a TCP socket for reuse.
2024-10-15 16:09:48 +11:00
.transport = transport,
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
};
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
rcu_read_lock();
struct cds_lfht_iter iter;
dns_dispatch_t *disp = NULL;
cds_lfht_for_each_entry_duplicate(mgr->tcps[tid], dispatch_hash(&key),
dispatch_match, &key, &iter, disp,
ht_node) {
INSIST(disp->tid == isc_tid());
INSIST(disp->socktype == isc_socktype_tcp);
Ignore TCP dispatches that have zero references The TCP dispatches are removed from the dispatchmgr->list in the dispatch_destroy() and there's a brief period of time where dns_dispatch_gettcp() can find a dispatch in connected state that's being destroyed. Set the dispatch state to DNS_DISPATCHSTATE_NONE in the TCP connection callback if there are no responses waiting, and ignore TCP dispatches with zero references in dns_dispatch_gettcp().
2022-12-20 06:11:26 +01:00
switch (disp->state) {
case DNS_DISPATCHSTATE_NONE:
/* A dispatch in indeterminate state, skip it */
break;
case DNS_DISPATCHSTATE_CONNECTED:
if (ISC_LIST_EMPTY(disp->active)) {
/* Ignore dispatch with no responses */
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
break;
Ignore TCP dispatches that have zero references The TCP dispatches are removed from the dispatchmgr->list in the dispatch_destroy() and there's a brief period of time where dns_dispatch_gettcp() can find a dispatch in connected state that's being destroyed. Set the dispatch state to DNS_DISPATCHSTATE_NONE in the TCP connection callback if there are no responses waiting, and ignore TCP dispatches with zero references in dns_dispatch_gettcp().
2022-12-20 06:11:26 +01:00
}
/* We found a connected dispatch */
dns_dispatch_attach(disp, &disp_connected);
break;
case DNS_DISPATCHSTATE_CONNECTING:
if (ISC_LIST_EMPTY(disp->pending)) {
/* Ignore dispatch with no responses */
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
break;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
}
Ignore TCP dispatches that have zero references The TCP dispatches are removed from the dispatchmgr->list in the dispatch_destroy() and there's a brief period of time where dns_dispatch_gettcp() can find a dispatch in connected state that's being destroyed. Set the dispatch state to DNS_DISPATCHSTATE_NONE in the TCP connection callback if there are no responses waiting, and ignore TCP dispatches with zero references in dns_dispatch_gettcp().
2022-12-20 06:11:26 +01:00
/* We found "a" dispatch, store it for later */
if (disp_fallback == NULL) {
dns_dispatch_attach(disp, &disp_fallback);
}
break;
case DNS_DISPATCHSTATE_CANCELED:
/* A canceled dispatch, skip it. */
break;
default:
UNREACHABLE();
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (disp_connected != NULL) {
break;
}
[master] allow shared TCP sockets when connecting 4041. [func] TCP sockets can now be shared while connecting. (This will be used to enable client-side support of pipelined queries.) [RT #38231]
2015-01-20 17:22:31 -08:00
}
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
rcu_read_unlock();
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
if (disp_connected != NULL) {
/* We found connected dispatch */
INSIST(disp_connected->handle != NULL);
[master] allow shared TCP sockets when connecting 4041. [func] TCP sockets can now be shared while connecting. (This will be used to enable client-side support of pipelined queries.) [RT #38231]
2015-01-20 17:22:31 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
*dispp = disp_connected;
disp_connected = NULL;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
result = ISC_R_SUCCESS;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (disp_fallback != NULL) {
dns_dispatch_detach(&disp_fallback);
}
} else if (disp_fallback != NULL) {
*dispp = disp_fallback;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
result = ISC_R_SUCCESS;
[master] allow shared TCP sockets when connecting 4041. [func] TCP sockets can now be shared while connecting. (This will be used to enable client-side support of pipelined queries.) [RT #38231]
2015-01-20 17:22:31 -08:00
}
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
return result;
[master] allow shared TCP sockets when connecting 4041. [func] TCP sockets can now be shared while connecting. (This will be used to enable client-side support of pipelined queries.) [RT #38231]
2015-01-20 17:22:31 -08:00
}
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
isc_result_t
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
dns_dispatch_createudp(dns_dispatchmgr_t *mgr, const isc_sockaddr_t *localaddr,
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
dns_dispatch_t **dispp) {
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
isc_result_t result;
2129. [func] Provide a pool of UDP sockets for queries to be made over. See use-queryport-pool, queryport-pool-ports and queryport-pool-updateinterval. [RT #16415]
2007-02-02 02:18:06 +00:00
dns_dispatch_t *disp = NULL;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
REQUIRE(VALID_DISPATCHMGR(mgr));
REQUIRE(localaddr != NULL);
REQUIRE(dispp != NULL && *dispp == NULL);
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
result = dispatch_createudp(mgr, localaddr, isc_tid(), &disp);
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
if (result == ISC_R_SUCCESS) {
*dispp = disp;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
}
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
return result;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
}
static isc_result_t
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
dispatch_createudp(dns_dispatchmgr_t *mgr, const isc_sockaddr_t *localaddr,
Convert the isc/tid.h to use own signed integer isc_tid_t type Change the internal type used for isc_tid unit to isc_tid_t to hide the specific integer type being used for the 'tid'. Internally, the signed integer type is being used. This allows us to have negatively indexed arrays that works both for threads with assigned tid and the threads with unassigned tid. This should be used only in specific situations.
2025-06-04 17:54:20 +02:00
isc_tid_t tid, dns_dispatch_t **dispp) {
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
isc_result_t result = ISC_R_SUCCESS;
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
dns_dispatch_t *disp = NULL;
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
isc_sockaddr_t sa_any;
style: one-line statement braces, line length, etc
2018-11-08 19:34:51 -08:00
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
/*
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
* Check whether this address/port is available locally.
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
*/
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
isc_sockaddr_anyofpf(&sa_any, isc_sockaddr_pf(localaddr));
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
if (!isc_sockaddr_eqaddr(&sa_any, localaddr)) {
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
result = isc_nm_checkaddr(localaddr, isc_socktype_udp);
style: one-line statement braces, line length, etc
2018-11-08 19:34:51 -08:00
if (result != ISC_R_SUCCESS) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
return result;
style: one-line statement braces, line length, etc
2018-11-08 19:34:51 -08:00
}
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
}
3137. [func] Improve hardware scalability by allowing multiple worker threads to process incoming UDP packets. This can significantly increase query throughput on some systems. [RT #22992]
2011-07-28 04:04:37 +00:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dispatch_allocate(mgr, isc_socktype_udp, tid, &disp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
if (isc_log_wouldlog(90)) {
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
char addrbuf[ISC_SOCKADDR_FORMATSIZE];
2491. [func] Attempt to re-use a local port if we are already using the port. [RT #18548]
2008-11-12 23:10:57 +00:00
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
isc_sockaddr_format(localaddr, addrbuf,
ISC_SOCKADDR_FORMATSIZE);
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
mgr_log(mgr, ISC_LOG_DEBUG(90),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"dispatch_createudp: created UDP dispatch %p for %s",
disp, addrbuf);
1442. [func] New fuctions for manipulating port lists: dns_portlist_create(), dns_portlist_add(), dns_portlist_remove(), dns_portlist_match(), dns_portlist_attach() and dns_portlist_detach(). 1441. [func] It is now possible to tell dig to bind to a specific source port. 1440. [func] It is now possible to tell named to avoid using certian source ports (avoid-v4-udp-ports, avoid-v6-udp-ports). developer: marka reviewer: explorer
2003-02-26 05:05:16 +00:00
}
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
when code is duplicated (which it shoudn't be), it should at least be duplicated consistently
2001-01-07 22:02:48 +00:00
disp->local = *localaddr;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
/*
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
* Don't append it to the dispatcher list, we don't care about UDP, only
* TCP should be searched
*
* ISC_LIST_APPEND(mgr->list, disp, link);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
*/
*dispp = disp;
3137. [func] Improve hardware scalability by allowing multiple worker threads to process incoming UDP packets. This can significantly increase query throughput on some systems. [RT #22992]
2011-07-28 04:04:37 +00:00
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
return result;
checkpoint
1999-06-16 01:32:31 +00:00
}
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
static void
dispatch_destroy_rcu(struct rcu_head *rcu_head) {
dns_dispatch_t *disp = caa_container_of(rcu_head, dns_dispatch_t,
rcu_head);
isc_mem_putanddetach(&disp->mctx, disp, sizeof(*disp));
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
static void
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
dispatch_destroy(dns_dispatch_t *disp) {
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dns_dispatchmgr_t *mgr = disp->mgr;
Convert the isc/tid.h to use own signed integer isc_tid_t type Change the internal type used for isc_tid unit to isc_tid_t to hide the specific integer type being used for the 'tid'. Internally, the signed integer type is being used. This allows us to have negatively indexed arrays that works both for threads with assigned tid and the threads with unassigned tid. This should be used only in specific situations.
2025-06-04 17:54:20 +02:00
isc_tid_t tid = isc_tid();
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
disp->magic = 0;
Add option to mark TCP dispatch as unshared The current dispatch code could reuse the TCP connection when dns_dispatch_gettcp() would be used first. This is problematic as the dns_resolver doesn't use TCP connection sharing, but dns_request could get the TCP stream that was created outside of the dns_request. Add new DNS_DISPATCHOPT_UNSHARED option to dns_dispatch_createtcp() that would prevent the TCP stream to be reused. Use that option in the dns_resolver call to dns_dispatch_createtcp() to prevent dns_request from reusing the TCP connections created by dns_resolver. Additionally, the dns_xfrin unit added TCP connection sharing for incoming transfers. While interleaving *xfr streams on a TCP connection should work this should be a deliberate change and be property of the server that can be controlled. Additionally some level of parallel TCP streams is desirable. Revert to the old behaviour by removing the dns_dispatch_gettcp() calls from dns_xfrin and use the new option to prevent from sharing the transfer streams with dns_request.
2023-10-20 08:14:27 +02:00
if (disp->socktype == isc_socktype_tcp &&
(disp->options & DNS_DISPATCHOPT_UNSHARED) == 0)
{
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
(void)cds_lfht_del(mgr->tcps[tid], &disp->ht_node);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(disp->requests == 0);
INSIST(ISC_LIST_EMPTY(disp->pending));
INSIST(ISC_LIST_EMPTY(disp->active));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(90), "destroying dispatch %p", disp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (disp->handle) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(90),
"detaching TCP handle %p from %p", disp->handle,
&disp->handle);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_nmhandle_detach(&disp->handle);
}
Fix TCP dispatches and transport Dispatch needs to know the transport that is being used over the TCP connection to correctly allow for it to be reused. Add a transport parameter to dns_dispatch_createtcp and dns_dispatch_gettcp and use it when selecting a TCP socket for reuse.
2024-10-15 16:09:48 +11:00
if (disp->transport != NULL) {
dns_transport_detach(&disp->transport);
}
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
dns_dispatchmgr_detach(&disp->mgr);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
call_rcu(&disp->rcu_head, dispatch_destroy_rcu);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
ISC_REFCOUNT_TRACE_IMPL(dns_dispatch, dispatch_destroy);
#else
ISC_REFCOUNT_IMPL(dns_dispatch, dispatch_destroy);
#endif
checkpoint
1999-06-16 01:32:31 +00:00
fix many bugs.
1999-07-22 01:34:31 +00:00
isc_result_t
Add option to mark TCP dispatch as unshared The current dispatch code could reuse the TCP connection when dns_dispatch_gettcp() would be used first. This is problematic as the dns_resolver doesn't use TCP connection sharing, but dns_request could get the TCP stream that was created outside of the dns_request. Add new DNS_DISPATCHOPT_UNSHARED option to dns_dispatch_createtcp() that would prevent the TCP stream to be reused. Use that option in the dns_resolver call to dns_dispatch_createtcp() to prevent dns_request from reusing the TCP connections created by dns_resolver. Additionally, the dns_xfrin unit added TCP connection sharing for incoming transfers. While interleaving *xfr streams on a TCP connection should work this should be a deliberate change and be property of the server that can be controlled. Additionally some level of parallel TCP streams is desirable. Revert to the old behaviour by removing the dns_dispatch_gettcp() calls from dns_xfrin and use the new option to prevent from sharing the transfer streams with dns_request.
2023-10-20 08:14:27 +02:00
dns_dispatch_add(dns_dispatch_t *disp, isc_loop_t *loop,
Separate the connect and the read timeouts in dispatch The network manager layer has two different timers with their own timeout values for TCP connections: connect timeout and read timeout. Separate the connect and the read TCP timeouts in the dispatch module too.
2024-10-30 09:23:33 +00:00
dns_dispatchopt_t options, unsigned int connect_timeout,
unsigned int timeout, const isc_sockaddr_t *dest,
dns_transport_t *transport, isc_tlsctx_cache_t *tlsctx_cache,
dispatch_cb_t connected, dispatch_cb_t sent,
dispatch_cb_t response, void *arg, dns_messageid_t *idp,
dns_dispentry_t **respp) {
checkpoint
1999-06-18 02:01:42 +00:00
REQUIRE(VALID_DISPATCH(disp));
REQUIRE(dest != NULL);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(respp != NULL && *respp == NULL);
checkpoint
1999-06-18 02:01:42 +00:00
REQUIRE(idp != NULL);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
REQUIRE(disp->socktype == isc_socktype_tcp ||
disp->socktype == isc_socktype_udp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(connected != NULL);
REQUIRE(response != NULL);
REQUIRE(sent != NULL);
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
REQUIRE(loop != NULL);
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix TCP dispatches and transport Dispatch needs to know the transport that is being used over the TCP connection to correctly allow for it to be reused. Add a transport parameter to dns_dispatch_createtcp and dns_dispatch_gettcp and use it when selecting a TCP socket for reuse.
2024-10-15 16:09:48 +11:00
REQUIRE(disp->transport == transport);
checkpoint
1999-06-18 02:01:42 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (disp->state == DNS_DISPATCHSTATE_CANCELED) {
return ISC_R_CANCELED;
checkpoint
1999-07-09 02:47:55 +00:00
}
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
in_port_t localport = isc_sockaddr_getport(&disp->local);
dns_dispentry_t *resp = isc_mem_get(disp->mctx, sizeof(*resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
*resp = (dns_dispentry_t){
Separate the connect and the read timeouts in dispatch The network manager layer has two different timers with their own timeout values for TCP connections: connect timeout and read timeout. Separate the connect and the read TCP timeouts in the dispatch module too.
2024-10-30 09:23:33 +00:00
.connect_timeout = connect_timeout,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
.timeout = timeout,
Don't ignore the local port number in dns_dispatch_add() for TCP The dns_dispatch_add() function registers the 'resp' entry in 'disp->mgr->qids' hash table with 'resp->port' being 0, but in tcp_recv_success(), when looking up an entry in the hash table after a successfully received data the port is used, so if the local port was set (i.e. it was not 0) it fails to find the entry and results in an unexpected error. Set the 'resp->port' to the given local port value extracted from 'disp->local'.
2024-10-01 14:16:47 +00:00
.port = localport,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
.peer = *dest,
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
.loop = loop,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
.connected = connected,
.sent = sent,
.response = response,
.arg = arg,
.alink = ISC_LINK_INITIALIZER,
.plink = ISC_LINK_INITIALIZER,
.rlink = ISC_LINK_INITIALIZER,
.magic = RESPONSE_MAGIC,
};
#if DNS_DISPATCH_TRACE
fprintf(stderr, "dns_dispentry__init:%s:%s:%d:%p->references = 1\n",
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
__func__, __FILE__, __LINE__, resp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#endif
isc_refcount_init(&resp->references, 1); /* DISPENTRY000 */
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
if (disp->socktype == isc_socktype_udp) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
isc_result_t result = setup_socket(disp, resp, dest,
&localport);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
if (result != ISC_R_SUCCESS) {
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
isc_mem_put(disp->mctx, resp, sizeof(*resp));
2541. [bug] Conditionally update dispatch manager statistics. [RT #19247]
2009-01-31 00:10:24 +00:00
inc_stats(disp->mgr, dns_resstatscounter_dispsockfail);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
return result;
}
}
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
isc_result_t result = ISC_R_NOMORE;
size_t i = 0;
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_lock();
silence compiler warning
2014-01-09 15:57:59 +11:00
do {
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
/*
* Try somewhat hard to find a unique ID. Start with
* a random number unless DNS_DISPATCHOPT_FIXEDID is set,
* in which case we start with the ID passed in via *idp.
*/
resp->id = ((options & DNS_DISPATCHOPT_FIXEDID) != 0)
? *idp
: (dns_messageid_t)isc_random16();
struct cds_lfht_node *node =
cds_lfht_add_unique(disp->mgr->qids, qid_hash(resp),
qid_match, resp, &resp->ht_node);
if (node != &resp->ht_node) {
if ((options & DNS_DISPATCHOPT_FIXEDID) != 0) {
/*
* When using fixed ID, we either must
* use it or fail
*/
goto fail;
}
} else {
result = ISC_R_SUCCESS;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
break;
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
} while (i++ < QID_MAX_TRIES);
fail:
if (result != ISC_R_SUCCESS) {
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
isc_mem_put(disp->mctx, resp, sizeof(*resp));
Add a missing rcu_read_unlock() call on exit path An exit path in the dns_dispatch_add() function fails to get out of the RCU critical section when returning early. Add the missing rcu_read_unlock() call.
2024-09-27 12:48:22 +00:00
rcu_read_unlock();
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
return result;
checkpoint
1999-06-18 02:01:42 +00:00
}
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
isc_mem_attach(disp->mctx, &resp->mctx);
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
if (transport != NULL) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_transport_attach(transport, &resp->transport);
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
}
if (tlsctx_cache != NULL) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_tlsctx_cache_attach(tlsctx_cache, &resp->tlsctx_cache);
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_attach(disp, &resp->disp); /* DISPATCH001 */
checkpoint
1999-06-18 02:01:42 +00:00
snapshot
1999-07-09 00:51:08 +00:00
disp->requests++;
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
inc_stats(disp->mgr, (disp->socktype == isc_socktype_udp)
? dns_resstatscounter_disprequdp
: dns_resstatscounter_dispreqtcp);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_unlock();
checkpoint
1999-06-18 02:01:42 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
*idp = resp->id;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
*respp = resp;
checkpoint
1999-06-18 02:01:42 +00:00
s/DNS_R_/ISC_R_/ change for some codes.
2000-04-06 22:03:35 +00:00
return ISC_R_SUCCESS;
checkpoint
1999-06-16 01:32:31 +00:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_t
dns_dispatch_getnext(dns_dispentry_t *resp) {
REQUIRE(VALID_RESPONSE(resp));
REQUIRE(VALID_DISPATCH(resp->disp));
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
isc_result_t result = ISC_R_SUCCESS;
Remove dispatch timeout INT16_MAX limitation In some places there was a limitation of the maximum timeout value of INT16_MAX, which is only about 32 seconds. Refactor the code to remove the limitation.
2024-10-30 13:31:34 +00:00
int64_t timeout = 0;
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "getnext for QID %d", resp->id);
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
Modify dns_dispatch API to accept zero timeout The dns_dispatch_add() has timeout parameter that could not be 0 (for not timeout). Modify the dns_dispatch implementation to accept a zero timeout for cases where the timeouts are undesirable because they are managed externally.
2024-09-20 15:13:09 +02:00
if (resp->timeout > 0) {
isc_time_t now = isc_loop_now(resp->loop);
timeout = resp->timeout - dispentry_runtime(resp, &now);
if (timeout <= 0) {
return ISC_R_TIMEDOUT;
}
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
}
Remove dispatch timeout INT16_MAX limitation In some places there was a limitation of the maximum timeout value of INT16_MAX, which is only about 32 seconds. Refactor the code to remove the limitation.
2024-10-30 13:31:34 +00:00
INSIST(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (disp->socktype) {
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
case isc_socktype_udp:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
udp_dispatch_getnext(resp, timeout);
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
break;
case isc_socktype_tcp:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_dispatch_getnext(disp, resp, timeout);
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
break;
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
}
4408. [func] Continue waiting for expected response when we the response we get does not match the request. [RT #41026]
2016-07-11 13:36:16 +10:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
return result;
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
/*
* NOTE: Must be RCU read locked!
*/
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
udp_dispentry_cancel(dns_dispentry_t *resp, isc_result_t result) {
4408. [func] Continue waiting for expected response when we the response we get does not match the request. [RT #41026]
2016-07-11 13:36:16 +10:00
REQUIRE(VALID_RESPONSE(resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(VALID_DISPATCH(resp->disp));
REQUIRE(VALID_DISPATCHMGR(resp->disp->mgr));
4408. [func] Continue waiting for expected response when we the response we get does not match the request. [RT #41026]
2016-07-11 13:36:16 +10:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
bool respond = false;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"canceling response: %s, %s/%s (%s/%s), "
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
"requests %" PRIuFAST32,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(result), state2str(resp->state),
resp->reading ? "reading" : "not reading",
state2str(disp->state),
disp->reading ? "reading" : "not reading",
disp->requests);
4408. [func] Continue waiting for expected response when we the response we get does not match the request. [RT #41026]
2016-07-11 13:36:16 +10:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (ISC_LINK_LINKED(resp, alink)) {
ISC_LIST_UNLINK(disp->active, resp, alink);
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (resp->state) {
case DNS_DISPATCHSTATE_NONE:
break;
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTING:
break;
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTED:
if (resp->reading) {
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
respond = true;
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
"canceling read on %p", resp->handle);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_nm_cancelread(resp->handle);
}
break;
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CANCELED:
goto unlock;
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
default:
UNREACHABLE();
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dec_stats(disp->mgr, dns_resstatscounter_disprequdp);
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
(void)cds_lfht_del(disp->mgr->qids, &resp->ht_node);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->state = DNS_DISPATCHSTATE_CANCELED;
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
unlock:
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
if (respond) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "read callback: %s",
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(result));
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
resp->response(result, NULL, resp->arg);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
/*
* NOTE: Must be RCU read locked!
*/
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
tcp_dispentry_cancel(dns_dispentry_t *resp, isc_result_t result) {
REQUIRE(VALID_RESPONSE(resp));
REQUIRE(VALID_DISPATCH(resp->disp));
REQUIRE(VALID_DISPATCHMGR(resp->disp->mgr));
checkpoint
1999-06-18 02:01:42 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
dns_displist_t resps = ISC_LIST_INITIALIZER;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"canceling response: %s, %s/%s (%s/%s), "
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
"requests %" PRIuFAST32,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(result), state2str(resp->state),
resp->reading ? "reading" : "not reading",
state2str(disp->state),
disp->reading ? "reading" : "not reading",
disp->requests);
switch (resp->state) {
case DNS_DISPATCHSTATE_NONE:
break;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTING:
break;
checkpoint
1999-06-18 02:01:42 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTED:
if (resp->reading) {
tcp_recv_add(&resps, resp, ISC_R_CANCELED);
}
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(!ISC_LINK_LINKED(resp, alink));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (ISC_LIST_EMPTY(disp->active)) {
INSIST(disp->handle != NULL);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DISPATCH_TCP_KEEPALIVE
/*
* This is an experimental code that keeps the TCP
* connection open for 1 second before it is finally
* closed. By keeping the TCP connection open, it can
* be reused by dns_request that uses
* dns_dispatch_gettcp() to join existing TCP
* connections.
*
* It is disabled for now, because it changes the
* behaviour, but I am keeping the code here for future
* reference when we improve the dns_dispatch to reuse
* the TCP connections also in the resolver.
*
* The TCP connection reuse should be seamless and not
* require any extra handling on the client side though.
*/
isc_nmhandle_cleartimeout(disp->handle);
isc_nmhandle_settimeout(disp->handle, 1000);
if (!disp->reading) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"final 1 second timeout on %p",
disp->handle);
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
tcp_startrecv(disp, NULL);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
#else
if (disp->reading) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"canceling read on %p",
disp->handle);
isc_nm_cancelread(disp->handle);
}
#endif
}
break;
469. [bug] "query-source address * port 53;" now works.
2000-09-18 04:50:05 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CANCELED:
goto unlock;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
default:
UNREACHABLE();
}
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dec_stats(disp->mgr, dns_resstatscounter_dispreqtcp);
checkpoint
1999-06-18 02:01:42 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
(void)cds_lfht_del(disp->mgr->qids, &resp->ht_node);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->state = DNS_DISPATCHSTATE_CANCELED;
unlock:
snapshot
1999-07-09 00:51:08 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/*
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
* NOTE: Calling the response callback directly from here should be done
* asynchronously, as the dns_dispatch_done() is usually called directly
* from the response callback, so there's a slight chance that the call
* stack will get higher here, but it's mitigated by the ".reading"
* flag, so we don't ever go into a loop.
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
*/
tcp_recv_processall(&resps, NULL);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
}
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_cancel(dns_dispentry_t *resp, isc_result_t result) {
REQUIRE(VALID_RESPONSE(resp));
REQUIRE(VALID_DISPATCH(resp->disp));
dns_dispatch_t *disp = resp->disp;
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_lock();
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
switch (disp->socktype) {
case isc_socktype_udp:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
udp_dispentry_cancel(resp, result);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
break;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
case isc_socktype_tcp:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_dispentry_cancel(resp, result);
break;
default:
UNREACHABLE();
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_unlock();
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
REQUIRE should not have side effects it's a style violation to have REQUIRE or INSIST contain code that must run for the server to work. this was being done with some atomic_compare_exchange calls. these have been cleaned up. uses of atomic_compare_exchange in assertions have been replaced with a new macro atomic_compare_exchange_enforced, which uses RUNTIME_CHECK to ensure that the exchange was successful.
2021-10-21 02:28:48 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
void
dns_dispatch_done(dns_dispentry_t **respp) {
REQUIRE(VALID_RESPONSE(*respp));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_t *resp = *respp;
*respp = NULL;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_cancel(resp, ISC_R_CANCELED);
dns_dispentry_detach(&resp); /* DISPENTRY000 */
}
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
udp_startrecv(isc_nmhandle_t *handle, dns_dispentry_t *resp) {
REQUIRE(VALID_RESPONSE(resp));
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "attaching handle %p to %p",
handle, &resp->handle);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_nmhandle_attach(handle, &resp->handle);
dns_dispentry_ref(resp); /* DISPENTRY003 */
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "reading");
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_nm_read(resp->handle, udp_recv, resp);
resp->reading = true;
}
static void
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
tcp_startrecv(dns_dispatch_t *disp, dns_dispentry_t *resp) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(VALID_DISPATCH(disp));
REQUIRE(disp->socktype == isc_socktype_tcp);
dns_dispatch_ref(disp); /* DISPATCH002 */
if (resp != NULL) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "reading from %p",
disp->handle);
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
INSIST(!isc_time_isepoch(&resp->start));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
} else {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(90),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"TCP reading without response from %p",
disp->handle);
Move isc_socket_connect() calls into dispatch dns_dispatch_connect() connects a dispatch socket (for TCP) or a dispatch entry socket (for UDP). This is the next step in moving all uses of the isc_socket code into the dispatch module. This API is temporary; it needs to be cleaned up further so that it can be called the same way for both TCP and UDP.
2020-12-19 01:34:41 -08:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_nm_read(disp->handle, tcp_recv, disp);
disp->reading = true;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
}
Always call the TCP dispatch connected callbacks asynchronously The TCP dispatch connected callbacks could be called synchronously which in turn could destroy xfrin before we return from dns_xfrin_create(). Delay the calling the callback called from tcp_dispatch_connect() by calling it always asynchronously.
2024-03-04 12:58:56 +01:00
static void
resp_connected(void *arg) {
dns_dispentry_t *resp = arg;
dispentry_log(resp, ISC_LOG_DEBUG(90), "connect callback: %s",
isc_result_totext(resp->result));
resp->connected(resp->result, NULL, resp->arg);
dns_dispentry_detach(&resp); /* DISPENTRY005 */
}
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
static void
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
tcp_connected(isc_nmhandle_t *handle, isc_result_t eresult, void *arg) {
dns_dispatch_t *disp = (dns_dispatch_t *)arg;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_displist_t resps = ISC_LIST_INITIALIZER;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Remove logging context (isc_log_t) from the public namespace Now that the logging uses single global context, remove the isc_log_t from the public namespace.
2024-08-13 18:20:26 +02:00
if (isc_log_wouldlog(90)) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
char localbuf[ISC_SOCKADDR_FORMATSIZE];
char peerbuf[ISC_SOCKADDR_FORMATSIZE];
if (handle != NULL) {
isc_sockaddr_t local = isc_nmhandle_localaddr(handle);
isc_sockaddr_t peer = isc_nmhandle_peeraddr(handle);
isc_sockaddr_format(&local, localbuf,
ISC_SOCKADDR_FORMATSIZE);
isc_sockaddr_format(&peer, peerbuf,
ISC_SOCKADDR_FORMATSIZE);
} else {
isc_sockaddr_format(&disp->local, localbuf,
ISC_SOCKADDR_FORMATSIZE);
isc_sockaddr_format(&disp->peer, peerbuf,
ISC_SOCKADDR_FORMATSIZE);
}
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispatch_log(disp, ISC_LOG_DEBUG(90),
"connected from %s to %s: %s", localbuf, peerbuf,
isc_result_totext(eresult));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix assignment vs comparison typo in tcp_connected() In tcp_connected() a typo has turned a DbC check into an assignment breaking the state machine and making the dns_dispatch_gettcp() try to attach to dispatch in process of destruction.
2022-12-19 18:17:46 +01:00
INSIST(disp->state == DNS_DISPATCHSTATE_CONNECTING);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
/*
* If there are pending responses, call the connect
* callbacks for all of them.
*/
make all ISC_LIST_FOREACH calls safe previously, ISC_LIST_FOREACH and ISC_LIST_FOREACH_SAFE were two separate macros, with the _SAFE version allowing entries to be unlinked during the loop. ISC_LIST_FOREACH is now also safe, and the separate _SAFE macro has been removed. similarly, the ISC_LIST_FOREACH_REV macro is now safe, and ISC_LIST_FOREACH_REV_SAFE has also been removed.
2025-05-23 13:02:22 -07:00
ISC_LIST_FOREACH(disp->pending, resp, plink) {
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
ISC_LIST_UNLINK(disp->pending, resp, plink);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
ISC_LIST_APPEND(resps, resp, rlink);
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
resp->result = eresult;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
if (resp->state == DNS_DISPATCHSTATE_CANCELED) {
resp->result = ISC_R_CANCELED;
} else if (eresult == ISC_R_SUCCESS) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->state = DNS_DISPATCHSTATE_CONNECTED;
ISC_LIST_APPEND(disp->active, resp, alink);
resp->reading = true;
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "start reading");
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
} else {
resp->state = DNS_DISPATCHSTATE_NONE;
}
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
}
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
Use a suitable response in tcp_connected() when initiating a read When 'ISC_R_TIMEDOUT' is received in 'tcp_recv()', it times out the oldest response in the active responses queue, and only after that it checks whether other active responses have also timed out. So when setting a timeout value for a read operation after a successful connection, it makes sense to take the timeout value from the oldest response in the active queue too, because, theoretically, the responses can have different timeout values, e.g. when the TCP dispatch is shared. Currently 'resp' is always NULL. Previously when connect and read timeouts were not separated in dispatch this affected only logging, but now since we are setting a new timeout after a successful connection, we need to choose a suitable response from the active queue.
2024-12-19 14:22:54 +00:00
/* Take the oldest active response. */
switch to ISC_LIST_FOREACH everywhere the pattern `for (x = ISC_LIST_HEAD(...); x != NULL; ISC_LIST_NEXT(...)` has been changed to `ISC_LIST_FOREACH` throughout BIND, except in a few cases where the change would be excessively complex. in most cases this was a straightforward change. in some places, however, the list element variable was referenced after the loop ended, and the code was refactored to avoid this necessity. also, because `ISC_LIST_FOREACH` uses typeof(list.head) to declare the list elements, compilation failures can occur if the list object has a `const` qualifier. some `const` qualifiers have been removed from function parameters to avoid this problem, and where that was not possible, `UNCONST` was used.
2025-03-20 22:25:56 -07:00
dns_dispentry_t *oldest = ISC_LIST_HEAD(disp->active);
if (oldest == NULL) {
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
/* All responses have been canceled */
disp->state = DNS_DISPATCHSTATE_CANCELED;
} else if (eresult == ISC_R_SUCCESS) {
disp->state = DNS_DISPATCHSTATE_CONNECTED;
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
isc_nmhandle_attach(handle, &disp->handle);
Use a suitable response in tcp_connected() when initiating a read When 'ISC_R_TIMEDOUT' is received in 'tcp_recv()', it times out the oldest response in the active responses queue, and only after that it checks whether other active responses have also timed out. So when setting a timeout value for a read operation after a successful connection, it makes sense to take the timeout value from the oldest response in the active queue too, because, theoretically, the responses can have different timeout values, e.g. when the TCP dispatch is shared. Currently 'resp' is always NULL. Previously when connect and read timeouts were not separated in dispatch this affected only logging, but now since we are setting a new timeout after a successful connection, we need to choose a suitable response from the active queue.
2024-12-19 14:22:54 +00:00
isc_nmhandle_cleartimeout(disp->handle);
switch to ISC_LIST_FOREACH everywhere the pattern `for (x = ISC_LIST_HEAD(...); x != NULL; ISC_LIST_NEXT(...)` has been changed to `ISC_LIST_FOREACH` throughout BIND, except in a few cases where the change would be excessively complex. in most cases this was a straightforward change. in some places, however, the list element variable was referenced after the loop ended, and the code was refactored to avoid this necessity. also, because `ISC_LIST_FOREACH` uses typeof(list.head) to declare the list elements, compilation failures can occur if the list object has a `const` qualifier. some `const` qualifiers have been removed from function parameters to avoid this problem, and where that was not possible, `UNCONST` was used.
2025-03-20 22:25:56 -07:00
if (oldest->timeout != 0) {
isc_nmhandle_settimeout(disp->handle, oldest->timeout);
Separate the connect and the read timeouts in dispatch The network manager layer has two different timers with their own timeout values for TCP connections: connect timeout and read timeout. Separate the connect and the read TCP timeouts in the dispatch module too.
2024-10-30 09:23:33 +00:00
}
switch to ISC_LIST_FOREACH everywhere the pattern `for (x = ISC_LIST_HEAD(...); x != NULL; ISC_LIST_NEXT(...)` has been changed to `ISC_LIST_FOREACH` throughout BIND, except in a few cases where the change would be excessively complex. in most cases this was a straightforward change. in some places, however, the list element variable was referenced after the loop ended, and the code was refactored to avoid this necessity. also, because `ISC_LIST_FOREACH` uses typeof(list.head) to declare the list elements, compilation failures can occur if the list object has a `const` qualifier. some `const` qualifiers have been removed from function parameters to avoid this problem, and where that was not possible, `UNCONST` was used.
2025-03-20 22:25:56 -07:00
tcp_startrecv(disp, oldest);
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
} else {
disp->state = DNS_DISPATCHSTATE_NONE;
}
make all ISC_LIST_FOREACH calls safe previously, ISC_LIST_FOREACH and ISC_LIST_FOREACH_SAFE were two separate macros, with the _SAFE version allowing entries to be unlinked during the loop. ISC_LIST_FOREACH is now also safe, and the separate _SAFE macro has been removed. similarly, the ISC_LIST_FOREACH_REV macro is now safe, and ISC_LIST_FOREACH_REV_SAFE has also been removed.
2025-05-23 13:02:22 -07:00
ISC_LIST_FOREACH(resps, resp, rlink) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
ISC_LIST_UNLINK(resps, resp, rlink);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Always call the TCP dispatch connected callbacks asynchronously The TCP dispatch connected callbacks could be called synchronously which in turn could destroy xfrin before we return from dns_xfrin_create(). Delay the calling the callback called from tcp_dispatch_connect() by calling it always asynchronously.
2024-03-04 12:58:56 +01:00
resp_connected(resp);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
}
Move isc_socket_connect() calls into dispatch dns_dispatch_connect() connects a dispatch socket (for TCP) or a dispatch entry socket (for UDP). This is the next step in moving all uses of the isc_socket code into the dispatch module. This API is temporary; it needs to be cleaned up further so that it can be called the same way for both TCP and UDP.
2020-12-19 01:34:41 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_detach(&disp); /* DISPATCH003 */
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
}
static void
udp_connected(isc_nmhandle_t *handle, isc_result_t eresult, void *arg) {
dns_dispentry_t *resp = (dns_dispentry_t *)arg;
dns_dispatch_t *disp = resp->disp;
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "connected: %s",
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(eresult));
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (resp->state) {
case DNS_DISPATCHSTATE_CANCELED:
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
eresult = ISC_R_CANCELED;
ISC_LIST_UNLINK(disp->pending, resp, plink);
goto unlock;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTING:
ISC_LIST_UNLINK(disp->pending, resp, plink);
break;
default:
UNREACHABLE();
}
switch (eresult) {
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
case ISC_R_CANCELED:
break;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case ISC_R_SUCCESS:
resp->state = DNS_DISPATCHSTATE_CONNECTED;
udp_startrecv(handle, resp);
break;
Honour the source-port when retrying in dns_dispatch When retrying in the DNS dispatch, the local port would be forgotten on ISC_R_ADDRINUSE, keep the configured source-port even when retrying. Additionally, treat ISC_R_NOPERM same as ISC_R_ADDRINUSE. Closes: #3986
2023-04-05 15:28:32 +02:00
case ISC_R_NOPERM:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case ISC_R_ADDRINUSE: {
Honour the source-port when retrying in dns_dispatch When retrying in the DNS dispatch, the local port would be forgotten on ISC_R_ADDRINUSE, keep the configured source-port even when retrying. Additionally, treat ISC_R_NOPERM same as ISC_R_ADDRINUSE. Closes: #3986
2023-04-05 15:28:32 +02:00
in_port_t localport = isc_sockaddr_getport(&disp->local);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
isc_result_t result;
/* probably a port collision; try a different one */
result = setup_socket(disp, resp, &resp->peer, &localport);
if (result == ISC_R_SUCCESS) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
udp_dispatch_connect(disp, resp);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto detach;
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->state = DNS_DISPATCHSTATE_NONE;
break;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
default:
resp->state = DNS_DISPATCHSTATE_NONE;
break;
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
}
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
unlock:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "connect callback: %s",
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(eresult));
resp->connected(eresult, NULL, resp->arg);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
detach:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_detach(&resp); /* DISPENTRY004 */
Move isc_socket_connect() calls into dispatch dns_dispatch_connect() connects a dispatch socket (for TCP) or a dispatch entry socket (for UDP). This is the next step in moving all uses of the isc_socket code into the dispatch module. This API is temporary; it needs to be cleaned up further so that it can be called the same way for both TCP and UDP.
2020-12-19 01:34:41 -08:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
udp_dispatch_connect(dns_dispatch_t *disp, dns_dispentry_t *resp) {
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->state = DNS_DISPATCHSTATE_CONNECTING;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp->start = isc_loop_now(resp->loop);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_ref(resp); /* DISPENTRY004 */
ISC_LIST_APPEND(disp->pending, resp, plink);
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
Refactor the network manager to be a singleton There is only a single network manager running on top of the loop manager (except for tests). Refactor the network manager to be a singleton (a single instance) and change the unit tests, so that the shorter read timeouts apply only to a specific handle, not the whole extra 'connect_nm' network manager instance.
2025-07-14 17:12:35 +02:00
isc_nm_udpconnect(&resp->local, &resp->peer, udp_connected, resp,
resp->timeout);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
Dispatch: carefully check if the server name for SNI is a hostname Previously the code would not check if the string intended to be used for SNI is a hostname.
2025-03-18 12:52:18 +02:00
static inline const char *
get_tls_sni_hostname(dns_dispentry_t *resp) {
char *hostname = NULL;
if (resp->transport != NULL) {
hostname = dns_transport_get_remote_hostname(resp->transport);
}
if (hostname == NULL) {
return NULL;
}
if (isc_tls_valid_sni_hostname(hostname)) {
return hostname;
}
return NULL;
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static isc_result_t
tcp_dispatch_connect(dns_dispatch_t *disp, dns_dispentry_t *resp) {
dns_transport_type_t transport_type = DNS_TRANSPORT_TCP;
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
isc_tlsctx_t *tlsctx = NULL;
isc_tlsctx_client_session_cache_t *sess_cache = NULL;
Move isc_socket_sendto2() calls into dispatch We now use dns_dispatch_send() for this purpose.
2021-01-04 23:03:50 -08:00
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
if (resp->transport != NULL) {
transport_type = dns_transport_get_type(resp->transport);
}
if (transport_type == DNS_TRANSPORT_TLS) {
isc_result_t result;
result = dns_transport_get_tlsctx(
resp->transport, &resp->peer, resp->tlsctx_cache,
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
resp->mctx, &tlsctx, &sess_cache);
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
if (result != ISC_R_SUCCESS) {
return result;
}
INSIST(tlsctx != NULL);
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/* Check whether the dispatch is already connecting or connected. */
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (disp->state) {
case DNS_DISPATCHSTATE_NONE:
/* First connection, continue with connecting */
disp->state = DNS_DISPATCHSTATE_CONNECTING;
resp->state = DNS_DISPATCHSTATE_CONNECTING;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp->start = isc_loop_now(resp->loop);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_ref(resp); /* DISPENTRY005 */
ISC_LIST_APPEND(disp->pending, resp, plink);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
char localbuf[ISC_SOCKADDR_FORMATSIZE];
char peerbuf[ISC_SOCKADDR_FORMATSIZE];
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_sockaddr_format(&disp->local, localbuf,
ISC_SOCKADDR_FORMATSIZE);
isc_sockaddr_format(&disp->peer, peerbuf,
ISC_SOCKADDR_FORMATSIZE);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_ref(disp); /* DISPATCH003 */
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"connecting from %s to %s, timeout %u", localbuf,
Separate the connect and the read timeouts in dispatch The network manager layer has two different timers with their own timeout values for TCP connections: connect timeout and read timeout. Separate the connect and the read TCP timeouts in the dispatch module too.
2024-10-30 09:23:33 +00:00
peerbuf, resp->connect_timeout);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Dispatch: carefully check if the server name for SNI is a hostname Previously the code would not check if the string intended to be used for SNI is a hostname.
2025-03-18 12:52:18 +02:00
const char *hostname = get_tls_sni_hostname(resp);
BIND - enable TLS SNI support for outgoing TLS connections This commit ensures that BIND enables TLS SNI support for outgoing DoT connections (when possible) in order to improve compatibility with other DNS server software.
2024-12-17 16:24:29 +02:00
Refactor the network manager to be a singleton There is only a single network manager running on top of the loop manager (except for tests). Refactor the network manager to be a singleton (a single instance) and change the unit tests, so that the shorter read timeouts apply only to a specific handle, not the whole extra 'connect_nm' network manager instance.
2025-07-14 17:12:35 +02:00
isc_nm_streamdnsconnect(&disp->local, &disp->peer,
tcp_connected, disp,
Separate the connect and the read timeouts in dispatch The network manager layer has two different timers with their own timeout values for TCP connections: connect timeout and read timeout. Separate the connect and the read TCP timeouts in the dispatch module too.
2024-10-30 09:23:33 +00:00
resp->connect_timeout, tlsctx, hostname,
BIND - enable TLS SNI support for outgoing TLS connections This commit ensures that BIND enables TLS SNI support for outgoing DoT connections (when possible) in order to improve compatibility with other DNS server software.
2024-12-17 16:24:29 +02:00
sess_cache, ISC_NM_PROXY_NONE, NULL);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
break;
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTING:
/* Connection pending; add resp to the list */
resp->state = DNS_DISPATCHSTATE_CONNECTING;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp->start = isc_loop_now(resp->loop);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_ref(resp); /* DISPENTRY005 */
ISC_LIST_APPEND(disp->pending, resp, plink);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
break;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTED:
resp->state = DNS_DISPATCHSTATE_CONNECTED;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp->start = isc_loop_now(resp->loop);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/* Add the resp to the reading list */
ISC_LIST_APPEND(disp->active, resp, alink);
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90),
"already connected; attaching");
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->reading = true;
if (!disp->reading) {
/* Restart the reading */
Separate the connect and the read timeouts in dispatch The network manager layer has two different timers with their own timeout values for TCP connections: connect timeout and read timeout. Separate the connect and the read TCP timeouts in the dispatch module too.
2024-10-30 09:23:33 +00:00
isc_nmhandle_cleartimeout(disp->handle);
if (resp->timeout != 0) {
isc_nmhandle_settimeout(disp->handle,
resp->timeout);
}
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
tcp_startrecv(disp, resp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
Always call the TCP dispatch connected callbacks asynchronously The TCP dispatch connected callbacks could be called synchronously which in turn could destroy xfrin before we return from dns_xfrin_create(). Delay the calling the callback called from tcp_dispatch_connect() by calling it always asynchronously.
2024-03-04 12:58:56 +01:00
/* Already connected; call the connected cb asynchronously */
dns_dispentry_ref(resp); /* DISPENTRY005 */
isc_async_run(resp->loop, resp_connected, resp);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
break;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
default:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
UNREACHABLE();
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
}
Move isc_socket_sendto2() calls into dispatch We now use dns_dispatch_send() for this purpose.
2021-01-04 23:03:50 -08:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
return ISC_R_SUCCESS;
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_t
dns_dispatch_connect(dns_dispentry_t *resp) {
REQUIRE(VALID_RESPONSE(resp));
REQUIRE(VALID_DISPATCH(resp->disp));
dns_dispatch_t *disp = resp->disp;
switch (disp->socktype) {
case isc_socktype_tcp:
return tcp_dispatch_connect(disp, resp);
case isc_socktype_udp:
udp_dispatch_connect(disp, resp);
return ISC_R_SUCCESS;
default:
UNREACHABLE();
}
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
static void
send_done(isc_nmhandle_t *handle, isc_result_t result, void *cbarg) {
dns_dispentry_t *resp = (dns_dispentry_t *)cbarg;
REQUIRE(VALID_RESPONSE(resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
REQUIRE(VALID_DISPATCH(disp));
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "sent: %s",
isc_result_totext(result));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
resp->sent(result, NULL, resp->arg);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (result != ISC_R_SUCCESS) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_cancel(resp, result);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_detach(&resp); /* DISPENTRY007 */
isc_nmhandle_detach(&handle);
}
static void
tcp_dispatch_getnext(dns_dispatch_t *disp, dns_dispentry_t *resp,
Remove dispatch timeout INT16_MAX limitation In some places there was a limitation of the maximum timeout value of INT16_MAX, which is only about 32 seconds. Refactor the code to remove the limitation.
2024-10-30 13:31:34 +00:00
int64_t timeout) {
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "continue reading");
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
if (!resp->reading) {
ISC_LIST_APPEND(disp->active, resp, alink);
resp->reading = true;
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (disp->reading) {
return;
}
Remove dispatch timeout INT16_MAX limitation In some places there was a limitation of the maximum timeout value of INT16_MAX, which is only about 32 seconds. Refactor the code to remove the limitation.
2024-10-30 13:31:34 +00:00
if (timeout != 0) {
INSIST(timeout > 0 && timeout <= UINT32_MAX);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_nmhandle_settimeout(disp->handle, timeout);
}
dns_dispatch_ref(disp); /* DISPATCH002 */
isc_nm_read(disp->handle, tcp_recv, disp);
disp->reading = true;
}
static void
Remove dispatch timeout INT16_MAX limitation In some places there was a limitation of the maximum timeout value of INT16_MAX, which is only about 32 seconds. Refactor the code to remove the limitation.
2024-10-30 13:31:34 +00:00
udp_dispatch_getnext(dns_dispentry_t *resp, int64_t timeout) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (resp->reading) {
return;
}
Remove dispatch timeout INT16_MAX limitation In some places there was a limitation of the maximum timeout value of INT16_MAX, which is only about 32 seconds. Refactor the code to remove the limitation.
2024-10-30 13:31:34 +00:00
if (timeout != 0) {
INSIST(timeout > 0 && timeout <= UINT32_MAX);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_nmhandle_settimeout(resp->handle, timeout);
}
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "continue reading");
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_ref(resp); /* DISPENTRY003 */
isc_nm_read(resp->handle, udp_recv, resp);
resp->reading = true;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
void
Remove dispatch timeout INT16_MAX limitation In some places there was a limitation of the maximum timeout value of INT16_MAX, which is only about 32 seconds. Refactor the code to remove the limitation.
2024-10-30 13:31:34 +00:00
dns_dispatch_resume(dns_dispentry_t *resp, unsigned int timeout) {
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
REQUIRE(VALID_RESPONSE(resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(VALID_DISPATCH(resp->disp));
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "resume");
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (disp->socktype) {
case isc_socktype_udp: {
udp_dispatch_getnext(resp, timeout);
break;
}
case isc_socktype_tcp:
INSIST(disp->timedout > 0);
disp->timedout--;
tcp_dispatch_getnext(disp, resp, timeout);
break;
default:
UNREACHABLE();
}
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
}
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
void
remove nonfunctional DSCP implementation DSCP has not been fully working since the network manager was introduced in 9.16, and has been completely broken since 9.18. This seems to have caused very few difficulties for anyone, so we have now marked it as obsolete and removed the implementation. To ensure that old config files don't fail, the code to parse dscp key-value pairs is still present, but a warning is logged that the feature is obsolete and should not be used. Nothing is done with configured values, and there is no longer any range checking.
2023-01-05 22:18:55 -08:00
dns_dispatch_send(dns_dispentry_t *resp, isc_region_t *r) {
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
REQUIRE(VALID_RESPONSE(resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(VALID_DISPATCH(resp->disp));
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
isc_nmhandle_t *sendhandle = NULL;
Cleanup unused .localport member of dns_dispatch_t and some macros The .localport member of dns_dispatch_t structure was unused, clean it up. Cleanup unused and/or redundant macros.
2023-09-15 15:59:28 +02:00
dispentry_log(resp, ISC_LOG_DEBUG(90), "sending");
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (disp->socktype) {
case isc_socktype_udp:
isc_nmhandle_attach(resp->handle, &sendhandle);
break;
case isc_socktype_tcp:
isc_nmhandle_attach(disp->handle, &sendhandle);
break;
default:
UNREACHABLE();
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_ref(resp); /* DISPENTRY007 */
isc_nm_send(sendhandle, r, send_done, resp);
Move isc_socket_sendto2() calls into dispatch We now use dns_dispatch_send() for this purpose.
2021-01-04 23:03:50 -08:00
}
add dns_dispatch_getlocaladdress().
2001-03-13 05:48:41 +00:00
isc_result_t
dns_dispatch_getlocaladdress(dns_dispatch_t *disp, isc_sockaddr_t *addrp) {
REQUIRE(VALID_DISPATCH(disp));
REQUIRE(addrp != NULL);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
if (disp->socktype == isc_socktype_udp) {
add dns_dispatch_getlocaladdress().
2001-03-13 05:48:41 +00:00
*addrp = disp->local;
return ISC_R_SUCCESS;
}
return ISC_R_NOTIMPLEMENTED;
}
Move isc_socket_getsockname() calls into dispatch We now use dns_dispentry_getlocaladdress(). (this API is likely to be cleaned up further later.)
2021-01-04 14:38:35 -08:00
isc_result_t
dns_dispentry_getlocaladdress(dns_dispentry_t *resp, isc_sockaddr_t *addrp) {
REQUIRE(VALID_RESPONSE(resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(VALID_DISPATCH(resp->disp));
Move isc_socket_getsockname() calls into dispatch We now use dns_dispentry_getlocaladdress(). (this API is likely to be cleaned up further later.)
2021-01-04 14:38:35 -08:00
REQUIRE(addrp != NULL);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
Move isc_socket_getsockname() calls into dispatch We now use dns_dispentry_getlocaladdress(). (this API is likely to be cleaned up further later.)
2021-01-04 14:38:35 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (disp->socktype) {
case isc_socktype_tcp:
Enable extraction of exact local socket addresses Extracting the exact address that each wildcard/TCP socket is bound to locally requires issuing the getsockname() system call, which libuv exposes via its uv_*_getsockname() functions. This is only required for detailed logging and comes at a noticeable performance cost, so it should not happen by default. However, it is useful for debugging certain problems (e.g. cryptic system test failures), so a convenient way of enabling that behavior should exist. Update isc_nmhandle_localaddr() so that it calls uv_*_getsockname() when the ISC_SOCKET_DETAILS preprocessor macro is set at compile time. Ensure proper handling of sockets that wrap other sockets. Set the new ISC_SOCKET_DETAILS macro by default when --enable-developer is passed to ./configure. This enables detailed logging in the system tests run in GitLab CI without affecting performance in non-development BIND 9 builds. Note that setting the ISC_SOCKET_DETAILS preprocessor macro at compile time enables all callers of isc_nmhandle_localaddr() to extract the exact address of a given local socket, which results e.g. in dnstap captures containing more accurate information. Mention the new preprocessor macro in the section of the ARM that discusses why exact socket addresses may not be logged by default.
2024-12-29 12:32:05 +01:00
*addrp = isc_nmhandle_localaddr(disp->handle);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
return ISC_R_SUCCESS;
case isc_socktype_udp:
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
*addrp = isc_nmhandle_localaddr(resp->handle);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
return ISC_R_SUCCESS;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
default:
UNREACHABLE();
Move isc_socket_getsockname() calls into dispatch We now use dns_dispentry_getlocaladdress(). (this API is likely to be cleaned up further later.)
2021-01-04 14:38:35 -08:00
}
}
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
dns_dispatch_t *
dns_dispatchset_get(dns_dispatchset_t *dset) {
Convert the isc/tid.h to use own signed integer isc_tid_t type Change the internal type used for isc_tid unit to isc_tid_t to hide the specific integer type being used for the 'tid'. Internally, the signed integer type is being used. This allows us to have negatively indexed arrays that works both for threads with assigned tid and the threads with unassigned tid. This should be used only in specific situations.
2025-06-04 17:54:20 +02:00
isc_tid_t tid = isc_tid();
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
/* check that dispatch set is configured */
if (dset == NULL || dset->ndisp == 0) {
return NULL;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
Convert the isc/tid.h to use own signed integer isc_tid_t type Change the internal type used for isc_tid unit to isc_tid_t to hide the specific integer type being used for the 'tid'. Internally, the signed integer type is being used. This allows us to have negatively indexed arrays that works both for threads with assigned tid and the threads with unassigned tid. This should be used only in specific situations.
2025-06-04 17:54:20 +02:00
INSIST((uint32_t)tid < dset->ndisp);
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
return dset->dispatches[tid];
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
}
isc_result_t
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
dns_dispatchset_create(isc_mem_t *mctx, dns_dispatch_t *source,
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dns_dispatchset_t **dsetp, uint32_t ndisp) {
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
isc_result_t result;
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
dns_dispatchset_t *dset = NULL;
dns_dispatchmgr_t *mgr = NULL;
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
size_t i;
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
REQUIRE(VALID_DISPATCH(source));
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
REQUIRE(source->socktype == isc_socktype_udp);
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
REQUIRE(dsetp != NULL && *dsetp == NULL);
mgr = source->mgr;
dset = isc_mem_get(mctx, sizeof(dns_dispatchset_t));
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
*dset = (dns_dispatchset_t){ .ndisp = ndisp };
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
isc_mem_attach(mctx, &dset->mctx);
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dset->dispatches = isc_mem_cget(dset->mctx, ndisp,
sizeof(dns_dispatch_t *));
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
dset->dispatches[0] = NULL;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_attach(source, &dset->dispatches[0]); /* DISPATCH004 */
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
for (i = 1; i < dset->ndisp; i++) {
result = dispatch_createudp(mgr, &source->local, i,
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
&dset->dispatches[i]);
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
if (result != ISC_R_SUCCESS) {
goto fail;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
}
*dsetp = dset;
return ISC_R_SUCCESS;
fail:
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
for (size_t j = 0; j < i; j++) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_detach(&(dset->dispatches[j])); /* DISPATCH004 */
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
isc_mem_cput(dset->mctx, dset->dispatches, ndisp,
sizeof(dns_dispatch_t *));
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
isc_mem_putanddetach(&dset->mctx, dset, sizeof(dns_dispatchset_t));
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
return result;
}
void
dns_dispatchset_destroy(dns_dispatchset_t **dsetp) {
REQUIRE(dsetp != NULL && *dsetp != NULL);
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dns_dispatchset_t *dset = *dsetp;
Clear the pointer to destroyed object early using the semantic patch Also disable the semantic patch as the code needs tweaks here and there because some destroy functions might not destroy the object and return early if the object is still in use.
2020-02-08 04:37:54 -08:00
*dsetp = NULL;
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
for (size_t i = 0; i < dset->ndisp; i++) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_detach(&(dset->dispatches[i])); /* DISPATCH004 */
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
isc_mem_cput(dset->mctx, dset->dispatches, dset->ndisp,
sizeof(dns_dispatch_t *));
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
isc_mem_putanddetach(&dset->mctx, dset, sizeof(dns_dispatchset_t));
}
refactor dns_xfrin to use dns_dispatch the dns_xfrin module was still using the network manager directly to manage TCP connections and send and receive messages. this commit changes it to use the dispatch manager instead.
2023-02-21 20:14:30 -08:00
isc_result_t
dns_dispatch_checkperm(dns_dispatch_t *disp) {
REQUIRE(VALID_DISPATCH(disp));
if (disp->handle == NULL || disp->socktype == isc_socktype_udp) {
return ISC_R_NOPERM;
}
return isc_nm_xfr_checkperm(disp->handle);
}
Reference in New Issue Copy Permalink