mir/bind
2
0
Fork 0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-22 18:19:42 +00:00
Code Issues Releases Wiki Activity
bind/lib/dns/dispatch.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2258 lines
55 KiB
C
Raw Normal View History

checkpoint
1999-06-16 01:32:31 +00:00
/*
update copyright notice
2011-02-03 12:18:12 +00:00
* Copyright (C) Internet Systems Consortium, Inc. ("ISC")
checkpoint
1999-06-16 01:32:31 +00:00
*
* SPDX-License-Identifier: MPL-2.0
Update the copyright information in all files in the repository This commit converts the license handling to adhere to the REUSE specification. It specifically: 1. Adds used licnses to LICENSES/ directory 2. Add "isc" template for adding the copyright boilerplate 3. Changes all source files to include copyright and SPDX license header, this includes all the C sources, documentation, zone files, configuration files. There are notes in the doc/dev/copyrights file on how to add correct headers to the new files. 4. Handle the rest that can't be modified via .reuse/dep5 file. The binary (or otherwise unmodifiable) files could have license places next to them in <foo>.license file, but this would lead to cluttered repository and most of the files handled in the .reuse/dep5 file are system test files.
2021-06-03 08:37:05 +02:00
*
checkpoint
1999-06-16 01:32:31 +00:00
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, you can obtain one at https://mozilla.org/MPL/2.0/.
Update license headers to not include years in copyright in all applicable files
2018-02-23 09:53:12 +01:00
*
checkpoint
1999-06-16 01:32:31 +00:00
* See the COPYRIGHT file distributed with this work for additional
checkpoint
1999-07-08 22:12:37 +00:00
* information regarding copyright ownership.
checkpoint
1999-06-16 01:32:31 +00:00
*/
1851. [doc] Doxygen comment markup. [RT #11398]
2005-04-27 04:57:32 +00:00
/*! \file */
add RCS id string
2000-06-22 22:00:42 +00:00
Replace custom isc_u?intNN_t types with C99 u?intNN_t types
2018-03-28 14:19:37 +02:00
#include <inttypes.h>
Replace custom isc_boolean_t with C standard bool type
2018-04-17 08:29:14 -07:00
#include <stdbool.h>
checkpoint
1999-06-16 01:32:31 +00:00
#include <stdlib.h>
Use the new sorting rules to regroup #include headers
2020-03-09 16:17:26 +01:00
#include <sys/types.h>
2203. [security] Query id generation was cryptographically weak. [RT # 16915]
2007-06-26 02:52:15 +00:00
#include <unistd.h>
checkpoint
1999-06-16 01:32:31 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
#include <isc/hash.h>
#include <isc/hashmap.h>
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
#include <isc/loop.h>
checkpoint
1999-06-18 02:01:42 +00:00
#include <isc/mem.h>
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
#include <isc/mutex.h>
Use system ephemeral ports for default portset In dispatch, use system ephemeral ports for default portset instead of hardcoded <1024,65535> range.
2021-10-04 14:11:57 +02:00
#include <isc/net.h>
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
#include <isc/netmgr.h>
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
#include <isc/portset.h>
#include <isc/random.h>
2537. [func] Added more statistics counters including those on socket I/O events and query RTT histograms. [RT #18802]
2009-01-27 22:30:00 +00:00
#include <isc/stats.h>
needs string.h for memset() prototype.
2000-05-13 21:57:02 +00:00
#include <isc/string.h>
Pin the dns_dispatch to threads when reusing Previously, dns_dispatch_gettcp() could pick a TCP connection created by different thread - this breaks our contractual promise to DNS dispatch by using the TCP connection on a different thread than it was created. Add .tid member to the dns_dispatch_t struct and skip the dispatches from other threads when looking up a TCP dispatch that we can reuse in dns_request. NOTE: This is going to be properly refactored, but this change could be also backported to 9.18 for better stability and thread-affinity.
2023-01-04 19:21:14 +01:00
#include <isc/tid.h>
replace gettimeofday with isc_time_now
2007-06-26 06:02:37 +00:00
#include <isc/time.h>
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
#include <isc/tls.h>
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
#include <isc/urcu.h>
move util.h to <isc/util.h>
1999-12-16 22:24:22 +00:00
#include <isc/util.h>
checkpoint
1999-06-16 01:32:31 +00:00
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
#include <dns/acl.h>
checkpoint
1999-06-16 01:32:31 +00:00
#include <dns/dispatch.h>
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
#include <dns/log.h>
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
#include <dns/message.h>
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
#include <dns/stats.h>
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
#include <dns/transport.h>
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
#include <dns/types.h>
469. [bug] "query-source address * port 53;" now works.
2000-09-18 04:50:05 +00:00
typedef ISC_LIST(dns_dispentry_t) dns_displist_t;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
struct dns_dispatchmgr {
/* Unlocked. */
unsigned int magic;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_refcount_t references;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
isc_mem_t *mctx;
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
dns_acl_t *blackhole;
2537. [func] Added more statistics counters including those on socket I/O events and query RTT histograms. [RT #18802]
2009-01-27 22:30:00 +00:00
isc_stats_t *stats;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_nm_t *nm;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
uint32_t nloops;
struct cds_lfht **tcps;
Move the UDP buffer pool to the dispatchmgr. UDP parameters are now set by the first call to dns_dispatch_getudp(), subsecquent calls can raise the maximum number of buffers. dns_dispatchmgr_setudp() could be made public. Tidy dispatch_allocate(). entropy and mctx are now gone from dns_qid_t.
2000-09-19 06:59:28 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
struct cds_lfht *qids;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
in_port_t *v4ports; /*%< available ports for IPv4 */
unsigned int nv4ports; /*%< # of available ports for IPv4 */
in_port_t *v6ports; /*%< available ports for IPv4 */
unsigned int nv6ports; /*%< # of available ports for IPv4 */
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
};
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
typedef enum {
DNS_DISPATCHSTATE_NONE = 0UL,
DNS_DISPATCHSTATE_CONNECTING,
DNS_DISPATCHSTATE_CONNECTED,
DNS_DISPATCHSTATE_CANCELED,
} dns_dispatchstate_t;
checkpoint
1999-07-06 19:32:40 +00:00
struct dns_dispentry {
checkpoint
1999-06-18 02:01:42 +00:00
unsigned int magic;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_refcount_t references;
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
isc_mem_t *mctx;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
dns_dispatch_t *disp;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
isc_loop_t *loop;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_nmhandle_t *handle; /*%< netmgr handle for UDP connection */
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatchstate_t state;
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
dns_transport_t *transport;
isc_tlsctx_cache_t *tlsctx_cache;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
unsigned int retries;
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
unsigned int timeout;
isc_time_t start;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_sockaddr_t local;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_sockaddr_t peer;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
in_port_t port;
dns_messageid_t id;
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
dispatch_cb_t connected;
dispatch_cb_t sent;
dispatch_cb_t response;
checkpoint
1999-06-16 01:32:31 +00:00
void *arg;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
bool reading;
isc_result_t result;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
ISC_LINK(dns_dispentry_t) alink;
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
ISC_LINK(dns_dispentry_t) plink;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
ISC_LINK(dns_dispentry_t) rlink;
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
struct cds_lfht_node ht_node;
struct rcu_head rcu_head;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
};
checkpoint
1999-06-16 01:32:31 +00:00
struct dns_dispatch {
/* Unlocked. */
Pin the dns_dispatch to threads when reusing Previously, dns_dispatch_gettcp() could pick a TCP connection created by different thread - this breaks our contractual promise to DNS dispatch by using the TCP connection on a different thread than it was created. Add .tid member to the dns_dispatch_t struct and skip the dispatches from other threads when looking up a TCP dispatch that we can reuse in dns_request. NOTE: This is going to be properly refactored, but this change could be also backported to 9.18 for better stability and thread-affinity.
2023-01-04 19:21:14 +01:00
unsigned int magic; /*%< magic */
uint32_t tid;
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
isc_mem_t *mctx;
1851. [doc] Doxygen comment markup. [RT #11398]
2005-04-27 04:57:32 +00:00
dns_dispatchmgr_t *mgr; /*%< dispatch manager */
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_nmhandle_t *handle; /*%< netmgr handle for TCP connection */
isc_sockaddr_t local; /*%< local address */
in_port_t localport; /*%< local UDP port */
isc_sockaddr_t peer; /*%< peer address (TCP) */
Improve locking performance in dispatch.c 3316. [tuning] Improved locking performance when recursing. [RT #28836] - Use one lock per memory pool instead of associating them all with a single 'pool_lock' in the dispatch manager. - Reduce the critical sections for qid->lock in get_dispsocket(), deref_portentry() and dns_dispatch_addresponse2(). - Added a memory pool for socket events in dns_dispatch_t. - Add an isc_socketevent_t member in the resquery_t object, and use it with isc_socket_sendto2() instead of using isc_socket_sendto() - Tuned the memory pools in dispatch.c for better performance under load
2012-04-28 14:52:28 -07:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_socktype_t socktype;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatchstate_t state;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_refcount_t references;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
bool reading;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
dns_displist_t pending;
dns_displist_t active;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
uint_fast32_t requests; /*%< how many requests we have */
Tear down the TCP connection on too many unexpected DNS messages When the outgoing TCP dispatch times-out active response, we might still receive the answer during the lifetime of the connection. Previously, we would just ignore any non-matching DNS answers, which would allow the server to feed us with otherwise valid DNS answer and keep the connection open. Add a counter for timed-out DNS queries over TCP and tear down the whole TCP connection if we receive unexpected number of DNS answers.
2021-11-29 09:59:33 +01:00
unsigned int timedout;
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
struct cds_lfht_node ht_node;
struct rcu_head rcu_head;
checkpoint
1999-06-16 01:32:31 +00:00
};
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
#define RESPONSE_MAGIC ISC_MAGIC('D', 'r', 's', 'p')
#define VALID_RESPONSE(e) ISC_MAGIC_VALID((e), RESPONSE_MAGIC)
checkpoint
1999-06-18 02:01:42 +00:00
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
#define DISPSOCK_MAGIC ISC_MAGIC('D', 's', 'o', 'c')
#define VALID_DISPSOCK(e) ISC_MAGIC_VALID((e), DISPSOCK_MAGIC)
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
#define DISPATCH_MAGIC ISC_MAGIC('D', 'i', 's', 'p')
#define VALID_DISPATCH(e) ISC_MAGIC_VALID((e), DISPATCH_MAGIC)
checkpoint
1999-06-18 02:01:42 +00:00
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
#define DNS_DISPATCHMGR_MAGIC ISC_MAGIC('D', 'M', 'g', 'r')
#define VALID_DISPATCHMGR(e) ISC_MAGIC_VALID((e), DNS_DISPATCHMGR_MAGIC)
checkpoint
1999-06-18 02:01:42 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
#define dns_dispentry_ref(ptr) \
dns_dispentry__ref(ptr, __func__, __FILE__, __LINE__)
#define dns_dispentry_unref(ptr) \
dns_dispentry__unref(ptr, __func__, __FILE__, __LINE__)
#define dns_dispentry_attach(ptr, ptrp) \
dns_dispentry__attach(ptr, ptrp, __func__, __FILE__, __LINE__)
#define dns_dispentry_detach(ptrp) \
dns_dispentry__detach(ptrp, __func__, __FILE__, __LINE__)
ISC_REFCOUNT_TRACE_DECL(dns_dispentry);
#else
ISC_REFCOUNT_DECL(dns_dispentry);
#endif
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
/*
* The number of attempts to find unique <addr, port, query_id> combination
*/
#define QID_MAX_TRIES 64
/*
* Initial and minimum QID table sizes.
*/
#define QIDS_INIT_SIZE (1 << 4) /* Must be power of 2 */
#define QIDS_MIN_SIZE (1 << 4) /* Must be power of 2 */
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
/*
needs string.h for memset() prototype.
2000-05-13 21:57:02 +00:00
* Statics.
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
*/
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
static void
dispatchmgr_destroy(dns_dispatchmgr_t *mgr);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
static void
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
udp_recv(isc_nmhandle_t *handle, isc_result_t eresult, isc_region_t *region,
void *arg);
Use an LFSR to generate unpredictable IDs.
1999-12-15 03:11:00 +00:00
static void
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
tcp_recv(isc_nmhandle_t *handle, isc_result_t eresult, isc_region_t *region,
void *arg);
1687. [bug] Race condition in dispatch. [RT #10272]
2004-07-21 00:48:19 +00:00
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_cancel(dns_dispentry_t *resp, isc_result_t result);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
static isc_result_t
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
dispatch_createudp(dns_dispatchmgr_t *mgr, const isc_sockaddr_t *localaddr,
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
uint32_t tid, dns_dispatch_t **dispp);
Dispatch API simplification - Many dispatch attributes can be set implicitly instead of being passed in. we can infer whether to set DNS_DISPATCHATTR_TCP or _UDP from whether we're calling dns_dispatch_createtcp() or _createudp(). we can also infer DNS_DISPATCHATTR_IPV4 or _IPV6 from the addresses or the socket that were passed in. - We no longer use dup'd sockets in UDP dispatches, so the 'dup_socket' parameter has been removed from dns_dispatch_createudp(), along with the code implementing it. also removed isc_socket_dup() since it no longer has any callers. - The 'buffersize' parameter was ignored and has now been removed; buffersize is now fixed at 4096. - Maxbuffers and maxrequests don't need to be passed in on every call to dns_dispatch_createtcp() and _createudp(). In all current uses, the value for mgr->maxbuffers will either be raised once from its default of 20000 to 32768, or else left alone. (passing in a value lower than 20000 does not lower it.) there isn't enough difference between these values for there to be any need to configure this. The value for disp->maxrequests controls both the quota of concurrent requests for a dispatch and also the size of the dispatch socket memory pool. it's not clear that this quota is necessary at all. the memory pool size currently starts at 32768, but is sometimes lowered to 4096, which is definitely unnecessary. This commit sets both values permanently to 32768. - Previously TCP dispatches allocated their own separate QID table, which didn't incorporate a port table. this commit removes per-dispatch QID tables and shares the same table between all dispatches. since dispatches are created for each TCP socket, this may speed up the dispatch allocation process. there may be a slight increase in lock contention since all dispatches are sharing a single QID table, but since TCP sockets are used less often than UDP sockets (which were already sharing a QID table), it should not be a substantial change. - The dispatch port table was being used to determine whether a port was already in use; if so, then a UDP socket would be bound with REUSEADDR. this commit removes the port table, and always binds UDP sockets that way.
2020-12-17 00:43:00 -08:00
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
udp_startrecv(isc_nmhandle_t *handle, dns_dispentry_t *resp);
static void
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
udp_dispatch_connect(dns_dispatch_t *disp, dns_dispentry_t *resp);
static void
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
tcp_startrecv(dns_dispatch_t *disp, dns_dispentry_t *resp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
tcp_dispatch_getnext(dns_dispatch_t *disp, dns_dispentry_t *resp,
int32_t timeout);
static void
udp_dispatch_getnext(dns_dispentry_t *resp, int32_t timeout);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
#define LVL(x) ISC_LOG_DEBUG(x)
checkpoint
1999-07-08 02:50:00 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static const char *
socktype2str(dns_dispentry_t *resp) {
dns_transport_type_t transport_type = DNS_TRANSPORT_UDP;
dns_dispatch_t *disp = resp->disp;
if (disp->socktype == isc_socktype_tcp) {
if (resp->transport != NULL) {
transport_type =
dns_transport_get_type(resp->transport);
} else {
transport_type = DNS_TRANSPORT_TCP;
}
}
switch (transport_type) {
case DNS_TRANSPORT_UDP:
return ("UDP");
case DNS_TRANSPORT_TCP:
return ("TCP");
case DNS_TRANSPORT_TLS:
return ("TLS");
case DNS_TRANSPORT_HTTP:
return ("HTTP");
default:
return ("<unexpected>");
}
}
static const char *
state2str(dns_dispatchstate_t state) {
switch (state) {
case DNS_DISPATCHSTATE_NONE:
return ("none");
case DNS_DISPATCHSTATE_CONNECTING:
return ("connecting");
case DNS_DISPATCHSTATE_CONNECTED:
return ("connected");
case DNS_DISPATCHSTATE_CANCELED:
return ("canceled");
default:
return ("<unexpected>");
}
}
format string bugs and improved format string checking [RT #1578]
2001-08-08 22:54:55 +00:00
static void
mgr_log(dns_dispatchmgr_t *mgr, int level, const char *fmt, ...)
ISC_FORMAT_PRINTF(3, 4);
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
static void
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
mgr_log(dns_dispatchmgr_t *mgr, int level, const char *fmt, ...) {
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
char msgbuf[2048];
va_list ap;
Added calls to isc_log_wouldlog() in additional logging functions.
2000-07-17 23:25:35 +00:00
if (!isc_log_wouldlog(dns_lctx, level)) {
return;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Added calls to isc_log_wouldlog() in additional logging functions.
2000-07-17 23:25:35 +00:00
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
va_start(ap, fmt);
vsnprintf(msgbuf, sizeof(msgbuf), fmt, ap);
va_end(ap);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
isc_log_write(dns_lctx, DNS_LOGCATEGORY_DISPATCH,
DNS_LOGMODULE_DISPATCH, level, "dispatchmgr %p: %s", mgr,
msgbuf);
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
isc_statscounter_t
2009-01-31 00:37:04 +00:00
inc_stats(dns_dispatchmgr_t *mgr, isc_statscounter_t counter) {
2541. [bug] Conditionally update dispatch manager statistics. [RT #19247]
2009-01-31 00:10:24 +00:00
if (mgr->stats != NULL) {
isc_stats_increment(mgr->stats, counter);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
2541. [bug] Conditionally update dispatch manager statistics. [RT #19247]
2009-01-31 00:10:24 +00:00
}
Remove use of the inline keyword used as suggestion to compiler Historically, the inline keyword was a strong suggestion to the compiler that it should inline the function marked inline. As compilers became better at optimising, this functionality has receded, and using inline as a suggestion to inline a function is obsolete. The compiler will happily ignore it and inline something else entirely if it finds that's a better optimisation. Therefore, remove all the occurences of the inline keyword with static functions inside single compilation unit and leave the decision whether to inline a function or not entirely on the compiler NOTE: We keep the usage the inline keyword when the purpose is to change the linkage behaviour.
2021-10-11 13:43:12 +02:00
static void
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
dec_stats(dns_dispatchmgr_t *mgr, isc_statscounter_t counter) {
if (mgr->stats != NULL) {
isc_stats_decrement(mgr->stats, counter);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Merge statistics code (ATT SoW, rt24117) This includes the following changes: 3326. [func] Added task list statistics: task model, worker threads, quantum, tasks running, tasks ready. [RT #27678] 3325. [func] Report cache statistics: memory use, number of nodes, number of hash buckets, hit and miss counts. [RT #27056] 3324. [test] Add better tests for ADB stats [RT #27057] 3323. [func] Report the number of buckets the resolver is using. [RT #27020] 3322. [func] Monitor the number of active TCP and UDP dispatches. [RT #27055] 3321. [func] Monitor the number of recursive fetches and the number of open sockets, and report these values in the statistics channel. [RT #27054] 3320. [func] Added support for monitoring of recursing client count. [RT #27009] 3319. [func] Added support for monitoring of ADB entry count and hash size. [RT #27057]
2012-05-14 10:06:05 -07:00
}
format string bugs and improved format string checking [RT #1578]
2001-08-08 22:54:55 +00:00
static void
dispatch_log(dns_dispatch_t *disp, int level, const char *fmt, ...)
ISC_FORMAT_PRINTF(3, 4);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
static void
dispatch_log(dns_dispatch_t *disp, int level, const char *fmt, ...) {
char msgbuf[2048];
va_list ap;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
int r;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
use isc_log_wouldlog()
2000-07-13 01:16:22 +00:00
if (!isc_log_wouldlog(dns_lctx, level)) {
return;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Trailing whitespace trimmed. Perhaps running "perl util/spacewhack.pl in your own CVS tree will help minimize CVS conflicts. Maybe not. Blame Graff for getting me to trim all trailing whitespace.
2000-08-01 01:33:37 +00:00
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
va_start(ap, fmt);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
r = vsnprintf(msgbuf, sizeof(msgbuf), fmt, ap);
if (r < 0) {
msgbuf[0] = '\0';
} else if ((unsigned int)r >= sizeof(msgbuf)) {
/* Truncated */
msgbuf[sizeof(msgbuf) - 1] = '\0';
}
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
va_end(ap);
isc_log_write(dns_lctx, DNS_LOGCATEGORY_DISPATCH,
DNS_LOGMODULE_DISPATCH, level, "dispatch %p: %s", disp,
msgbuf);
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
dispentry_log(dns_dispentry_t *resp, int level, const char *fmt, ...)
ISC_FORMAT_PRINTF(3, 4);
static void
dispentry_log(dns_dispentry_t *resp, int level, const char *fmt, ...) {
char msgbuf[2048];
va_list ap;
int r;
if (!isc_log_wouldlog(dns_lctx, level)) {
return;
}
va_start(ap, fmt);
r = vsnprintf(msgbuf, sizeof(msgbuf), fmt, ap);
if (r < 0) {
msgbuf[0] = '\0';
} else if ((unsigned int)r >= sizeof(msgbuf)) {
/* Truncated */
msgbuf[sizeof(msgbuf) - 1] = '\0';
}
va_end(ap);
dispatch_log(resp->disp, level, "%s response %p: %s",
socktype2str(resp), resp, msgbuf);
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
/*%
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
* Choose a random port number for a dispatch entry.
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
*/
static isc_result_t
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
setup_socket(dns_dispatch_t *disp, dns_dispentry_t *resp,
const isc_sockaddr_t *dest, in_port_t *portp) {
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
dns_dispatchmgr_t *mgr = disp->mgr;
unsigned int nports;
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
in_port_t *ports = NULL;
use configured source ports for UDP requests the optional 'port' option, when used with notify-source, transfer-source, etc, is used to set up UDP dispatches with a particular source port, but when the actual UDP connection was established the port would be overridden with a random one. this has been fixed. (configuring source ports is deprecated in 9.20 and slated for removal in 9.22, but should still work correctly until then.)
2023-02-02 12:16:49 -08:00
in_port_t port = *portp;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
if (resp->retries++ > 5) {
return (ISC_R_FAILURE);
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
if (isc_sockaddr_pf(&disp->local) == AF_INET) {
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
nports = mgr->nv4ports;
ports = mgr->v4ports;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
} else {
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
nports = mgr->nv6ports;
ports = mgr->v6ports;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
}
if (nports == 0) {
return (ISC_R_ADDRNOTAVAIL);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
resp->local = disp->local;
resp->peer = *dest;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
use configured source ports for UDP requests the optional 'port' option, when used with notify-source, transfer-source, etc, is used to set up UDP dispatches with a particular source port, but when the actual UDP connection was established the port would be overridden with a random one. this has been fixed. (configuring source ports is deprecated in 9.20 and slated for removal in 9.22, but should still work correctly until then.)
2023-02-02 12:16:49 -08:00
if (port == 0) {
port = ports[isc_random_uniform(nports)];
isc_sockaddr_setport(&resp->local, port);
*portp = port;
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
resp->port = port;
Dispatch API simplification - Many dispatch attributes can be set implicitly instead of being passed in. we can infer whether to set DNS_DISPATCHATTR_TCP or _UDP from whether we're calling dns_dispatch_createtcp() or _createudp(). we can also infer DNS_DISPATCHATTR_IPV4 or _IPV6 from the addresses or the socket that were passed in. - We no longer use dup'd sockets in UDP dispatches, so the 'dup_socket' parameter has been removed from dns_dispatch_createudp(), along with the code implementing it. also removed isc_socket_dup() since it no longer has any callers. - The 'buffersize' parameter was ignored and has now been removed; buffersize is now fixed at 4096. - Maxbuffers and maxrequests don't need to be passed in on every call to dns_dispatch_createtcp() and _createudp(). In all current uses, the value for mgr->maxbuffers will either be raised once from its default of 20000 to 32768, or else left alone. (passing in a value lower than 20000 does not lower it.) there isn't enough difference between these values for there to be any need to configure this. The value for disp->maxrequests controls both the quota of concurrent requests for a dispatch and also the size of the dispatch socket memory pool. it's not clear that this quota is necessary at all. the memory pool size currently starts at 32768, but is sometimes lowered to 4096, which is definitely unnecessary. This commit sets both values permanently to 32768. - Previously TCP dispatches allocated their own separate QID table, which didn't incorporate a port table. this commit removes per-dispatch QID tables and shares the same table between all dispatches. since dispatches are created for each TCP socket, this may speed up the dispatch allocation process. there may be a slight increase in lock contention since all dispatches are sharing a single QID table, but since TCP sockets are used less often than UDP sockets (which were already sharing a QID table), it should not be a substantial change. - The dispatch port table was being used to determine whether a port was already in use; if so, then a UDP socket would be bound with REUSEADDR. this commit removes the port table, and always binds UDP sockets that way.
2020-12-17 00:43:00 -08:00
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
return (ISC_R_SUCCESS);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
static uint32_t
qid_hash(const dns_dispentry_t *dispentry) {
/*
* TODO(OS): Add incremental isc_sockaddr_hash() function and then use
* isc_hash32 API
*/
uint32_t hashval = isc_sockaddr_hash(&dispentry->peer, true);
return (hashval ^ (((uint32_t)dispentry->id << 16) | dispentry->port));
}
add another REQUIRE() where it might help
1999-07-09 20:34:26 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
static int
qid_match(struct cds_lfht_node *node, const void *key0) {
const dns_dispentry_t *dispentry =
caa_container_of(node, dns_dispentry_t, ht_node);
const dns_dispentry_t *key = key0;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
return (dispentry->id == key->id && dispentry->port == key->port &&
isc_sockaddr_equal(&dispentry->peer, &key->peer));
}
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
static void
dispentry_destroy_rcu(struct rcu_head *rcu_head) {
dns_dispentry_t *resp = caa_container_of(rcu_head, dns_dispentry_t,
rcu_head);
isc_mem_putanddetach(&resp->mctx, resp, sizeof(*resp));
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_destroy(dns_dispentry_t *resp) {
dns_dispatch_t *disp = resp->disp;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/*
* We need to call this from here in case there's an external event that
* shuts down our dispatch (like ISC_R_SHUTTINGDOWN).
*/
dispentry_cancel(resp, ISC_R_CANCELED);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(disp->requests > 0);
disp->requests--;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_refcount_destroy(&resp->references);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
resp->magic = 0;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(!ISC_LINK_LINKED(resp, plink));
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
INSIST(!ISC_LINK_LINKED(resp, alink));
INSIST(!ISC_LINK_LINKED(resp, rlink));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90), "destroying");
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (resp->handle != NULL) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90), "detaching handle %p from %p",
resp->handle, &resp->handle);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_nmhandle_detach(&resp->handle);
}
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
if (resp->tlsctx_cache != NULL) {
isc_tlsctx_cache_detach(&resp->tlsctx_cache);
}
if (resp->transport != NULL) {
dns_transport_detach(&resp->transport);
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_detach(&disp); /* DISPATCH001 */
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
call_rcu(&resp->rcu_head, dispentry_destroy_rcu);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
ISC_REFCOUNT_TRACE_IMPL(dns_dispentry, dispentry_destroy);
#else
ISC_REFCOUNT_IMPL(dns_dispentry, dispentry_destroy);
#endif
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
/*
* How long in milliseconds has it been since this dispentry
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
* started reading?
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
*/
static unsigned int
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
dispentry_runtime(dns_dispentry_t *resp, const isc_time_t *now) {
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
if (isc_time_isepoch(&resp->start)) {
return (0);
}
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
return (isc_time_microdiff(now, &resp->start) / 1000);
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
}
checkpoint
1999-07-08 02:50:00 +00:00
/*
* General flow:
*
don't retry UDP receives after they return an error - an error return from the socket library means the library has already decided that the error is not retryable and likely to throw us into an infinite loop if retried
2001-05-14 23:10:19 +00:00
* If I/O result == CANCELED or error, free the buffer.
checkpoint
1999-07-08 02:50:00 +00:00
*
fixed a bug that named crashes with an assertion failure on exit when sharing the same port for listening and querying, and changing listening addresses several times. [RT# 3509] additionally, + limited the canceled socket tasks in dispatch.c + made dns_dispatch_changeattributes() care about the NOLISTEN mask + described side effects of dns_dispatch_changeattributes() in its description comment
2002-09-04 02:26:13 +00:00
* If query, free the buffer, restart.
checkpoint
1999-07-08 02:50:00 +00:00
*
* If response:
* Allocate event, fill in details.
* If cannot allocate, free buffer, restart.
* find target. If not found, free buffer, restart.
* if event queue is not empty, queue. else, send.
* restart.
*/
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
static void
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
udp_recv(isc_nmhandle_t *handle, isc_result_t eresult, isc_region_t *region,
void *arg) {
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dns_dispentry_t *resp = (dns_dispentry_t *)arg;
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
dns_dispatch_t *disp = NULL;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
dns_messageid_t id;
fix many bugs.
1999-07-22 01:34:31 +00:00
isc_result_t dres;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
isc_buffer_t source;
unsigned int flags;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_sockaddr_t peer;
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
isc_netaddr_t netaddr;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
int match, timeout = 0;
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
bool respond = true;
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
isc_time_t now;
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
REQUIRE(VALID_RESPONSE(resp));
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
REQUIRE(VALID_DISPATCH(resp->disp));
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
disp = resp->disp;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(resp->reading);
resp->reading = false;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (resp->state == DNS_DISPATCHSTATE_CANCELED) {
/*
* Nobody is interested in the callback if the response
* has been canceled already. Detach from the response
* and the handle.
*/
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
respond = false;
Handle ISC_R_SUCCESS on a deactivated response in udp_recv() There is a possibility for `udp_recv()` to be called with `eresult` being `ISC_R_SUCCESS`, but nevertheless with already deactivated `resp`, which can happen when the request has been canceled in the meantime.
2022-04-27 12:29:50 +00:00
eresult = ISC_R_CANCELED;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
}
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dispentry_log(resp, LVL(90), "read callback:%s, requests %" PRIuFAST32,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(eresult), disp->requests);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
if (eresult != ISC_R_SUCCESS) {
add missing exit checks to udp_recv and tcp_recv
2000-02-02 23:29:47 +00:00
/*
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
* This is most likely a network error on a connected
* socket, a timeout, or the query has been canceled.
* It makes no sense to check the address or parse the
* packet, but we can return the error to the caller.
add missing exit checks to udp_recv and tcp_recv
2000-02-02 23:29:47 +00:00
*/
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto done;
add missing exit checks to udp_recv and tcp_recv
2000-02-02 23:29:47 +00:00
}
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
peer = isc_nmhandle_peeraddr(handle);
isc_netaddr_fromsockaddr(&netaddr, &peer);
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
/*
* If this is from a blackholed address, drop it.
*/
there is no need for each dispatch to have a separate reference to the blackhole list; they can use the blackhole list of the dispatch manager directly. This eliminates some unsafe unlocked accesses to the reference count of the blackhole list dns_acl_t object
2001-02-08 18:25:09 +00:00
if (disp->mgr->blackhole != NULL &&
remove the experimental authoritative ECS support from named - mark the 'geoip-use-ecs' option obsolete; warn when it is used in named.conf - prohibit 'ecs' ACL tags in named.conf; note that this is a fatal error since simply ignoring the tags could make ACLs behave unpredictably - re-simplify the radix and iptable code - clean up dns_acl_match(), dns_aclelement_match(), dns_acl_allowed() and dns_geoip_match() so they no longer take ecs options - remove the ECS-specific unit and system test cases - remove references to ECS from the ARM
2018-04-26 20:57:41 -07:00
dns_acl_match(&netaddr, NULL, disp->mgr->blackhole, NULL, &match,
2353. [func] Add support for Name Server ID (RFC 5001). 'dig +nsid' requests NSID from server. 'request-nsid yes;' causes recursive server to send NSID requests to upstream servers. Server responds to NSID requests with the string configured by 'server-id' option. [RT #17091]
2008-04-03 02:01:08 +00:00
NULL) == ISC_R_SUCCESS &&
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
match > 0)
{
if (isc_log_wouldlog(dns_lctx, LVL(10))) {
char netaddrstr[ISC_NETADDR_FORMATSIZE];
isc_netaddr_format(&netaddr, netaddrstr,
sizeof(netaddrstr));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(10),
"blackholed packet from %s", netaddrstr);
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
}
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto next;
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
}
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
/*
* Peek into the buffer to see what we can see.
*/
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
id = resp->id;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_buffer_init(&source, region->base, region->length);
isc_buffer_add(&source, region->length);
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
dres = dns_message_peekheader(&source, &id, &flags);
s/DNS_R_/ISC_R_/ change for some codes.
2000-04-06 22:03:35 +00:00
if (dres != ISC_R_SUCCESS) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
char netaddrstr[ISC_NETADDR_FORMATSIZE];
isc_netaddr_format(&netaddr, netaddrstr, sizeof(netaddrstr));
dispentry_log(resp, LVL(10), "got garbage packet from %s",
netaddrstr);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto next;
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(92),
"got valid DNS message header, /QR %c, id %u",
(((flags & DNS_MESSAGEFLAG_QR) != 0) ? '1' : '0'), id);
checkpoint
1999-07-08 22:12:37 +00:00
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
/*
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
* Look at the message flags. If it's a query, ignore it.
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
*/
checkpoint
1999-07-06 19:32:40 +00:00
if ((flags & DNS_MESSAGEFLAG_QR) == 0) {
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto next;
1687. [bug] Race condition in dispatch. [RT #10272]
2004-07-21 00:48:19 +00:00
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
/*
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
* The QID and the address must match the expected ones.
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
*/
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
if (resp->id != id || !isc_sockaddr_equal(&peer, &resp->peer)) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90), "response doesn't match");
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
inc_stats(disp->mgr, dns_resstatscounter_mismatch);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto next;
Silence cppcheck 2.2 false positive in udp_recv() cppcheck 2.2 reports the following false positive: lib/dns/dispatch.c:1239:14: warning: Either the condition 'resp==NULL' is redundant or there is possible null pointer dereference: resp. [nullPointerRedundantCheck] if (disp != resp->disp) { ^ lib/dns/dispatch.c:1210:11: note: Assuming that condition 'resp==NULL' is not redundant if (resp == NULL) { ^ lib/dns/dispatch.c:1239:14: note: Null pointer dereference if (disp != resp->disp) { ^ Apparently this version of cppcheck gets confused about conditional "goto" statements because line 1239 can never be reached if 'resp' is NULL. Move a code block to prevent the above false positive from being reported without affecting the processing logic.
2020-11-25 12:45:47 +01:00
}
checkpoint
1999-07-08 22:12:37 +00:00
/*
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
* We have the right resp, so call the caller back.
checkpoint
1999-07-08 22:12:37 +00:00
*/
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto done;
checkpoint
1999-07-08 22:12:37 +00:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
next:
/*
Fix handling of mismatched responses past timeout When a UDP dispatch receives a mismatched response, it checks whether there is still enough time to wait for the correct one to arrive before the timeout fires. If there is not, the result code is set to ISC_R_TIMEDOUT, but it is not subsequently used anywhere as 'response' is set to NULL a few lines earlier. This results in the higher-level read callback (resquery_response() in case of resolver code) not being called. However, shortly afterwards, a few levels up the call chain, isc__nm_udp_read_cb() calls isc__nmsocket_timer_stop() on the dispatch socket, effectively disabling read timeout handling for that socket. Combined with the fact that reading is not restarted in such a case (e.g. by calling dispatch_getnext() from udp_recv()), this leads to the higher-level query structure remaining referenced indefinitely because the dispatch socket it uses will neither be read from nor closed due to a timeout. This in turn causes fetch contexts to linger around indefinitely, which in turn i.a. prevents certain cache nodes (those containing rdatasets used by fetch contexts, like fctx->nameservers) from being cleaned. Fix by making sure the higher-level callback does get invoked with the ISC_R_TIMEDOUT result code when udp_recv() determines there is no more time left to receive the correct UDP response before the timeout fires. This allows the higher-level callback to clean things up, preventing the reference leak described above.
2021-11-23 15:35:39 +01:00
* This is the wrong response. Check whether there is still enough
* time to wait for the correct one to arrive before the timeout fires.
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
*/
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
now = isc_loop_now(resp->loop);
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
timeout = resp->timeout - dispentry_runtime(resp, &now);
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
if (timeout <= 0) {
Fix handling of mismatched responses past timeout When a UDP dispatch receives a mismatched response, it checks whether there is still enough time to wait for the correct one to arrive before the timeout fires. If there is not, the result code is set to ISC_R_TIMEDOUT, but it is not subsequently used anywhere as 'response' is set to NULL a few lines earlier. This results in the higher-level read callback (resquery_response() in case of resolver code) not being called. However, shortly afterwards, a few levels up the call chain, isc__nm_udp_read_cb() calls isc__nmsocket_timer_stop() on the dispatch socket, effectively disabling read timeout handling for that socket. Combined with the fact that reading is not restarted in such a case (e.g. by calling dispatch_getnext() from udp_recv()), this leads to the higher-level query structure remaining referenced indefinitely because the dispatch socket it uses will neither be read from nor closed due to a timeout. This in turn causes fetch contexts to linger around indefinitely, which in turn i.a. prevents certain cache nodes (those containing rdatasets used by fetch contexts, like fctx->nameservers) from being cleaned. Fix by making sure the higher-level callback does get invoked with the ISC_R_TIMEDOUT result code when udp_recv() determines there is no more time left to receive the correct UDP response before the timeout fires. This allows the higher-level callback to clean things up, preventing the reference leak described above.
2021-11-23 15:35:39 +01:00
/*
* The time window for receiving the correct response is
* already closed, libuv has just not processed the socket
* timer yet. Invoke the read callback, indicating a timeout.
*/
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
eresult = ISC_R_TIMEDOUT;
goto done;
}
Fix handling of mismatched responses past timeout When a UDP dispatch receives a mismatched response, it checks whether there is still enough time to wait for the correct one to arrive before the timeout fires. If there is not, the result code is set to ISC_R_TIMEDOUT, but it is not subsequently used anywhere as 'response' is set to NULL a few lines earlier. This results in the higher-level read callback (resquery_response() in case of resolver code) not being called. However, shortly afterwards, a few levels up the call chain, isc__nm_udp_read_cb() calls isc__nmsocket_timer_stop() on the dispatch socket, effectively disabling read timeout handling for that socket. Combined with the fact that reading is not restarted in such a case (e.g. by calling dispatch_getnext() from udp_recv()), this leads to the higher-level query structure remaining referenced indefinitely because the dispatch socket it uses will neither be read from nor closed due to a timeout. This in turn causes fetch contexts to linger around indefinitely, which in turn i.a. prevents certain cache nodes (those containing rdatasets used by fetch contexts, like fctx->nameservers) from being cleaned. Fix by making sure the higher-level callback does get invoked with the ISC_R_TIMEDOUT result code when udp_recv() determines there is no more time left to receive the correct UDP response before the timeout fires. This allows the higher-level callback to clean things up, preventing the reference leak described above.
2021-11-23 15:35:39 +01:00
/*
* Do not invoke the read callback just yet and instead wait for the
* proper response to arrive until the original timeout fires.
*/
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
respond = false;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
udp_dispatch_getnext(resp, timeout);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
done:
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
if (respond) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90), "UDP read callback on %p: %s",
handle, isc_result_totext(eresult));
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
resp->response(eresult, region, resp->arg);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_detach(&resp); /* DISPENTRY003 */
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
}
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
static isc_result_t
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_recv_oldest(dns_dispatch_t *disp, dns_dispentry_t **respp) {
dns_dispentry_t *resp = NULL;
resp = ISC_LIST_HEAD(disp->active);
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
if (resp != NULL) {
disp->timedout++;
*respp = resp;
return (ISC_R_TIMEDOUT);
}
return (ISC_R_NOTFOUND);
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
/*
* NOTE: Must be RCU read locked!
*/
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
static isc_result_t
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
tcp_recv_success(dns_dispatch_t *disp, isc_region_t *region,
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
isc_sockaddr_t *peer, dns_dispentry_t **respp) {
isc_buffer_t source;
dns_messageid_t id;
unsigned int flags;
isc_result_t result = ISC_R_SUCCESS;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispatch_log(disp, LVL(90), "TCP read success, length == %d, addr = %p",
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
region->length, region->base);
/*
* Peek into the buffer to see what we can see.
*/
isc_buffer_init(&source, region->base, region->length);
isc_buffer_add(&source, region->length);
result = dns_message_peekheader(&source, &id, &flags);
if (result != ISC_R_SUCCESS) {
dispatch_log(disp, LVL(10), "got garbage packet");
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
return (ISC_R_UNEXPECTED);
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
dispatch_log(disp, LVL(92),
"got valid DNS message header, /QR %c, id %u",
(((flags & DNS_MESSAGEFLAG_QR) != 0) ? '1' : '0'), id);
/*
* Look at the message flags. If it's a query, ignore it and keep
* reading.
*/
if ((flags & DNS_MESSAGEFLAG_QR) == 0) {
dispatch_log(disp, LVL(10), "got DNS query instead of answer");
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
return (ISC_R_UNEXPECTED);
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
/*
* We have a valid response; find the associated dispentry object
* and call the caller back.
*/
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
dns_dispentry_t key = {
.id = id,
.peer = *peer,
.port = disp->localport,
};
struct cds_lfht_iter iter;
cds_lfht_lookup(disp->mgr->qids, qid_hash(&key), qid_match, &key,
&iter);
dns_dispentry_t *resp = cds_lfht_entry(cds_lfht_iter_get_node(&iter),
dns_dispentry_t, ht_node);
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
if (resp != NULL) {
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
if (!resp->reading) {
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
/*
* We already got a message for this QID and weren't
* expecting any more.
*/
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
result = ISC_R_UNEXPECTED;
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
} else {
*respp = resp;
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
}
Tear down the TCP connection on too many unexpected DNS messages When the outgoing TCP dispatch times-out active response, we might still receive the answer during the lifetime of the connection. Previously, we would just ignore any non-matching DNS answers, which would allow the server to feed us with otherwise valid DNS answer and keep the connection open. Add a counter for timed-out DNS queries over TCP and tear down the whole TCP connection if we receive unexpected number of DNS answers.
2021-11-29 09:59:33 +01:00
} else {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
result = ISC_R_NOTFOUND;
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
dispatch_log(disp, LVL(90), "search for response in hashtable: %s",
isc_result_totext(result));
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
return (result);
}
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_recv_add(dns_displist_t *resps, dns_dispentry_t *resp,
isc_result_t result) {
dns_dispentry_ref(resp); /* DISPENTRY009 */
ISC_LIST_UNLINK(resp->disp->active, resp, alink);
ISC_LIST_APPEND(*resps, resp, rlink);
INSIST(resp->reading);
resp->reading = false;
resp->result = result;
}
static void
tcp_recv_shutdown(dns_dispatch_t *disp, dns_displist_t *resps,
isc_result_t result) {
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
dns_dispentry_t *resp = NULL, *next = NULL;
/*
* If there are any active responses, shut them all down.
*/
for (resp = ISC_LIST_HEAD(disp->active); resp != NULL; resp = next) {
next = ISC_LIST_NEXT(resp, alink);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_recv_add(resps, resp, result);
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
Ignore TCP dispatches that have zero references The TCP dispatches are removed from the dispatchmgr->list in the dispatch_destroy() and there's a brief period of time where dns_dispatch_gettcp() can find a dispatch in connected state that's being destroyed. Set the dispatch state to DNS_DISPATCHSTATE_NONE in the TCP connection callback if there are no responses waiting, and ignore TCP dispatches with zero references in dns_dispatch_gettcp().
2022-12-20 06:11:26 +01:00
disp->state = DNS_DISPATCHSTATE_CANCELED;
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_recv_processall(dns_displist_t *resps, isc_region_t *region) {
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
dns_dispentry_t *resp = NULL, *next = NULL;
for (resp = ISC_LIST_HEAD(*resps); resp != NULL; resp = next) {
next = ISC_LIST_NEXT(resp, rlink);
ISC_LIST_UNLINK(*resps, resp, rlink);
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
dispentry_log(resp, LVL(90), "read callback: %s",
isc_result_totext(resp->result));
resp->response(resp->result, region, resp->arg);
dns_dispentry_detach(&resp); /* DISPENTRY009 */
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
}
}
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
/*
* General flow:
*
fixed a bug that named crashes with an assertion failure on exit when sharing the same port for listening and querying, and changing listening addresses several times. [RT# 3509] additionally, + limited the canceled socket tasks in dispatch.c + made dns_dispatch_changeattributes() care about the NOLISTEN mask + described side effects of dns_dispatch_changeattributes() in its description comment
2002-09-04 02:26:13 +00:00
* If I/O result == CANCELED, EOF, or error, notify everyone as the
* various queues drain.
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
*
* If response:
* Allocate event, fill in details.
fixed a bug that named crashes with an assertion failure on exit when sharing the same port for listening and querying, and changing listening addresses several times. [RT# 3509] additionally, + limited the canceled socket tasks in dispatch.c + made dns_dispatch_changeattributes() care about the NOLISTEN mask + described side effects of dns_dispatch_changeattributes() in its description comment
2002-09-04 02:26:13 +00:00
* If cannot allocate, restart.
* find target. If not found, restart.
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
* if event queue is not empty, queue. else, send.
* restart.
*/
static void
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
tcp_recv(isc_nmhandle_t *handle, isc_result_t result, isc_region_t *region,
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
void *arg) {
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
dns_dispatch_t *disp = (dns_dispatch_t *)arg;
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
dns_dispentry_t *resp = NULL;
1342, [func] Log remote address with TCP dispatch failures.
2002-07-29 01:03:24 +00:00
char buf[ISC_SOCKADDR_FORMATSIZE];
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_sockaddr_t peer;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_displist_t resps = ISC_LIST_INITIALIZER;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
isc_time_t now;
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
int timeout;
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
REQUIRE(VALID_DISPATCH(disp));
In tcp_recv(), free the event before calling destroy(), since the event is a statically allocated part of the tcpmsg.
1999-11-16 21:05:09 +00:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(disp->reading);
disp->reading = false;
lock dispatch before reporting state
2019-11-22 10:49:40 +11:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dispatch_log(disp, LVL(90), "TCP read:%s:requests %" PRIuFAST32,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(result), disp->requests);
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
peer = isc_nmhandle_peeraddr(handle);
2353. [func] Add support for Name Server ID (RFC 5001). 'dig +nsid' requests NSID from server. 'request-nsid yes;' causes recursive server to send NSID requests to upstream servers. Server responds to NSID requests with the string configured by 'server-id' option. [RT #17091]
2008-04-03 02:01:08 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_lock();
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/*
* Phase 1: Process timeout and success.
*/
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
switch (result) {
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
case ISC_R_TIMEDOUT:
/*
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
* Time out the oldest response in the active queue.
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
*/
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
result = tcp_recv_oldest(disp, &resp);
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
break;
case ISC_R_SUCCESS:
/* We got an answer */
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
result = tcp_recv_success(disp, region, &peer, &resp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
break;
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
default:
break;
}
if (resp != NULL) {
tcp_recv_add(&resps, resp, result);
}
/*
* Phase 2: Look if we timed out before.
*/
if (result == ISC_R_NOTFOUND) {
if (disp->timedout > 0) {
/* There was active query that timed-out before */
disp->timedout--;
} else {
result = ISC_R_UNEXPECTED;
}
}
/*
* Phase 3: Trigger timeouts. It's possible that the responses would
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
* have been timed out out already, but non-matching TCP reads have
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
* prevented this.
*/
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp = ISC_LIST_HEAD(disp->active);
if (resp != NULL) {
now = isc_loop_now(resp->loop);
}
while (resp != NULL) {
dns_dispentry_t *next = ISC_LIST_NEXT(resp, alink);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
timeout = resp->timeout - dispentry_runtime(resp, &now);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (timeout <= 0) {
tcp_recv_add(&resps, resp, ISC_R_TIMEDOUT);
TCP receives should never be retried
2000-09-08 22:02:21 +00:00
}
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp = next;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
/*
* Phase 4: log if we errored out.
*/
switch (result) {
case ISC_R_SUCCESS:
case ISC_R_TIMEDOUT:
case ISC_R_NOTFOUND:
break;
add dispatcher logging functions
2000-04-29 00:45:26 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case ISC_R_SHUTTINGDOWN:
case ISC_R_CANCELED:
case ISC_R_EOF:
case ISC_R_CONNECTIONRESET:
isc_sockaddr_format(&peer, buf, sizeof(buf));
dispatch_log(disp, LVL(90), "shutting down TCP: %s: %s", buf,
isc_result_totext(result));
tcp_recv_shutdown(disp, &resps, result);
break;
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
default:
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
isc_sockaddr_format(&peer, buf, sizeof(buf));
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
dispatch_log(disp, ISC_LOG_ERROR,
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
"shutting down due to TCP "
"receive error: %s: %s",
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
buf, isc_result_totext(result));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_recv_shutdown(disp, &resps, result);
Refactor tcp_recv() The tcp_recv() function used lot of gotos that made the function hard to read. Refactor the function by splitting it into smaller logical chunks.
2021-11-26 09:14:58 +01:00
break;
Shutdown all TCP connection on invalid DNS message Previously, when invalid DNS message is received over TCP we throw the garbage DNS message away and continued looking for valid DNS message that would match our outgoing queries. This logic makes sense for UDP, because anyone can send DNS message over UDP. Change the logic that the TCP connection is closed when we receive garbage, because the other side is acting malicious.
2021-11-26 09:14:58 +01:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/*
* Phase 5: Resume reading if there are still active responses
*/
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
resp = ISC_LIST_HEAD(disp->active);
if (resp != NULL) {
timeout = resp->timeout - dispentry_runtime(resp, &now);
INSIST(timeout > 0);
tcp_startrecv(disp, resp);
isc_nmhandle_settimeout(handle, timeout);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_unlock();
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/*
* Phase 6: Process all scheduled callbacks.
*/
tcp_recv_processall(&resps, region);
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_detach(&disp); /* DISPATCH002 */
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
/*%
* Create a temporary port list to set the initial default set of dispatch
Use system ephemeral ports for default portset In dispatch, use system ephemeral ports for default portset instead of hardcoded <1024,65535> range.
2021-10-04 14:11:57 +02:00
* ephemeral ports. This is almost meaningless as the application will
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
* normally set the ports explicitly, but is provided to fill some minor corner
* cases.
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
*/
static void
Use system ephemeral ports for default portset In dispatch, use system ephemeral ports for default portset instead of hardcoded <1024,65535> range.
2021-10-04 14:11:57 +02:00
create_default_portset(isc_mem_t *mctx, int family, isc_portset_t **portsetp) {
in_port_t low, high;
isc_net_getudpportrange(family, &low, &high);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_portset_create(mctx, portsetp);
Use system ephemeral ports for default portset In dispatch, use system ephemeral ports for default portset instead of hardcoded <1024,65535> range.
2021-10-04 14:11:57 +02:00
isc_portset_addrange(*portsetp, low, high);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
static isc_result_t
setavailports(dns_dispatchmgr_t *mgr, isc_portset_t *v4portset,
isc_portset_t *v6portset) {
in_port_t *v4ports, *v6ports, p = 0;
unsigned int nv4ports, nv6ports, i4 = 0, i6 = 0;
nv4ports = isc_portset_nports(v4portset);
nv6ports = isc_portset_nports(v6portset);
v4ports = NULL;
if (nv4ports != 0) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
v4ports = isc_mem_cget(mgr->mctx, nv4ports, sizeof(in_port_t));
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
v6ports = NULL;
if (nv6ports != 0) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
v6ports = isc_mem_cget(mgr->mctx, nv6ports, sizeof(in_port_t));
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
}
do {
if (isc_portset_isset(v4portset, p)) {
INSIST(i4 < nv4ports);
v4ports[i4++] = p;
}
if (isc_portset_isset(v6portset, p)) {
INSIST(i6 < nv6ports);
v6ports[i6++] = p;
}
} while (p++ < 65535);
INSIST(i4 == nv4ports && i6 == nv6ports);
if (mgr->v4ports != NULL) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
isc_mem_cput(mgr->mctx, mgr->v4ports, mgr->nv4ports,
sizeof(in_port_t));
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
}
mgr->v4ports = v4ports;
mgr->nv4ports = nv4ports;
if (mgr->v6ports != NULL) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
isc_mem_cput(mgr->mctx, mgr->v6ports, mgr->nv6ports,
sizeof(in_port_t));
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
}
mgr->v6ports = v6ports;
mgr->nv6ports = nv6ports;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
return (ISC_R_SUCCESS);
}
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
/*
* Publics.
*/
fix many bugs.
1999-07-22 01:34:31 +00:00
isc_result_t
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
dns_dispatchmgr_create(isc_mem_t *mctx, isc_loopmgr_t *loopmgr, isc_nm_t *nm,
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
dns_dispatchmgr_t **mgrp) {
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
dns_dispatchmgr_t *mgr = NULL;
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
isc_portset_t *v4portset = NULL;
isc_portset_t *v6portset = NULL;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
REQUIRE(mctx != NULL);
REQUIRE(mgrp != NULL && *mgrp == NULL);
mgr = isc_mem_get(mctx, sizeof(dns_dispatchmgr_t));
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
*mgr = (dns_dispatchmgr_t){
.magic = 0,
.nloops = isc_loopmgr_nloops(loopmgr),
};
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
fprintf(stderr, "dns_dispatchmgr__init:%s:%s:%d:%p->references = 1\n",
__func__, __FILE__, __LINE__, mgr);
#endif
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_refcount_init(&mgr->references, 1);
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
isc_mem_attach(mctx, &mgr->mctx);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_nm_attach(nm, &mgr->nm);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
mgr->tcps = isc_mem_cget(mgr->mctx, mgr->nloops, sizeof(mgr->tcps[0]));
for (size_t i = 0; i < mgr->nloops; i++) {
mgr->tcps[i] = cds_lfht_new(
2, 2, 0, CDS_LFHT_AUTO_RESIZE | CDS_LFHT_ACCOUNTING,
NULL);
}
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
create_default_portset(mgr->mctx, AF_INET, &v4portset);
create_default_portset(mgr->mctx, AF_INET6, &v6portset);
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
setavailports(mgr, v4portset, v6portset);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
isc_portset_destroy(mgr->mctx, &v4portset);
isc_portset_destroy(mgr->mctx, &v6portset);
mgr->qids = cds_lfht_new(QIDS_INIT_SIZE, QIDS_MIN_SIZE, 0,
CDS_LFHT_AUTO_RESIZE | CDS_LFHT_ACCOUNTING,
NULL);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
mgr->magic = DNS_DISPATCHMGR_MAGIC;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
*mgrp = mgr;
return (ISC_R_SUCCESS);
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
ISC_REFCOUNT_TRACE_IMPL(dns_dispatchmgr, dispatchmgr_destroy);
#else
ISC_REFCOUNT_IMPL(dns_dispatchmgr, dispatchmgr_destroy);
#endif
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
void
dns_dispatchmgr_setblackhole(dns_dispatchmgr_t *mgr, dns_acl_t *blackhole) {
REQUIRE(VALID_DISPATCHMGR(mgr));
635. [bug] Reloading a server with a configured blackhole list would cause an assertion. [RT #590]
2000-12-26 09:48:41 +00:00
if (mgr->blackhole != NULL) {
dns_acl_detach(&mgr->blackhole);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
dns_acl_attach(blackhole, &mgr->blackhole);
}
removed more needless attaching and detaching of the blackhole ACL
2001-02-09 00:23:16 +00:00
dns_acl_t *
dns_dispatchmgr_getblackhole(dns_dispatchmgr_t *mgr) {
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
REQUIRE(VALID_DISPATCHMGR(mgr));
removed more needless attaching and detaching of the blackhole ACL
2001-02-09 00:23:16 +00:00
return (mgr->blackhole);
539. [func] Support the blackhole option.
2000-11-03 02:45:55 +00:00
}
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
isc_result_t
dns_dispatchmgr_setavailports(dns_dispatchmgr_t *mgr, isc_portset_t *v4portset,
isc_portset_t *v6portset) {
REQUIRE(VALID_DISPATCHMGR(mgr));
Simplify dns_dispatchmgr_create with fixed buffersize - UDP buffersize is now established when creating dispatch manager and is always set to 4096. - Set up the default port range in dispatchmgr before setting the magic number. - Magic is not set until dispatchmgr is fully created.
2020-12-09 15:45:13 -08:00
return (setavailports(mgr, v4portset, v6portset));
1442. [func] New fuctions for manipulating port lists: dns_portlist_create(), dns_portlist_add(), dns_portlist_remove(), dns_portlist_match(), dns_portlist_attach() and dns_portlist_detach(). 1441. [func] It is now possible to tell dig to bind to a specific source port. 1440. [func] It is now possible to tell named to avoid using certian source ports (avoid-v4-udp-ports, avoid-v6-udp-ports). developer: marka reviewer: explorer
2003-02-26 05:05:16 +00:00
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
static void
dispatchmgr_destroy(dns_dispatchmgr_t *mgr) {
REQUIRE(VALID_DISPATCHMGR(mgr));
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_refcount_destroy(&mgr->references);
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
mgr->magic = 0;
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
RUNTIME_CHECK(!cds_lfht_destroy(mgr->qids, NULL));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
for (size_t i = 0; i < mgr->nloops; i++) {
RUNTIME_CHECK(!cds_lfht_destroy(mgr->tcps[i], NULL));
}
isc_mem_cput(mgr->mctx, mgr->tcps, mgr->nloops, sizeof(mgr->tcps[0]));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (mgr->blackhole != NULL) {
dns_acl_detach(&mgr->blackhole);
}
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (mgr->stats != NULL) {
isc_stats_detach(&mgr->stats);
}
Trailing whitespace trimmed. Perhaps running "perl util/spacewhack.pl in your own CVS tree will help minimize CVS conflicts. Maybe not. Blame Graff for getting me to trim all trailing whitespace.
2000-08-01 01:33:37 +00:00
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (mgr->v4ports != NULL) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
isc_mem_cput(mgr->mctx, mgr->v4ports, mgr->nv4ports,
sizeof(in_port_t));
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (mgr->v6ports != NULL) {
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
isc_mem_cput(mgr->mctx, mgr->v6ports, mgr->nv6ports,
sizeof(in_port_t));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
isc_nm_detach(&mgr->nm);
isc_mem_putanddetach(&mgr->mctx, mgr, sizeof(dns_dispatchmgr_t));
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
}
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
void
2537. [func] Added more statistics counters including those on socket I/O events and query RTT histograms. [RT #18802]
2009-01-27 22:30:00 +00:00
dns_dispatchmgr_setstats(dns_dispatchmgr_t *mgr, isc_stats_t *stats) {
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
REQUIRE(VALID_DISPATCHMGR(mgr));
REQUIRE(mgr->stats == NULL);
2537. [func] Added more statistics counters including those on socket I/O events and query RTT histograms. [RT #18802]
2009-01-27 22:30:00 +00:00
isc_stats_attach(stats, &mgr->stats);
2355. [func] Extend the number statistics counters available. [RT #17590]
2008-04-03 05:55:52 +00:00
}
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
/*
* Allocate and set important limits.
*/
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
static void
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dispatch_allocate(dns_dispatchmgr_t *mgr, isc_socktype_t type, uint32_t tid,
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
dns_dispatch_t **dispp) {
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
dns_dispatch_t *disp = NULL;
checkpoint
1999-06-18 02:01:42 +00:00
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
REQUIRE(VALID_DISPATCHMGR(mgr));
checkpoint
1999-06-18 23:54:59 +00:00
REQUIRE(dispp != NULL && *dispp == NULL);
checkpoint
1999-06-18 02:01:42 +00:00
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
/*
* Set up the dispatcher, mostly. Don't bother setting some of
* the options that are controlled by tcp vs. udp, etc.
*/
checkpoint
1999-06-18 02:01:42 +00:00
Replace locked mempools with memory contexts Current mempools are kind of hybrid structures - they serve two purposes: 1. mempool with a lock is basically static sized allocator with pre-allocated free items 2. mempool without a lock is a doubly-linked list of preallocated items The first kind of usage could be easily replaced with jemalloc small sized arena objects and thread-local caches. The second usage not-so-much and we need to keep this (in libdns:message.c) for performance reasons.
2021-05-12 21:16:17 +02:00
disp = isc_mem_get(mgr->mctx, sizeof(*disp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
*disp = (dns_dispatch_t){
.socktype = type,
.active = ISC_LIST_INITIALIZER,
.pending = ISC_LIST_INITIALIZER,
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
.tid = tid,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
.magic = DISPATCH_MAGIC,
};
checkpoint
1999-06-18 02:01:42 +00:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
isc_mem_attach(mgr->mctx, &disp->mctx);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dns_dispatchmgr_attach(mgr, &disp->mgr);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
fprintf(stderr, "dns_dispatch__init:%s:%s:%d:%p->references = 1\n",
__func__, __FILE__, __LINE__, disp);
#endif
isc_refcount_init(&disp->references, 1); /* DISPATCH000 */
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
*dispp = disp;
}
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
struct dispatch_key {
const isc_sockaddr_t *local;
const isc_sockaddr_t *peer;
};
static uint32_t
dispatch_hash(struct dispatch_key *key) {
uint32_t hashval = isc_sockaddr_hash(key->peer, false);
if (key->local) {
hashval ^= isc_sockaddr_hash(key->local, true);
}
return (hashval);
}
static int
dispatch_match(struct cds_lfht_node *node, const void *key0) {
dns_dispatch_t *disp = caa_container_of(node, dns_dispatch_t, ht_node);
const struct dispatch_key *key = key0;
isc_sockaddr_t local;
isc_sockaddr_t peer;
if (disp->handle != NULL) {
local = isc_nmhandle_localaddr(disp->handle);
peer = isc_nmhandle_peeraddr(disp->handle);
} else {
local = disp->local;
peer = disp->peer;
}
return (isc_sockaddr_equal(&peer, key->peer) &&
(key->local == NULL || isc_sockaddr_equal(&local, key->local)));
}
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
isc_result_t
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
dns_dispatch_createtcp(dns_dispatchmgr_t *mgr, const isc_sockaddr_t *localaddr,
remove nonfunctional DSCP implementation DSCP has not been fully working since the network manager was introduced in 9.16, and has been completely broken since 9.18. This seems to have caused very few difficulties for anyone, so we have now marked it as obsolete and removed the implementation. To ensure that old config files don't fail, the code to parse dscp key-value pairs is still present, but a warning is logged that the feature is obsolete and should not be used. Nothing is done with configured values, and there is no longer any range checking.
2023-01-05 22:18:55 -08:00
const isc_sockaddr_t *destaddr, dns_dispatch_t **dispp) {
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
dns_dispatch_t *disp = NULL;
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
uint32_t tid = isc_tid();
Move the UDP buffer pool to the dispatchmgr. UDP parameters are now set by the first call to dns_dispatch_getudp(), subsecquent calls can raise the maximum number of buffers. dns_dispatchmgr_setudp() could be made public. Tidy dispatch_allocate(). entropy and mctx are now gone from dns_qid_t.
2000-09-19 06:59:28 +00:00
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
REQUIRE(VALID_DISPATCHMGR(mgr));
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
REQUIRE(destaddr != NULL);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
dispatch_allocate(mgr, isc_socktype_tcp, tid, &disp);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
disp->peer = *destaddr;
if (localaddr != NULL) {
disp->local = *localaddr;
} else {
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
int pf;
pf = isc_sockaddr_pf(destaddr);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
isc_sockaddr_anyofpf(&disp->local, pf);
isc_sockaddr_setport(&disp->local, 0);
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
}
implement dns_dispatchmgr_t, replacing dns_dispatchlist_t. Use it throughout the library/server.
2000-05-10 21:34:50 +00:00
/*
* Append it to the dispatcher list.
*/
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
struct dispatch_key key = {
.local = &disp->local,
.peer = &disp->peer,
};
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
rcu_read_lock();
cds_lfht_add(mgr->tcps[tid], dispatch_hash(&key), &disp->ht_node);
rcu_read_unlock();
make this (mostly) work with tcp
1999-07-12 23:44:31 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (isc_log_wouldlog(dns_lctx, 90)) {
char addrbuf[ISC_SOCKADDR_FORMATSIZE];
add dns_dispatch_attach/detach(), and start on serious dispatcher hacking.
1999-06-30 01:33:11 +00:00
Don't pass a NULL pointer to isc_sockaddr_format() The 'localaddr' pointer can be NULL, which causes an assertion failure. Use '&disp->local' instead when printing a debug log message.
2022-12-28 12:10:09 +00:00
isc_sockaddr_format(&disp->local, addrbuf,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
ISC_SOCKADDR_FORMATSIZE);
mgr_log(mgr, LVL(90),
"dns_dispatch_createtcp: created TCP dispatch %p for "
"%s",
disp, addrbuf);
}
*dispp = disp;
return (ISC_R_SUCCESS);
}
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
3939. [func] Improve UPDATE forwarding performance by allowing TCP connections to be shared. [RT #37039]
2014-09-04 10:37:45 +10:00
isc_result_t
4546. [func] Extend the use of const declarations. [RT #43379]
2016-12-30 15:45:08 +11:00
dns_dispatch_gettcp(dns_dispatchmgr_t *mgr, const isc_sockaddr_t *destaddr,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
const isc_sockaddr_t *localaddr, dns_dispatch_t **dispp) {
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
dns_dispatch_t *disp_connected = NULL;
dns_dispatch_t *disp_fallback = NULL;
isc_result_t result = ISC_R_NOTFOUND;
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
uint32_t tid = isc_tid();
[master] allow shared TCP sockets when connecting 4041. [func] TCP sockets can now be shared while connecting. (This will be used to enable client-side support of pipelined queries.) [RT #38231]
2015-01-20 17:22:31 -08:00
REQUIRE(VALID_DISPATCHMGR(mgr));
REQUIRE(destaddr != NULL);
REQUIRE(dispp != NULL && *dispp == NULL);
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
struct dispatch_key key = {
.local = localaddr,
.peer = destaddr,
};
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
rcu_read_lock();
struct cds_lfht_iter iter;
dns_dispatch_t *disp = NULL;
cds_lfht_for_each_entry_duplicate(mgr->tcps[tid], dispatch_hash(&key),
dispatch_match, &key, &iter, disp,
ht_node) {
INSIST(disp->tid == isc_tid());
INSIST(disp->socktype == isc_socktype_tcp);
Ignore TCP dispatches that have zero references The TCP dispatches are removed from the dispatchmgr->list in the dispatch_destroy() and there's a brief period of time where dns_dispatch_gettcp() can find a dispatch in connected state that's being destroyed. Set the dispatch state to DNS_DISPATCHSTATE_NONE in the TCP connection callback if there are no responses waiting, and ignore TCP dispatches with zero references in dns_dispatch_gettcp().
2022-12-20 06:11:26 +01:00
switch (disp->state) {
case DNS_DISPATCHSTATE_NONE:
/* A dispatch in indeterminate state, skip it */
break;
case DNS_DISPATCHSTATE_CONNECTED:
if (ISC_LIST_EMPTY(disp->active)) {
/* Ignore dispatch with no responses */
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
break;
Ignore TCP dispatches that have zero references The TCP dispatches are removed from the dispatchmgr->list in the dispatch_destroy() and there's a brief period of time where dns_dispatch_gettcp() can find a dispatch in connected state that's being destroyed. Set the dispatch state to DNS_DISPATCHSTATE_NONE in the TCP connection callback if there are no responses waiting, and ignore TCP dispatches with zero references in dns_dispatch_gettcp().
2022-12-20 06:11:26 +01:00
}
/* We found a connected dispatch */
dns_dispatch_attach(disp, &disp_connected);
break;
case DNS_DISPATCHSTATE_CONNECTING:
if (ISC_LIST_EMPTY(disp->pending)) {
/* Ignore dispatch with no responses */
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
break;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
}
Ignore TCP dispatches that have zero references The TCP dispatches are removed from the dispatchmgr->list in the dispatch_destroy() and there's a brief period of time where dns_dispatch_gettcp() can find a dispatch in connected state that's being destroyed. Set the dispatch state to DNS_DISPATCHSTATE_NONE in the TCP connection callback if there are no responses waiting, and ignore TCP dispatches with zero references in dns_dispatch_gettcp().
2022-12-20 06:11:26 +01:00
/* We found "a" dispatch, store it for later */
if (disp_fallback == NULL) {
dns_dispatch_attach(disp, &disp_fallback);
}
break;
case DNS_DISPATCHSTATE_CANCELED:
/* A canceled dispatch, skip it. */
break;
default:
UNREACHABLE();
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (disp_connected != NULL) {
break;
}
[master] allow shared TCP sockets when connecting 4041. [func] TCP sockets can now be shared while connecting. (This will be used to enable client-side support of pipelined queries.) [RT #38231]
2015-01-20 17:22:31 -08:00
}
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
rcu_read_unlock();
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
if (disp_connected != NULL) {
/* We found connected dispatch */
INSIST(disp_connected->handle != NULL);
[master] allow shared TCP sockets when connecting 4041. [func] TCP sockets can now be shared while connecting. (This will be used to enable client-side support of pipelined queries.) [RT #38231]
2015-01-20 17:22:31 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
*dispp = disp_connected;
disp_connected = NULL;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
result = ISC_R_SUCCESS;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (disp_fallback != NULL) {
dns_dispatch_detach(&disp_fallback);
}
} else if (disp_fallback != NULL) {
*dispp = disp_fallback;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
result = ISC_R_SUCCESS;
[master] allow shared TCP sockets when connecting 4041. [func] TCP sockets can now be shared while connecting. (This will be used to enable client-side support of pipelined queries.) [RT #38231]
2015-01-20 17:22:31 -08:00
}
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
return (result);
[master] allow shared TCP sockets when connecting 4041. [func] TCP sockets can now be shared while connecting. (This will be used to enable client-side support of pipelined queries.) [RT #38231]
2015-01-20 17:22:31 -08:00
}
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
isc_result_t
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
dns_dispatch_createudp(dns_dispatchmgr_t *mgr, const isc_sockaddr_t *localaddr,
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
dns_dispatch_t **dispp) {
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
isc_result_t result;
2129. [func] Provide a pool of UDP sockets for queries to be made over. See use-queryport-pool, queryport-pool-ports and queryport-pool-updateinterval. [RT #16415]
2007-02-02 02:18:06 +00:00
dns_dispatch_t *disp = NULL;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
REQUIRE(VALID_DISPATCHMGR(mgr));
REQUIRE(localaddr != NULL);
REQUIRE(dispp != NULL && *dispp == NULL);
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
result = dispatch_createudp(mgr, localaddr, isc_tid(), &disp);
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
if (result == ISC_R_SUCCESS) {
*dispp = disp;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
}
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
return (result);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
}
static isc_result_t
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
dispatch_createudp(dns_dispatchmgr_t *mgr, const isc_sockaddr_t *localaddr,
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
uint32_t tid, dns_dispatch_t **dispp) {
Creating TCP dispatch now creates/binds the socket Previously, creation of TCP dispatches differed from UDP in that a TCP dispatch was created to attach to an existing socket, whereas a UDP dispatch would be created in a vacuum and sockets would be opened on demand when a transaction was initiated. We are moving as much socket code as possible into the dispatch module, so that it can be replaced with a netmgr version as easily as possible. (This will also have the side effect of making TCP and UDP dispatches more similar.) As a step in that direction, this commit changes dns_dispatch_createtcp() so that it creates the TCP socket.
2020-12-18 14:59:50 -08:00
isc_result_t result = ISC_R_SUCCESS;
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
dns_dispatch_t *disp = NULL;
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
isc_sockaddr_t sa_any;
style: one-line statement braces, line length, etc
2018-11-08 19:34:51 -08:00
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
/*
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
* Check whether this address/port is available locally.
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
*/
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
isc_sockaddr_anyofpf(&sa_any, isc_sockaddr_pf(localaddr));
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
if (!isc_sockaddr_eqaddr(&sa_any, localaddr)) {
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
result = isc_nm_checkaddr(localaddr, isc_socktype_udp);
style: one-line statement braces, line length, etc
2018-11-08 19:34:51 -08:00
if (result != ISC_R_SUCCESS) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
return (result);
style: one-line statement braces, line length, etc
2018-11-08 19:34:51 -08:00
}
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
}
3137. [func] Improve hardware scalability by allowing multiple worker threads to process incoming UDP packets. This can significantly increase query throughput on some systems. [RT #22992]
2011-07-28 04:04:37 +00:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dispatch_allocate(mgr, isc_socktype_udp, tid, &disp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
if (isc_log_wouldlog(dns_lctx, 90)) {
char addrbuf[ISC_SOCKADDR_FORMATSIZE];
2491. [func] Attempt to re-use a local port if we are already using the port. [RT #18548]
2008-11-12 23:10:57 +00:00
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
isc_sockaddr_format(localaddr, addrbuf,
ISC_SOCKADDR_FORMATSIZE);
mgr_log(mgr, LVL(90),
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
"dispatch_createudp: created UDP dispatch %p for %s",
disp, addrbuf);
1442. [func] New fuctions for manipulating port lists: dns_portlist_create(), dns_portlist_add(), dns_portlist_remove(), dns_portlist_match(), dns_portlist_attach() and dns_portlist_detach(). 1441. [func] It is now possible to tell dig to bind to a specific source port. 1440. [func] It is now possible to tell named to avoid using certian source ports (avoid-v4-udp-ports, avoid-v6-udp-ports). developer: marka reviewer: explorer
2003-02-26 05:05:16 +00:00
}
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
when code is duplicated (which it shoudn't be), it should at least be duplicated consistently
2001-01-07 22:02:48 +00:00
disp->local = *localaddr;
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
/*
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
* Don't append it to the dispatcher list, we don't care about UDP, only
* TCP should be searched
*
* ISC_LIST_APPEND(mgr->list, disp, link);
Merge the mlg-20000518 branch onto the mainline. Change summary: dns_dispatch_create() no longer exists. dns_dispatch_createtcp() and dns_dispatch_getudp() are the replacements. _createtcp() takes a bound, connected TCP socket, while _getudp() will search for a sharable UDP socket, and if found, attach to it and return a pointer to it. If one is not found, it will create a udp socket, bind it to a supplied local address, and create a new dispatcher around it. dns_dispatch_remove{request,response}() no longer take the dispatch as an argument. query-source can now be set per view. The dispatch manager holds onto three memory pools, one for allocating dispatchers from, one for events, and one for requests/replies. The free list on these pools is hard-coded, but set to 1024. This keeps us from having to dig into the isc_mem_t the pools draw from as often. dns_resolver_create() and dns_view_createresolver() require that valid dispatchers be passed in; dispatchers are no longer created for the caller.
2000-05-19 21:46:46 +00:00
*/
*dispp = disp;
3137. [func] Improve hardware scalability by allowing multiple worker threads to process incoming UDP packets. This can significantly increase query throughput on some systems. [RT #22992]
2011-07-28 04:04:37 +00:00
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
return (result);
checkpoint
1999-06-16 01:32:31 +00:00
}
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
static void
dispatch_destroy_rcu(struct rcu_head *rcu_head) {
dns_dispatch_t *disp = caa_container_of(rcu_head, dns_dispatch_t,
rcu_head);
isc_mem_putanddetach(&disp->mctx, disp, sizeof(*disp));
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
static void
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
dispatch_destroy(dns_dispatch_t *disp) {
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
dns_dispatchmgr_t *mgr = disp->mgr;
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
uint32_t tid = isc_tid();
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_refcount_destroy(&disp->references);
disp->magic = 0;
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
if (disp->socktype == isc_socktype_tcp) {
(void)cds_lfht_del(mgr->tcps[tid], &disp->ht_node);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(disp->requests == 0);
INSIST(ISC_LIST_EMPTY(disp->pending));
INSIST(ISC_LIST_EMPTY(disp->active));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispatch_log(disp, LVL(90), "destroying dispatch %p", disp);
if (disp->handle) {
dispatch_log(disp, LVL(90), "detaching TCP handle %p from %p",
disp->handle, &disp->handle);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
isc_nmhandle_detach(&disp->handle);
}
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
dns_dispatchmgr_detach(&disp->mgr);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
call_rcu(&disp->rcu_head, dispatch_destroy_rcu);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DNS_DISPATCH_TRACE
ISC_REFCOUNT_TRACE_IMPL(dns_dispatch, dispatch_destroy);
#else
ISC_REFCOUNT_IMPL(dns_dispatch, dispatch_destroy);
#endif
checkpoint
1999-06-16 01:32:31 +00:00
fix many bugs.
1999-07-22 01:34:31 +00:00
isc_result_t
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
dns_dispatch_add(dns_dispatch_t *disp, isc_loop_t *loop, unsigned int options,
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
unsigned int timeout, const isc_sockaddr_t *dest,
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
dns_transport_t *transport, isc_tlsctx_cache_t *tlsctx_cache,
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
dispatch_cb_t connected, dispatch_cb_t sent,
dispatch_cb_t response, void *arg, dns_messageid_t *idp,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_t **respp) {
dns_dispentry_t *resp = NULL;
Honour the source-port when retrying in dns_dispatch When retrying in the DNS dispatch, the local port would be forgotten on ISC_R_ADDRINUSE, keep the configured source-port even when retrying. Additionally, treat ISC_R_NOPERM same as ISC_R_ADDRINUSE. Closes: #3986
2023-04-05 15:28:32 +02:00
in_port_t localport;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
int i = 0;
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
isc_result_t result = ISC_R_UNSET;
checkpoint
1999-06-18 02:01:42 +00:00
REQUIRE(VALID_DISPATCH(disp));
REQUIRE(dest != NULL);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(respp != NULL && *respp == NULL);
checkpoint
1999-06-18 02:01:42 +00:00
REQUIRE(idp != NULL);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
REQUIRE(disp->socktype == isc_socktype_tcp ||
disp->socktype == isc_socktype_udp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(connected != NULL);
REQUIRE(response != NULL);
REQUIRE(sent != NULL);
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
REQUIRE(loop != NULL);
checkpoint
1999-06-18 02:01:42 +00:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
checkpoint
1999-06-18 02:01:42 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (disp->state == DNS_DISPATCHSTATE_CANCELED) {
return (ISC_R_CANCELED);
checkpoint
1999-07-09 02:47:55 +00:00
}
Honour the source-port when retrying in dns_dispatch When retrying in the DNS dispatch, the local port would be forgotten on ISC_R_ADDRINUSE, keep the configured source-port even when retrying. Additionally, treat ISC_R_NOPERM same as ISC_R_ADDRINUSE. Closes: #3986
2023-04-05 15:28:32 +02:00
localport = isc_sockaddr_getport(&disp->local);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
resp = isc_mem_get(disp->mctx, sizeof(*resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
*resp = (dns_dispentry_t){
.port = localport,
.timeout = timeout,
.peer = *dest,
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
.loop = loop,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
.connected = connected,
.sent = sent,
.response = response,
.arg = arg,
.alink = ISC_LINK_INITIALIZER,
.plink = ISC_LINK_INITIALIZER,
.rlink = ISC_LINK_INITIALIZER,
.magic = RESPONSE_MAGIC,
};
#if DNS_DISPATCH_TRACE
fprintf(stderr, "dns_dispentry__init:%s:%s:%d:%p->references = 1\n",
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
__func__, __FILE__, __LINE__, resp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#endif
isc_refcount_init(&resp->references, 1); /* DISPENTRY000 */
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
if (disp->socktype == isc_socktype_udp) {
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
result = setup_socket(disp, resp, dest, &localport);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
if (result != ISC_R_SUCCESS) {
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
isc_mem_put(disp->mctx, resp, sizeof(*resp));
2541. [bug] Conditionally update dispatch manager statistics. [RT #19247]
2009-01-31 00:10:24 +00:00
inc_stats(disp->mgr, dns_resstatscounter_dispsockfail);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
return (result);
}
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_lock();
silence compiler warning
2014-01-09 15:57:59 +11:00
do {
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
/*
* Try somewhat hard to find a unique ID. Start with
* a random number unless DNS_DISPATCHOPT_FIXEDID is set,
* in which case we start with the ID passed in via *idp.
*/
resp->id = ((options & DNS_DISPATCHOPT_FIXEDID) != 0)
? *idp
: (dns_messageid_t)isc_random16();
struct cds_lfht_node *node =
cds_lfht_add_unique(disp->mgr->qids, qid_hash(resp),
qid_match, resp, &resp->ht_node);
if (node != &resp->ht_node) {
result = ISC_R_EXISTS;
if ((options & DNS_DISPATCHOPT_FIXEDID) != 0) {
/*
* When using fixed ID, we either must
* use it or fail
*/
goto fail;
}
} else {
result = ISC_R_SUCCESS;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
break;
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
} while (i++ < QID_MAX_TRIES);
fail:
if (result != ISC_R_SUCCESS) {
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
isc_mem_put(disp->mctx, resp, sizeof(*resp));
s/DNS_R_/ISC_R_/ change for some codes.
2000-04-06 22:03:35 +00:00
return (ISC_R_NOMORE);
checkpoint
1999-06-18 02:01:42 +00:00
}
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
isc_mem_attach(disp->mctx, &resp->mctx);
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
if (transport != NULL) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_transport_attach(transport, &resp->transport);
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
}
if (tlsctx_cache != NULL) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_tlsctx_cache_attach(tlsctx_cache, &resp->tlsctx_cache);
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_attach(disp, &resp->disp); /* DISPATCH001 */
checkpoint
1999-06-18 02:01:42 +00:00
snapshot
1999-07-09 00:51:08 +00:00
disp->requests++;
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
inc_stats(disp->mgr, (disp->socktype == isc_socktype_udp)
? dns_resstatscounter_disprequdp
: dns_resstatscounter_dispreqtcp);
2375. [security] Fully randomize UDP query ports to improve forgery resilience. [RT #17949, #18098]
2008-06-23 19:41:20 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_unlock();
checkpoint
1999-06-18 02:01:42 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
*idp = resp->id;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
*respp = resp;
checkpoint
1999-06-18 02:01:42 +00:00
s/DNS_R_/ISC_R_/ change for some codes.
2000-04-06 22:03:35 +00:00
return (ISC_R_SUCCESS);
checkpoint
1999-06-16 01:32:31 +00:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_t
dns_dispatch_getnext(dns_dispentry_t *resp) {
REQUIRE(VALID_RESPONSE(resp));
REQUIRE(VALID_DISPATCH(resp->disp));
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
isc_result_t result = ISC_R_SUCCESS;
int32_t timeout = -1;
On non-matching answer, check for missed timeout A TCP connection may be held open past its proper timeout if it's receiving a stream of DNS responses that don't match any queries. In this case, we now check whether the oldest query should have timed out.
2021-11-30 09:57:27 +01:00
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
dispentry_log(resp, LVL(90), "getnext for QID %d", resp->id);
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
isc_time_t now = isc_loop_now(resp->loop);
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
timeout = resp->timeout - dispentry_runtime(resp, &now);
if (timeout <= 0) {
return (ISC_R_TIMEDOUT);
}
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (disp->socktype) {
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
case isc_socktype_udp:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
udp_dispatch_getnext(resp, timeout);
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
break;
case isc_socktype_tcp:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_dispatch_getnext(disp, resp, timeout);
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
break;
default:
Simplify way we tag unreachable code with only ISC_UNREACHABLE() Previously, the unreachable code paths would have to be tagged with: INSIST(0); ISC_UNREACHABLE(); There was also older parts of the code that used comment annotation: /* NOTREACHED */ Unify the handling of unreachable code paths to just use: UNREACHABLE(); The UNREACHABLE() macro now asserts when reached and also uses __builtin_unreachable(); when such builtin is available in the compiler.
2021-10-11 12:50:17 +02:00
UNREACHABLE();
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
}
4408. [func] Continue waiting for expected response when we the response we get does not match the request. [RT #41026]
2016-07-11 13:36:16 +10:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
return (result);
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
/*
* NOTE: Must be RCU read locked!
*/
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
udp_dispentry_cancel(dns_dispentry_t *resp, isc_result_t result) {
4408. [func] Continue waiting for expected response when we the response we get does not match the request. [RT #41026]
2016-07-11 13:36:16 +10:00
REQUIRE(VALID_RESPONSE(resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(VALID_DISPATCH(resp->disp));
REQUIRE(VALID_DISPATCHMGR(resp->disp->mgr));
4408. [func] Continue waiting for expected response when we the response we get does not match the request. [RT #41026]
2016-07-11 13:36:16 +10:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
bool respond = false;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90),
"canceling response: %s, %s/%s (%s/%s), "
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
"requests %" PRIuFAST32,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(result), state2str(resp->state),
resp->reading ? "reading" : "not reading",
state2str(disp->state),
disp->reading ? "reading" : "not reading",
disp->requests);
4408. [func] Continue waiting for expected response when we the response we get does not match the request. [RT #41026]
2016-07-11 13:36:16 +10:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (ISC_LINK_LINKED(resp, alink)) {
ISC_LIST_UNLINK(disp->active, resp, alink);
dispatch: Enforce original timeout when calling _getnext() udp_recv() will call dispatch_getnext() if the message received is invalid or doesn't match; we need to reduce the timeout each time this happens so we can't be starved forever by someone sending garbage packets.
2021-10-01 12:53:31 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (resp->state) {
case DNS_DISPATCHSTATE_NONE:
break;
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTING:
break;
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTED:
if (resp->reading) {
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
respond = true;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90), "canceling read on %p",
resp->handle);
isc_nm_cancelread(resp->handle);
}
break;
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CANCELED:
goto unlock;
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
default:
UNREACHABLE();
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dec_stats(disp->mgr, dns_resstatscounter_disprequdp);
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
(void)cds_lfht_del(disp->mgr->qids, &resp->ht_node);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->state = DNS_DISPATCHSTATE_CANCELED;
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
unlock:
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
if (respond) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90), "read callback: %s",
isc_result_totext(result));
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
resp->response(result, NULL, resp->arg);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
refactor dispatch cancellation Renamed some functions for clarity and readability: - dns_dispatch_addresponse() -> dns_dispatch_add() - dns_dispatch_removeresponse() -> dns_dispatch_done() The dns_dispatch_cancel() function now calls dns_dispatch_done() directly, so it is no longer ever necessary to call both functions. dns_dispatch_cancel() is used to terminate dispatch connections that are still pending, while dns_dispatch_done() is used when they are complete.
2021-10-03 15:15:50 -07:00
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
/*
* NOTE: Must be RCU read locked!
*/
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
tcp_dispentry_cancel(dns_dispentry_t *resp, isc_result_t result) {
REQUIRE(VALID_RESPONSE(resp));
REQUIRE(VALID_DISPATCH(resp->disp));
REQUIRE(VALID_DISPATCHMGR(resp->disp->mgr));
checkpoint
1999-06-18 02:01:42 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
dns_displist_t resps = ISC_LIST_INITIALIZER;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90),
"canceling response: %s, %s/%s (%s/%s), "
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
"requests %" PRIuFAST32,
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_totext(result), state2str(resp->state),
resp->reading ? "reading" : "not reading",
state2str(disp->state),
disp->reading ? "reading" : "not reading",
disp->requests);
switch (resp->state) {
case DNS_DISPATCHSTATE_NONE:
break;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTING:
break;
checkpoint
1999-06-18 02:01:42 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTED:
if (resp->reading) {
tcp_recv_add(&resps, resp, ISC_R_CANCELED);
}
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
INSIST(!ISC_LINK_LINKED(resp, alink));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (ISC_LIST_EMPTY(disp->active)) {
INSIST(disp->handle != NULL);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
#if DISPATCH_TCP_KEEPALIVE
/*
* This is an experimental code that keeps the TCP
* connection open for 1 second before it is finally
* closed. By keeping the TCP connection open, it can
* be reused by dns_request that uses
* dns_dispatch_gettcp() to join existing TCP
* connections.
*
* It is disabled for now, because it changes the
* behaviour, but I am keeping the code here for future
* reference when we improve the dns_dispatch to reuse
* the TCP connections also in the resolver.
*
* The TCP connection reuse should be seamless and not
* require any extra handling on the client side though.
*/
isc_nmhandle_cleartimeout(disp->handle);
isc_nmhandle_settimeout(disp->handle, 1000);
if (!disp->reading) {
dispentry_log(resp, LVL(90),
"final 1 second timeout on %p",
disp->handle);
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
tcp_startrecv(disp, NULL);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
#else
if (disp->reading) {
dispentry_log(resp, LVL(90),
"canceling read on %p",
disp->handle);
isc_nm_cancelread(disp->handle);
}
#endif
}
break;
469. [bug] "query-source address * port 53;" now works.
2000-09-18 04:50:05 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CANCELED:
goto unlock;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
default:
UNREACHABLE();
}
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dec_stats(disp->mgr, dns_resstatscounter_dispreqtcp);
checkpoint
1999-06-18 02:01:42 +00:00
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
(void)cds_lfht_del(disp->mgr->qids, &resp->ht_node);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->state = DNS_DISPATCHSTATE_CANCELED;
unlock:
snapshot
1999-07-09 00:51:08 +00:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/*
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
* NOTE: Calling the response callback directly from here should be done
* asynchronously, as the dns_dispatch_done() is usually called directly
* from the response callback, so there's a slight chance that the call
* stack will get higher here, but it's mitigated by the ".reading"
* flag, so we don't ever go into a loop.
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
*/
tcp_recv_processall(&resps, NULL);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
}
static void
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_cancel(dns_dispentry_t *resp, isc_result_t result) {
REQUIRE(VALID_RESPONSE(resp));
REQUIRE(VALID_DISPATCH(resp->disp));
dns_dispatch_t *disp = resp->disp;
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_lock();
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
switch (disp->socktype) {
case isc_socktype_udp:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
udp_dispentry_cancel(resp, result);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
break;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
case isc_socktype_tcp:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
tcp_dispentry_cancel(resp, result);
break;
default:
UNREACHABLE();
}
Rewrite the QID lookup table to cds_lfht Looking up unique message ID in the dns_dispatch has been using custom hash tables. Rewrite the custom hashtable to use cds_lfht API, removing one extra lock in the cold-cache resolver hot path.
2023-09-14 18:01:39 +02:00
rcu_read_unlock();
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
REQUIRE should not have side effects it's a style violation to have REQUIRE or INSIST contain code that must run for the server to work. this was being done with some atomic_compare_exchange calls. these have been cleaned up. uses of atomic_compare_exchange in assertions have been replaced with a new macro atomic_compare_exchange_enforced, which uses RUNTIME_CHECK to ensure that the exchange was successful.
2021-10-21 02:28:48 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
void
dns_dispatch_done(dns_dispentry_t **respp) {
REQUIRE(VALID_RESPONSE(*respp));
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_t *resp = *respp;
*respp = NULL;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_cancel(resp, ISC_R_CANCELED);
dns_dispentry_detach(&resp); /* DISPENTRY000 */
}
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
udp_startrecv(isc_nmhandle_t *handle, dns_dispentry_t *resp) {
REQUIRE(VALID_RESPONSE(resp));
dispentry_log(resp, LVL(90), "attaching handle %p to %p", handle,
&resp->handle);
isc_nmhandle_attach(handle, &resp->handle);
dns_dispentry_ref(resp); /* DISPENTRY003 */
dispentry_log(resp, LVL(90), "reading");
isc_nm_read(resp->handle, udp_recv, resp);
resp->reading = true;
}
static void
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
tcp_startrecv(dns_dispatch_t *disp, dns_dispentry_t *resp) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(VALID_DISPATCH(disp));
REQUIRE(disp->socktype == isc_socktype_tcp);
dns_dispatch_ref(disp); /* DISPATCH002 */
if (resp != NULL) {
dispentry_log(resp, LVL(90), "reading from %p", disp->handle);
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
INSIST(!isc_time_isepoch(&resp->start));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
} else {
dispatch_log(disp, LVL(90),
"TCP reading without response from %p",
disp->handle);
Move isc_socket_connect() calls into dispatch dns_dispatch_connect() connects a dispatch socket (for TCP) or a dispatch entry socket (for UDP). This is the next step in moving all uses of the isc_socket code into the dispatch module. This API is temporary; it needs to be cleaned up further so that it can be called the same way for both TCP and UDP.
2020-12-19 01:34:41 -08:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_nm_read(disp->handle, tcp_recv, disp);
disp->reading = true;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
}
static void
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
tcp_connected(isc_nmhandle_t *handle, isc_result_t eresult, void *arg) {
dns_dispatch_t *disp = (dns_dispatch_t *)arg;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_t *resp = NULL;
dns_dispentry_t *next = NULL;
dns_displist_t resps = ISC_LIST_INITIALIZER;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (isc_log_wouldlog(dns_lctx, 90)) {
char localbuf[ISC_SOCKADDR_FORMATSIZE];
char peerbuf[ISC_SOCKADDR_FORMATSIZE];
if (handle != NULL) {
isc_sockaddr_t local = isc_nmhandle_localaddr(handle);
isc_sockaddr_t peer = isc_nmhandle_peeraddr(handle);
isc_sockaddr_format(&local, localbuf,
ISC_SOCKADDR_FORMATSIZE);
isc_sockaddr_format(&peer, peerbuf,
ISC_SOCKADDR_FORMATSIZE);
} else {
isc_sockaddr_format(&disp->local, localbuf,
ISC_SOCKADDR_FORMATSIZE);
isc_sockaddr_format(&disp->peer, peerbuf,
ISC_SOCKADDR_FORMATSIZE);
}
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispatch_log(disp, LVL(90), "connected from %s to %s: %s",
localbuf, peerbuf, isc_result_totext(eresult));
}
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix assignment vs comparison typo in tcp_connected() In tcp_connected() a typo has turned a DbC check into an assignment breaking the state machine and making the dns_dispatch_gettcp() try to attach to dispatch in process of destruction.
2022-12-19 18:17:46 +01:00
INSIST(disp->state == DNS_DISPATCHSTATE_CONNECTING);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
/*
* If there are pending responses, call the connect
* callbacks for all of them.
*/
for (resp = ISC_LIST_HEAD(disp->pending); resp != NULL; resp = next) {
next = ISC_LIST_NEXT(resp, plink);
ISC_LIST_UNLINK(disp->pending, resp, plink);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
ISC_LIST_APPEND(resps, resp, rlink);
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
resp->result = eresult;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
if (resp->state == DNS_DISPATCHSTATE_CANCELED) {
resp->result = ISC_R_CANCELED;
} else if (eresult == ISC_R_SUCCESS) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->state = DNS_DISPATCHSTATE_CONNECTED;
ISC_LIST_APPEND(disp->active, resp, alink);
resp->reading = true;
dispentry_log(resp, LVL(90), "start reading");
} else {
resp->state = DNS_DISPATCHSTATE_NONE;
}
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
}
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
if (ISC_LIST_EMPTY(disp->active)) {
/* All responses have been canceled */
disp->state = DNS_DISPATCHSTATE_CANCELED;
} else if (eresult == ISC_R_SUCCESS) {
disp->state = DNS_DISPATCHSTATE_CONNECTED;
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
isc_nmhandle_attach(handle, &disp->handle);
tcp_startrecv(disp, resp);
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
} else {
disp->state = DNS_DISPATCHSTATE_NONE;
}
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
for (resp = ISC_LIST_HEAD(resps); resp != NULL; resp = next) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
next = ISC_LIST_NEXT(resp, rlink);
ISC_LIST_UNLINK(resps, resp, rlink);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90), "connect callback: %s",
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
isc_result_totext(resp->result));
resp->connected(resp->result, NULL, resp->arg);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_detach(&resp); /* DISPENTRY005 */
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
}
Move isc_socket_connect() calls into dispatch dns_dispatch_connect() connects a dispatch socket (for TCP) or a dispatch entry socket (for UDP). This is the next step in moving all uses of the isc_socket code into the dispatch module. This API is temporary; it needs to be cleaned up further so that it can be called the same way for both TCP and UDP.
2020-12-19 01:34:41 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_detach(&disp); /* DISPATCH003 */
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
}
static void
udp_connected(isc_nmhandle_t *handle, isc_result_t eresult, void *arg) {
dns_dispentry_t *resp = (dns_dispentry_t *)arg;
dns_dispatch_t *disp = resp->disp;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90), "connected: %s",
isc_result_totext(eresult));
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (resp->state) {
case DNS_DISPATCHSTATE_CANCELED:
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
eresult = ISC_R_CANCELED;
ISC_LIST_UNLINK(disp->pending, resp, plink);
goto unlock;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTING:
ISC_LIST_UNLINK(disp->pending, resp, plink);
break;
default:
UNREACHABLE();
}
switch (eresult) {
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
case ISC_R_CANCELED:
break;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case ISC_R_SUCCESS:
resp->state = DNS_DISPATCHSTATE_CONNECTED;
udp_startrecv(handle, resp);
break;
Honour the source-port when retrying in dns_dispatch When retrying in the DNS dispatch, the local port would be forgotten on ISC_R_ADDRINUSE, keep the configured source-port even when retrying. Additionally, treat ISC_R_NOPERM same as ISC_R_ADDRINUSE. Closes: #3986
2023-04-05 15:28:32 +02:00
case ISC_R_NOPERM:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case ISC_R_ADDRINUSE: {
Honour the source-port when retrying in dns_dispatch When retrying in the DNS dispatch, the local port would be forgotten on ISC_R_ADDRINUSE, keep the configured source-port even when retrying. Additionally, treat ISC_R_NOPERM same as ISC_R_ADDRINUSE. Closes: #3986
2023-04-05 15:28:32 +02:00
in_port_t localport = isc_sockaddr_getport(&disp->local);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
isc_result_t result;
/* probably a port collision; try a different one */
result = setup_socket(disp, resp, &resp->peer, &localport);
if (result == ISC_R_SUCCESS) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
udp_dispatch_connect(disp, resp);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
goto detach;
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->state = DNS_DISPATCHSTATE_NONE;
break;
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
default:
resp->state = DNS_DISPATCHSTATE_NONE;
break;
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
}
Call the connected dns_dispatch callback asynchronously The dns_request code is very sensitive about calling the connected and deadlocks when the timing is "right" in several places. Move the call to the connected callback to the (udp|tcp)_connected() functions, so they are called asynchronously instead of directly from the (udp|tcp)_dispentry_cancel() functions.
2022-12-20 08:39:36 +01:00
unlock:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_log(resp, LVL(90), "connect callback: %s",
isc_result_totext(eresult));
resp->connected(eresult, NULL, resp->arg);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
detach:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_detach(&resp); /* DISPENTRY004 */
Move isc_socket_connect() calls into dispatch dns_dispatch_connect() connects a dispatch socket (for TCP) or a dispatch entry socket (for UDP). This is the next step in moving all uses of the isc_socket code into the dispatch module. This API is temporary; it needs to be cleaned up further so that it can be called the same way for both TCP and UDP.
2020-12-19 01:34:41 -08:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
static void
udp_dispatch_connect(dns_dispatch_t *disp, dns_dispentry_t *resp) {
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
resp->state = DNS_DISPATCHSTATE_CONNECTING;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp->start = isc_loop_now(resp->loop);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_ref(resp); /* DISPENTRY004 */
ISC_LIST_APPEND(disp->pending, resp, plink);
fix handling of TCP timeouts when a TCP dispatch times out, we call tcp_recv() with a result value of ISC_R_TIMEDOUT; this cancels the oldest dispatch entry in the dispatch's active queue, plus any additional entries that have waited longer than their configured timeouts. if, at that point, there were more dispatch entries still on the active queue, it resumes reading, but until now it failed to restart the timer. this has been corrected: we now calculate a new timeout based on the oldest dispatch entry still remaining. this requires us to initialize the start time of each dispatch entry when it's first added to the queue. in order to ensure that the handling of timed-out requests is consistent, we now calculate the runtime of each dispatch entry based on the same value for 'now'. incidentally also fixed a compile error that turned up when DNS_DISPATCH_TRACE was turned on.
2023-05-16 15:35:00 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_nm_udpconnect(disp->mgr->nm, &resp->local, &resp->peer,
udp_connected, resp, resp->timeout);
}
static isc_result_t
tcp_dispatch_connect(dns_dispatch_t *disp, dns_dispentry_t *resp) {
dns_transport_type_t transport_type = DNS_TRANSPORT_TCP;
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
isc_tlsctx_t *tlsctx = NULL;
isc_tlsctx_client_session_cache_t *sess_cache = NULL;
Move isc_socket_sendto2() calls into dispatch We now use dns_dispatch_send() for this purpose.
2021-01-04 23:03:50 -08:00
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
if (resp->transport != NULL) {
transport_type = dns_transport_get_type(resp->transport);
}
if (transport_type == DNS_TRANSPORT_TLS) {
isc_result_t result;
result = dns_transport_get_tlsctx(
resp->transport, &resp->peer, resp->tlsctx_cache,
Replace the linked list of TCP dispatches with hash table Reusing TCP connections with dns_dispatch_gettcp() used linear linked list to lookup existing outgoing TCP connections that could be reused. Replace the linked list with per-loop cds_lfht hashtable to speedup the lookups. We use cds_lfht because it allows non-unique node insertion that we need to check for dispatches in different connection states.
2023-09-15 14:38:02 +02:00
resp->mctx, &tlsctx, &sess_cache);
Implement TLS transport support for dns_request and dns_dispatch This change prepares ground for sending DNS requests using DoT, which, in particular, will be used for forwarding dynamic updates to TLS-enabled primaries.
2022-09-19 11:04:22 +00:00
if (result != ISC_R_SUCCESS) {
return (result);
}
INSIST(tlsctx != NULL);
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/* Check whether the dispatch is already connecting or connected. */
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (disp->state) {
case DNS_DISPATCHSTATE_NONE:
/* First connection, continue with connecting */
disp->state = DNS_DISPATCHSTATE_CONNECTING;
resp->state = DNS_DISPATCHSTATE_CONNECTING;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp->start = isc_loop_now(resp->loop);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_ref(resp); /* DISPENTRY005 */
ISC_LIST_APPEND(disp->pending, resp, plink);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
char localbuf[ISC_SOCKADDR_FORMATSIZE];
char peerbuf[ISC_SOCKADDR_FORMATSIZE];
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_sockaddr_format(&disp->local, localbuf,
ISC_SOCKADDR_FORMATSIZE);
isc_sockaddr_format(&disp->peer, peerbuf,
ISC_SOCKADDR_FORMATSIZE);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_ref(disp); /* DISPATCH003 */
dispentry_log(resp, LVL(90),
"connecting from %s to %s, timeout %u", localbuf,
peerbuf, resp->timeout);
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
isc_nm_streamdnsconnect(disp->mgr->nm, &disp->local,
&disp->peer, tcp_connected, disp,
resp->timeout, tlsctx, sess_cache);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
break;
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTING:
/* Connection pending; add resp to the list */
resp->state = DNS_DISPATCHSTATE_CONNECTING;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp->start = isc_loop_now(resp->loop);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_ref(resp); /* DISPENTRY005 */
ISC_LIST_APPEND(disp->pending, resp, plink);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
break;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
case DNS_DISPATCHSTATE_CONNECTED:
resp->state = DNS_DISPATCHSTATE_CONNECTED;
use isc_loop_now() for dispentry timeouts store a pointer to the running loop when creating a dispatch entry with dns_dispatch_add(), and use isc_loop_now() to get the timestamp for the current event loop tick when we initialize the dispentry start time and check for timeouts.
2023-05-27 19:20:47 -07:00
resp->start = isc_loop_now(resp->loop);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
/* Add the resp to the reading list */
ISC_LIST_APPEND(disp->active, resp, alink);
dispentry_log(resp, LVL(90), "already connected; attaching");
resp->reading = true;
if (!disp->reading) {
/* Restart the reading */
minor cleanups in dispatch.c - simplified tcp_startrecv() - removed a short function that was only called once - removed an unnecessary if statement
2023-02-21 20:14:30 -08:00
tcp_startrecv(disp, resp);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
}
/* We are already connected; call the connected cb */
dispentry_log(resp, LVL(90), "connect callback: %s",
isc_result_totext(ISC_R_SUCCESS));
resp->connected(ISC_R_SUCCESS, NULL, resp->arg);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
break;
dispatch: Clean up connect and recv callbacks - disp_connected() has been split into two functions, udp_connected() (which takes 'resp' as an argument) and tcp_connected() (which takes 'disp', and calls the connect callbacks for all pending resps). - In dns_dispatch_connect(), if a connection is already open, we need to detach the dispentry immediately because we won't be running tcp_connected(). - dns_disptach_cancel() also now calls the connect callbacks for pending TCP responses, and the response callbacks for open TCP connections waiting on read. - If udp_connected() runs after dns_dispatch_cancel() has been called, ensure that the caller's connect callback is run. - If a UDP connection fails with EADDRINUSE, we try again up to five times with a different local port number before giving up. - If a TCP connection is canceled while still pending connection, the connect timeout may still fire. we attach the dispatch before connecting to ensure that it won't be detached too soon in this case. - The dispentry is no longer removed from the pending list when deactivating, so that the connect callback can still be run if dns_dispatch_removeresponse() was run while the connecting was pending. - Rewrote dns_dispatch_gettcp() to avoid a data race. - startrecv() and dispatch_getnext() can be called with a NULL resp when using TCP. - Refactored udp_recv() and tcp_recv() and added result logging. - EOF is now treated the same as CANCELED in response callbacks. - ISC_R_SHUTTINGDOWN is sent to the reponse callbacks for all resps if tcp_recv() is triggered by a netmgr shutdown. (response callbacks are *not* sent by udp_recv() in this case.)
2021-08-04 13:14:11 -07:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
default:
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
UNREACHABLE();
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
}
Move isc_socket_sendto2() calls into dispatch We now use dns_dispatch_send() for this purpose.
2021-01-04 23:03:50 -08:00
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
return (ISC_R_SUCCESS);
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
isc_result_t
dns_dispatch_connect(dns_dispentry_t *resp) {
REQUIRE(VALID_RESPONSE(resp));
REQUIRE(VALID_DISPATCH(resp->disp));
dns_dispatch_t *disp = resp->disp;
switch (disp->socktype) {
case isc_socktype_tcp:
return (tcp_dispatch_connect(disp, resp));
case isc_socktype_udp:
udp_dispatch_connect(disp, resp);
return (ISC_R_SUCCESS);
default:
UNREACHABLE();
}
}
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
static void
send_done(isc_nmhandle_t *handle, isc_result_t result, void *cbarg) {
dns_dispentry_t *resp = (dns_dispentry_t *)cbarg;
REQUIRE(VALID_RESPONSE(resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
REQUIRE(VALID_DISPATCH(disp));
dispentry_log(resp, LVL(90), "sent: %s", isc_result_totext(result));
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
resp->sent(result, NULL, resp->arg);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
if (result != ISC_R_SUCCESS) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dispentry_cancel(resp, result);
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_detach(&resp); /* DISPENTRY007 */
isc_nmhandle_detach(&handle);
}
static void
tcp_dispatch_getnext(dns_dispatch_t *disp, dns_dispentry_t *resp,
int32_t timeout) {
REQUIRE(timeout <= INT16_MAX);
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
dispentry_log(resp, LVL(90), "continue reading");
if (!resp->reading) {
ISC_LIST_APPEND(disp->active, resp, alink);
resp->reading = true;
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
if (disp->reading) {
return;
}
if (timeout > 0) {
isc_nmhandle_settimeout(disp->handle, timeout);
}
dns_dispatch_ref(disp); /* DISPATCH002 */
isc_nm_read(disp->handle, tcp_recv, disp);
disp->reading = true;
}
static void
udp_dispatch_getnext(dns_dispentry_t *resp, int32_t timeout) {
REQUIRE(timeout <= INT16_MAX);
if (resp->reading) {
return;
}
if (timeout > 0) {
isc_nmhandle_settimeout(resp->handle, timeout);
}
dispentry_log(resp, LVL(90), "continue reading");
dns_dispentry_ref(resp); /* DISPENTRY003 */
isc_nm_read(resp->handle, udp_recv, resp);
resp->reading = true;
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
}
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
void
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
dns_dispatch_resume(dns_dispentry_t *resp, uint16_t timeout) {
REQUIRE(VALID_RESPONSE(resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(VALID_DISPATCH(resp->disp));
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
fix a bug in dns_dispatch_getnext() when a message arrives over a TCP connection matching an expected QID, the dispatch is updated so it no longer expects that QID, but continues reading. subsequent messages with the same QID are ignored, unless the dispatch entry has called dns_dispatch_getnext() or dns_dispatch_resume(). however, a coding error caused those functions to have no effect when the dispatch was reading, so streams of messages with the same QID could not be received over a single TCP connection, breaking *XFR. this has been corrected by changing the order of operations in tcp_dispatch_getnext() so that disp->reading isn't checked until after the dispatch entry has been reactivated.
2023-02-22 11:04:12 -08:00
dispentry_log(resp, LVL(90), "resume");
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
REQUIRE(disp->tid == isc_tid());
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (disp->socktype) {
case isc_socktype_udp: {
udp_dispatch_getnext(resp, timeout);
break;
}
case isc_socktype_tcp:
INSIST(disp->timedout > 0);
disp->timedout--;
tcp_dispatch_getnext(disp, resp, timeout);
break;
default:
UNREACHABLE();
}
dispatch: Remove 'timeout' callback - It is no longer necessary to pass a 'timeout' callback to dns_dispatch_addresponse(); timeouts are handled directly by the 'response' callback instead. - The netmgr handle is no longer passed to dispatch callbacks, since they don't (and can't) use it. instead, dispatch_cb_t now takes a result code, region, and argument. - Cleaned up timeout-related tests in dispatch_test.c
2021-08-03 15:27:06 +02:00
}
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
void
remove nonfunctional DSCP implementation DSCP has not been fully working since the network manager was introduced in 9.16, and has been completely broken since 9.18. This seems to have caused very few difficulties for anyone, so we have now marked it as obsolete and removed the implementation. To ensure that old config files don't fail, the code to parse dscp key-value pairs is still present, but a warning is logged that the feature is obsolete and should not be used. Nothing is done with configured values, and there is no longer any range checking.
2023-01-05 22:18:55 -08:00
dns_dispatch_send(dns_dispentry_t *resp, isc_region_t *r) {
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
REQUIRE(VALID_RESPONSE(resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(VALID_DISPATCH(resp->disp));
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
isc_nmhandle_t *sendhandle = NULL;
dispentry_log(resp, LVL(90), "sending");
switch (disp->socktype) {
case isc_socktype_udp:
isc_nmhandle_attach(resp->handle, &sendhandle);
break;
case isc_socktype_tcp:
isc_nmhandle_attach(disp->handle, &sendhandle);
break;
default:
UNREACHABLE();
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
}
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispentry_ref(resp); /* DISPENTRY007 */
isc_nm_send(sendhandle, r, send_done, resp);
Move isc_socket_sendto2() calls into dispatch We now use dns_dispatch_send() for this purpose.
2021-01-04 23:03:50 -08:00
}
add dns_dispatch_getlocaladdress().
2001-03-13 05:48:41 +00:00
isc_result_t
dns_dispatch_getlocaladdress(dns_dispatch_t *disp, isc_sockaddr_t *addrp) {
REQUIRE(VALID_DISPATCH(disp));
REQUIRE(addrp != NULL);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
if (disp->socktype == isc_socktype_udp) {
add dns_dispatch_getlocaladdress().
2001-03-13 05:48:41 +00:00
*addrp = disp->local;
return (ISC_R_SUCCESS);
}
return (ISC_R_NOTIMPLEMENTED);
}
Move isc_socket_getsockname() calls into dispatch We now use dns_dispentry_getlocaladdress(). (this API is likely to be cleaned up further later.)
2021-01-04 14:38:35 -08:00
isc_result_t
dns_dispentry_getlocaladdress(dns_dispentry_t *resp, isc_sockaddr_t *addrp) {
REQUIRE(VALID_RESPONSE(resp));
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
REQUIRE(VALID_DISPATCH(resp->disp));
Move isc_socket_getsockname() calls into dispatch We now use dns_dispentry_getlocaladdress(). (this API is likely to be cleaned up further later.)
2021-01-04 14:38:35 -08:00
REQUIRE(addrp != NULL);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_t *disp = resp->disp;
Move isc_socket_getsockname() calls into dispatch We now use dns_dispentry_getlocaladdress(). (this API is likely to be cleaned up further later.)
2021-01-04 14:38:35 -08:00
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
switch (disp->socktype) {
case isc_socktype_tcp:
*addrp = disp->local;
return (ISC_R_SUCCESS);
case isc_socktype_udp:
Refactor dispatch, resolver and request Since every dispsock was associated with a dispentry anyway (though not always vice versa), the members of dispsock have been combined into dispentry, which is now reference-counted. dispentry objects are now attached before connecting and detached afterward to prevent races between the connect callback and dns_dispatch_removeresponse(). Dispatch and dispatchmgr objects are now reference counted as well, and the shutdown process has been simplified. reference counting of resquery and request objects has also been cleaned up significantly. dns_dispatch_cancel() now flags a dispentry as having been canceled, so that if the connect callback runs after cancellation, it will not initiate a read. The isblackholed() function has been simplified.
2021-05-25 22:54:17 -07:00
*addrp = isc_nmhandle_localaddr(resp->handle);
Convert dispatch to netmgr The flow of operations in dispatch is changing and will now be similar for both UDP and TCP queries: 1) Call dns_dispatch_addresponse() to assign a query ID and register that we'll be listening for a response with that ID soon. the parameters for this function include callback functions to inform the caller when the socket is connected and when the message has been sent, as well as a task action that will be sent when the response arrives. (later this could become a netmgr callback, but at this stage to minimize disruption to the calling code, we continue to use isc_task for the response event.) on successful completion of this function, a dispatch entry object will be instantiated. 2) Call dns_dispatch_connect() on the dispatch entry. this runs isc_nm_udpconnect() or isc_nm_tcpdnsconnect(), as needed, and begins listening for responses. the caller is informed via a callback function when the connection is established. 3) Call dns_dispatch_send() on the dispatch entry. this runs isc_nm_send() to send a request. 4) Call dns_dispatch_removeresponse() to terminate listening and close the connection. Implementation comments below: - As we will be using netmgr buffers now. code to send the length in TCP queries has also been removed as that is handled by the netmgr. - TCP dispatches can be used by multiple simultaneous queries, so dns_dispatch_connect() now checks whether the dispatch is already connected before calling isc_nm_tcpdnsconnect() again. - Running dns_dispatch_getnext() from a non-network thread caused a crash due to assertions in the netmgr read functions that appear to be unnecessary now. the assertions have been removed. - fctx->nqueries was formerly incremented when the connection was successful, but is now incremented when the query is started and decremented if the connection fails. - It's no longer necessary for each dispatch to have a pool of tasks, so there's now a single task per dispatch. - Dispatch code to avoid UDP ports already in use has been removed. - dns_resolver and dns_request have been modified to use netmgr callback functions instead of task events. some additional changes were needed to handle shutdown processing correctly. - Timeout processing is not yet fully converted to use netmgr timeouts. - Fixed a lock order cycle reported by TSAN (view -> zone-> adb -> view) by by calling dns_zt functions without holding the view lock.
2021-01-14 13:02:57 -08:00
return (ISC_R_SUCCESS);
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
default:
UNREACHABLE();
Move isc_socket_getsockname() calls into dispatch We now use dns_dispentry_getlocaladdress(). (this API is likely to be cleaned up further later.)
2021-01-04 14:38:35 -08:00
}
}
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
dns_dispatch_t *
dns_dispatchset_get(dns_dispatchset_t *dset) {
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
uint32_t tid = isc_tid();
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
/* check that dispatch set is configured */
if (dset == NULL || dset->ndisp == 0) {
return (NULL);
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
INSIST(tid < dset->ndisp);
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
return (dset->dispatches[tid]);
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
}
isc_result_t
dispatch: Refactor to eliminate dns_dispatchevent - Responses received by the dispatch are no longer sent to the caller via a task event, but via a netmgr-style recv callback. the 'action' parameter to dns_dispatch_addresponse() is now called 'response' and is called directly from udp_recv() or tcp_recv() when a valid response has been received. - All references to isc_task and isc_taskmgr have been removed from dispatch functions. - All references to dns_dispatchevent_t have been removed and the type has been deleted. - Added a task to the resolver response context, to be used for fctx events. - When the caller cancels an operation, the response handler will be called with ISC_R_CANCELED; it can abort immediately since the caller will presumably have taken care of cleanup already. - Cleaned up attach/detach in resquery and request.
2021-07-26 20:23:18 -07:00
dns_dispatchset_create(isc_mem_t *mctx, dns_dispatch_t *source,
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dns_dispatchset_t **dsetp, uint32_t ndisp) {
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
isc_result_t result;
General code refactoring - style cleanup - removed NULL checks in places where they are not currently needed - use isc_refcount for dispatch reference counting - revised code flow for readability - remove some #ifdefs that are no longer relevant - remove unused struct members - removed unnecessary function parameters - use C99 struct initialization
2020-12-09 19:44:41 -08:00
dns_dispatchset_t *dset = NULL;
dns_dispatchmgr_t *mgr = NULL;
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
size_t i;
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
REQUIRE(VALID_DISPATCH(source));
dispatch: Fix several connect-related issues - startrecv() and getnext() have been rewritten. - Don't set TCP flag when connecting a UDP dispatch. - Prevent TCP connections from trying to connect twice. - dns_dispatch_gettcp() can now find a matching TCP dispatch that has not yet fully connected, and attach to it. when the connection is completed, the connect callbacks are run for all of the pending entries. - An atomic 'state' variable is now used for connection state instead of attributes. - When dns_dispatch_cancel() is called on a TCP dispatch entry, only that one entry is canceled. the dispatch itself should not be shut down until there are no dispatch entries left associated with it. - Other incidental cleanup, including removing DNS_DISPATCHATTR_IPV4 and _IPV6 (they were being set in the dispatch attributes but never used), cleaning up dns_requestmgr_create(), and renaming dns_dispatch_read() to the more descriptive dns_dispatch_resume().
2021-08-03 18:24:27 -07:00
REQUIRE(source->socktype == isc_socktype_udp);
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
REQUIRE(dsetp != NULL && *dsetp == NULL);
mgr = source->mgr;
dset = isc_mem_get(mctx, sizeof(dns_dispatchset_t));
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
*dset = (dns_dispatchset_t){ .ndisp = ndisp };
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
isc_mem_attach(mctx, &dset->mctx);
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dset->dispatches = isc_mem_cget(dset->mctx, ndisp,
sizeof(dns_dispatch_t *));
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
dset->dispatches[0] = NULL;
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_attach(source, &dset->dispatches[0]); /* DISPATCH004 */
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
for (i = 1; i < dset->ndisp; i++) {
result = dispatch_createudp(mgr, &source->local, i,
Remove support for shared UDP dispatch sockets Currently the netmgr doesn't support unconnected, shared UDP sockets, so there's no reason to retain that functionality in the dispatcher prior to porting to the netmgr. In this commit, the DNS_DISPATCHATTR_EXCLUSIVE attribute has been removed as it is now non-optional; UDP dispatches are alwasy exclusive. Code implementing non-exclusive UDP dispatches has been removed. dns_dispatch_getentrysocket() now always returns the dispsocket for UDP dispatches and the dispatch socket for TCP dispatches. There is no longer any need to search for existing dispatches from dns_dispatch_getudp(), so the 'mask' option has been removed, and the function renamed to the more descriptive dns_dispatch_createudp().
2020-12-16 01:32:06 -08:00
&dset->dispatches[i]);
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
if (result != ISC_R_SUCCESS) {
goto fail;
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
}
*dsetp = dset;
return (ISC_R_SUCCESS);
fail:
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
for (size_t j = 0; j < i; j++) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_detach(&(dset->dispatches[j])); /* DISPATCH004 */
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
isc_mem_cput(dset->mctx, dset->dispatches, ndisp,
sizeof(dns_dispatch_t *));
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
isc_mem_putanddetach(&dset->mctx, dset, sizeof(dns_dispatchset_t));
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
return (result);
}
void
dns_dispatchset_destroy(dns_dispatchset_t **dsetp) {
REQUIRE(dsetp != NULL && *dsetp != NULL);
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
dns_dispatchset_t *dset = *dsetp;
Clear the pointer to destroyed object early using the semantic patch Also disable the semantic patch as the code needs tweaks here and there because some destroy functions might not destroy the object and return early if the object is still in use.
2020-02-08 04:37:54 -08:00
*dsetp = NULL;
Make dns_dispatch bound to threads Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the dispatches bound to threads and make dns_dispatchset_t create a dispatch for each thread (event loop). This required couple of other changes: 1. The dns_dispatch_createudp() must be called on loop, so the isc_tid() is already initialized - changes to nsupdate and mdig were required. 2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead of using single dispatch, use dns_dispatchset_t for each protocol - this is same as dns_resolver.
2023-09-15 11:36:28 +02:00
for (size_t i = 0; i < dset->ndisp; i++) {
Fix the thread safety in the dns_dispatch unit The dispatches are not thread-bound, and used freely between various threads (see the dns_resolver and dns_request units for details). This refactoring make sure that all non-const dns_dispatch_t and dns_dispentry_t members are accessed under a lock, and both object now track their internal state (NONE, CONNECTING, CONNECTED, CANCELED) instead of guessing the state from the state of various struct members. During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on UDP sockets per dispatch was removed as the limiting needs to happen and happens on in dns_resolver and limiting the number of UDP sockets artificially in dispatch could lead to unpredictable behaviour in case one dispatch has the limit exhausted by others are idle. The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less sense as the TCP connections are only reused in the dns_request API that's not a heavy user of the outgoing connections. As a side note, the fact that UDP and TCP dispatch pretends to be same thing, but in fact the connected UDP is handled from dns_dispentry_t and dns_dispatch_t acts as a broker, but connected TCP is handled from dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really help the clarity of this unit. This refactoring kept to API almost same - only dns_dispatch_cancel() and dns_dispatch_done() were merged into dns_dispatch_done() as we need to cancel active netmgr handles in any case to not leave dangling connections around. The functions handling UDP and TCP have been mostly split to their matching counterparts and the dns_dispatch_<function> functions are now thing wrappers that call <udp|tcp>_dispatch_<function> based on the socket type. More debugging-level logging was added to the unit to accomodate for this fact.
2022-11-30 17:58:35 +01:00
dns_dispatch_detach(&(dset->dispatches[i])); /* DISPATCH004 */
Use clang-tidy to add curly braces around one-line statements The command used to reformat the files in this commit was: ./util/run-clang-tidy \ -clang-tidy-binary clang-tidy-11 -clang-apply-replacements-binary clang-apply-replacements-11 \ -checks=-*,readability-braces-around-statements \ -j 9 \ -fix \ -format \ -style=file \ -quiet clang-format -i --style=format $(git ls-files '*.c' '*.h') uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h') clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 21:48:23 +01:00
}
Apply the isc_mem_cget semantic patch
2023-08-23 08:56:31 +02:00
isc_mem_cput(dset->mctx, dset->dispatches, dset->ndisp,
sizeof(dns_dispatch_t *));
create and use multiple fetch dispatches Added API to create a set of UDP dispatches which can be shared round-robin style when making upstream queries for authoritative data; this should reduce lock contention in the query source dispatch.
2012-04-27 16:07:24 -07:00
isc_mem_putanddetach(&dset->mctx, dset, sizeof(dns_dispatchset_t));
}
refactor dns_xfrin to use dns_dispatch the dns_xfrin module was still using the network manager directly to manage TCP connections and send and receive messages. this commit changes it to use the dispatch manager instead.
2023-02-21 20:14:30 -08:00
isc_result_t
dns_dispatch_checkperm(dns_dispatch_t *disp) {
REQUIRE(VALID_DISPATCH(disp));
if (disp->handle == NULL || disp->socktype == isc_socktype_udp) {
return (ISC_R_NOPERM);
}
return (isc_nm_xfr_checkperm(disp->handle));
}
Reference in New Issue Copy Permalink