2
0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-28 21:17:54 +00:00

31006 Commits

Author SHA1 Message Date
Michał Kępień
b25e6b51f6 Make a sed script in doc/arm/Makefile.in portable
BSD sed does not recognize \s as a whitespace matching token.  Make the
sed script in doc/arm/Makefile.in which ensures GitLab identifiers are
not split across lines portable by replacing \s with [[:space:]].
2020-02-20 12:23:36 +01:00
Michał Kępień
cc62ae9aaa Merge branch 'michal/increase-lifetime-of-docs-sid-amd64-artifacts' into 'master'
Increase lifetime of docs:sid:amd64 artifacts

See merge request isc-projects/bind9!3088
2020-02-20 10:55:19 +00:00
Michał Kępień
9751ba5a75 Increase lifetime of docs:sid:amd64 artifacts
Artifacts generated by the docs:sid:amd64 job need to be retained longer
than for other jobs as they are used for building bind.isc.org contents.
If these artifacts are removed too quickly, pipelines in the pages/bind
GitLab project start failing, preventing content updates from being
published.  Increase lifetime of the relevant job artifacts to prevent
this from happening.
2020-02-20 11:51:58 +01:00
Michał Kępień
192c7c29f9 Merge branch 'michal/fix-lib-isc-tests-socket_test-hangs' into 'master'
Fix lib/isc/tests/socket_test hangs

See merge request isc-projects/bind9!3072
2020-02-20 10:47:25 +00:00
Witold Krecicki
0fe149b2fa Fix lib/isc/tests/socket_test hangs 2020-02-20 11:39:15 +01:00
Witold Krecicki
dcb38db695 Merge branch 'wpk/perfork-2-libuv-tuning' into 'master'
Perfwork 2/6 - libuv tuning - support for uv_{recv/send}mmsg, use of libuv-provided uv_export/import

See merge request isc-projects/bind9!3066
2020-02-18 12:08:16 +00:00
Witold Kręcicki
093af1a609 Use libuv-provided uv_{export,import} if available.
We were using our own versions of isc_uv_{export,import} functions
for multithreaded TCP listeners. Upcoming libuv version will
contain proper uv_{export,import} functions - use them if they're
available.
2020-02-18 12:17:55 +01:00
Witold Kręcicki
a0d36d7601 Make nm->recvbuf larger and heap allocated, to allow uv_recvmmsg usage.
Upcoming version of libuv will suport uv_recvmmsg and uv_sendmmsg. To
use uv_recvmmsg we need to provide a larger buffer and be able to
properly free it.
2020-02-18 12:17:55 +01:00
Witold Krecicki
b9c660a023 Merge branch 'wpk/perfwork-1-client-mctxpool-taskpool' into 'master'
Perfwork 1/6 - Use thread-sharded mctxpool and taskpool in ns_client

See merge request isc-projects/bind9!3065
2020-02-18 11:13:07 +00:00
Witold Kręcicki
952f7b503d Use thread-friendly mctxpool and taskpool in ns_client.
Make ns_client mctxpool more thread-friendly by sharding it by
netmgr threadid, use task pool also sharded by thread id to avoid
lock contention.
2020-02-18 10:31:13 +01:00
Witold Kręcicki
23bd04d2f1 Make isc_task_pause/isc_task_unpause thread safe.
isc_task_pause/unpause were inherently thread-unsafe - a task
could be paused only once by one thread, if the task was running
while we paused it it led to races. Fix it by making sure that
the task will pause if requested to, and by using a 'pause reference
counter' to count task pause requests - a task will be unpaused
iff all threads unpause it.

Don't remove from queue when pausing task - we lock the queue lock
(expensive), while it's unlikely that the task will be running -
and we'll remove it anyway in dispatcher
2020-02-18 09:22:04 +00:00
Evan Hunt
b9a2b4cb74 Merge branch 'each-tweak-penalty' into 'master'
adjust the clang-format penalties to reduce string breaking

See merge request isc-projects/bind9!3079
2020-02-17 22:41:54 +00:00
Evan Hunt
0002377dca adjust the clang-format penalties to reduce string breaking
this corrects some style glitches such as:
```
        long_function_call(arg, arg2, arg3, arg4, arg5, "str"
                                                        "ing");
```
...by adjusting the penalties for breaking strings and call
parameter lists.
2020-02-17 14:23:58 -08:00
Ondřej Surý
e4671ef2fa Merge branch 'ondrej/fix-crash-on-arm64-from-weak-cmpxchg' into 'master'
Fix crash on arm64 from using atomic_compare_exchange_weak outside of the loop

See merge request isc-projects/bind9!3042
2020-02-16 17:41:30 +00:00
Ondřej Surý
4ff887db2c Add arm64 to GitLab CI 2020-02-16 18:09:19 +01:00
Ondřej Surý
4cf275ba8a Replace non-loop usage of atomic_compare_exchange_weak with strong variant
While testing BIND 9 on arm64 8+ core machine, it was discovered that
the weak variants in fact does spuriously fail, we haven't observed that
on other architectures.

This commit replaces all non-loop usage of atomic_compare_exchange_weak
with atomic_compare_exchange_strong.
2020-02-16 18:09:19 +01:00
Diego Fronza
fa68a0d869 Added atomic_compare_exchange_strong_acq_rel macro
It is much better to read than:
atomic_compare_exchange_strong_explicit() with 5 arguments.
2020-02-16 18:09:19 +01:00
Ondřej Surý
5e1202d1c9 Merge branch 'ondrej/clang-format-fixups' into 'master'
clang-format fixups

See merge request isc-projects/bind9!3073
2020-02-16 16:45:34 +00:00
Ondřej Surý
3832e3ecc9 Fixup the missing clang-format bits 2020-02-16 17:34:24 +01:00
Ondřej Surý
cbb127eb43 Switch to clang-format-10 that's more accessible on more platforms 2020-02-16 17:34:05 +01:00
Diego dos Santos Fronza
718ec80ed3 Merge branch '1472-threadsanitizer-lock-order-inversion-potential-deadlock-dns_resolver_createfetch-vs' into 'master'
Resolve "ThreadSanitizer: lock-order-inversion (potential deadlock) - dns_resolver_createfetch vs. dns_resolver_prime"

Closes #1472

See merge request isc-projects/bind9!3020
2020-02-14 18:21:12 +00:00
Diego Fronza
45543da802 Fixed disposing of resolver->references in destroy() function 2020-02-14 14:28:31 -03:00
Diego Fronza
e7b36924e2 Fixed potential-lock-inversion
This commit simplifies a bit the lock management within dns_resolver_prime()
and prime_done() functions by means of turning resolver's attribute
"priming" into an atomic_bool and by creating only one dependent object on the
lock "primelock", namely the "primefetch" attribute.

By having the attribute "priming" as an atomic type, it save us from having to
use a lock just to test if priming is on or off for the given resolver context
object, within "dns_resolver_prime" function.

The "primelock" lock is still necessary, since dns_resolver_prime() function
internally calls dns_resolver_createfetch(), and whenever this function
succeeds it registers an event in the task manager which could be called by
another thread, namely the "prime_done" function, and this function is
responsible for disposing the "primefetch" attribute in the resolver object,
also for resetting "priming" attribute to false.

It is important that the invariant "priming == false AND primefetch == NULL"
remains constant, so that any thread calling "dns_resolver_prime" knows for sure
that if the "priming" attribute is false, "primefetch" attribute should also be
NULL, so a new fetch context could be created to fulfill this purpose, and
assigned to "primefetch" attribute under the lock protection.

To honor the explanation above, dns_resolver_prime is implemented as follow:
	1. Atomically checks the attribute "priming" for the given resolver context.
	2. If "priming" is false, assumes that "primefetch" is NULL (this is
           ensured by the "prime_done" implementation), acquire "primelock"
	   lock and create a new fetch context, update "primefetch" pointer to
	   point to the newly allocated fetch context.
	3. If "priming" is true, assumes that the job is already in progress,
	   no locks are acquired, nothing else to do.

To keep the previous invariant consistent, "prime_done" is implemented as follow:
	1. Acquire "primefetch" lock.
	2. Keep a reference to the current "primefetch" object;
	3. Reset "primefetch" attribute to NULL.
	4. Release "primefetch" lock.
	5. Atomically update "priming" attribute to false.
	6. Destroy the "primefetch" object by using the temporary reference.

This ensures that if "priming" is false, "primefetch" was already reset to NULL.

It doesn't make any difference in having the "priming" attribute not protected
by a lock, since the visible state of this variable would depend on the calling
order of the functions "dns_resolver_prime" and "prime_done".

As an example, suppose that instead of using an atomic for the "priming" attribute
we employed a lock to protect it.
Now suppose that "prime_done" function is called by Thread A, it is then preempted
before acquiring the lock, thus not reseting "priming" to false.
In parallel to that suppose that a Thread B is scheduled and that it calls
"dns_resolver_prime()", it then acquires the lock and check that "priming" is true,
thus it will consider that this resolver object is already priming and it won't do
any more job.
Conversely if the lock order was acquired in the other direction, Thread B would check
that "priming" is false (since prime_done acquired the lock first and set "priming" to false)
and it would initiate a priming fetch for this resolver.

An atomic variable wouldn't change this behavior, since it would behave exactly the
same, depending on the function call order, with the exception that it would avoid
having to use a lock.

There should be no side effects resulting from this change, since the previous
implementation employed use of the more general resolver's "lock" mutex, which
is used in far more contexts, but in the specifics of the "dns_resolver_prime"
and "prime_done" it was only used to protect "primefetch" and "priming" attributes,
which are not used in any of the other critical sections protected by the same lock,
thus having zero dependency on those variables.
2020-02-14 14:28:31 -03:00
Diego Fronza
c210413a8a Added atomic_compare_exchange_strong_acq_rel macro
It is much better to read than:
atomic_compare_exchange_strong_explicit() with 5 arguments.
2020-02-14 11:41:36 -03:00
Ondřej Surý
a04cdde45d Merge branch '46-enforce-clang-format-rules' into 'master'
Start enforcing the clang-format rules on changed files

Closes #46

See merge request isc-projects/bind9!3063
2020-02-14 08:45:45 +00:00
Ondřej Surý
60d29f695b Don't enforce copyrights on .clang-format 2020-02-14 09:43:13 +01:00
Ondřej Surý
5777c44ad0 Reformat using the new rules 2020-02-14 09:31:05 +01:00
Ondřej Surý
654927c871 Add separate .clang-format files for headers 2020-02-14 09:31:05 +01:00
Ondřej Surý
618947c63b Switch AlwaysBreakAfterReturnType from TopLevelDefinitions to All
The AlwaysBreakAfterReturnType: TopLevelDefinitions was unwrapping
the declarations of the functions in the header files.
2020-02-14 09:07:46 +01:00
Ondřej Surý
d2b5853b8e Start enforcing the clang-format rules on changed files 2020-02-14 09:07:46 +01:00
Ondřej Surý
d3b49b6675 Merge branch 'each-style-tweak' into 'master'
adjust clang-format options to get closer to ISC style

See merge request isc-projects/bind9!3061
2020-02-14 05:35:17 +00:00
Evan Hunt
e851ed0bb5 apply the modified style 2020-02-13 15:05:06 -08:00
Evan Hunt
0255a97473 revise .clang-format and add a C formatting script in util
- add util/cformat.sh, which runs clang-format on all C files with
  the default .clang-format, and on all header files with a slightly
  modified version.
- use correct bracing after multi-line control statements
- stop aligning variable declarations to avoid problems with pointer
  alignment, but retain aligned declarations in header files so that
  struct definitions look cleaner.
- static function prototypes in C files can skip the line break after
  the return type, but function prototypes in header files still have
  the line break.
- don't break-before-brace in function definitions. ISC style calls
  for braces on the same line when function parameters fit on a single
  line, and a line break if they don't, but clang-format doesn't yet
  support that distinction. one-line function definitions are about
  four times more common than multi-line, so let's use the option that
  deviates less.
2020-02-13 15:04:36 -08:00
Ondřej Surý
67b68e06ad Merge branch '46-add-curly-braces' into 'master'
Add curly braces using uncrustify and then reformat with clang-format back

Closes #46

See merge request isc-projects/bind9!3057
2020-02-13 21:28:07 +00:00
Ondřej Surý
056e133c4c Use clang-tidy to add curly braces around one-line statements
The command used to reformat the files in this commit was:

./util/run-clang-tidy \
	-clang-tidy-binary clang-tidy-11
	-clang-apply-replacements-binary clang-apply-replacements-11 \
	-checks=-*,readability-braces-around-statements \
	-j 9 \
	-fix \
	-format \
	-style=file \
	-quiet
clang-format -i --style=format $(git ls-files '*.c' '*.h')
uncrustify -c .uncrustify.cfg --replace --no-backup $(git ls-files '*.c' '*.h')
clang-format -i --style=format $(git ls-files '*.c' '*.h')
2020-02-13 22:07:21 +01:00
Ondřej Surý
d14bb71319 Add copy of run-clang-tidy that can fixup the filepaths 2020-02-13 22:06:05 +01:00
Ondřej Surý
36c6105e4f Use coccinelle to add braces to nested single line statement
Both clang-tidy and uncrustify chokes on statement like this:

for (...)
	if (...)
		break;

This commit uses a very simple semantic patch (below) to add braces around such
statements.

Semantic patch used:

@@
statement S;
expression E;
@@

while (...)
- if (E) S
+ { if (E) { S } }

@@
statement S;
expression E;
@@

for (...;...;...)
- if (E) S
+ { if (E) { S } }

@@
statement S;
expression E;
@@

if (...)
- if (E) S
+ { if (E) { S } }
2020-02-13 21:58:55 +01:00
Michal Nowak
c823ed4f07 Merge branch 'mnowak/coverity-disable-web-tag' into 'master'
Run Coverity Scan only when specific variables are present

See merge request isc-projects/bind9!3050
2020-02-12 14:44:56 +00:00
Michal Nowak
eec29e3477 Run Coverity Scan only when specific variables are present
Submissions to Coverity Scan should be limited to those originated from
release branches and only from a specific schedule which holds
COVERITY_SCAN_PROJECT_NAME and COVERITY_SCAN_TOKEN variables.
2020-02-12 14:44:55 +00:00
Ondřej Surý
7099e79a9b Merge branch '46-just-use-clang-format-to-reformat-sources' into 'master'
Reformat source code with clang-format

Closes #46

See merge request isc-projects/bind9!2156
2020-02-12 14:33:32 +00:00
Ondřej Surý
df6c1f76ad Remove tkey_test (which is no-op anyway) 2020-02-12 15:04:17 +01:00
Ondřej Surý
11341c7688 Update the definition files for Windows 2020-02-12 15:04:17 +01:00
Ondřej Surý
f50b1e0685 Use clang-format to reformat the source files 2020-02-12 15:04:17 +01:00
Ondřej Surý
4c3b063ef8 Import Linux kernel .clang-format with small modifications 2020-02-12 14:01:08 +01:00
Michał Kępień
118692e949 Merge branch 'michal/minor-README-tweaks' into 'master'
Minor README tweaks

See merge request isc-projects/bind9!3036
2020-02-12 10:37:53 +00:00
Michał Kępień
a0349b18e0 Minor CHANGES tweaks 2020-02-12 11:23:48 +01:00
Mark Andrews
677aa65aad Merge branch '1616-autosign-not-waiting-long-enough-for-zone-to-be-signed-v9_11-and-maybe-others-master' into 'master'
wait for apex NSEC3 to be generated

Closes #1616

See merge request isc-projects/bind9!3043
2020-02-12 10:07:27 +00:00
Mark Andrews
dd2830eb97 wait for apex NSEC3 to be generated
(cherry picked from commit c99ad5c8c7d24f7faa339fe86e8ed1515b4a24ce)
2020-02-12 10:02:14 +00:00
Michal Nowak
c380e32e9a Merge branch 'mnowak/coverity2' into 'master'
Add Coverity Scan to CI

See merge request isc-projects/bind9!2979
2020-02-12 09:45:27 +00:00
Michal Nowak
c209ec90f3 Add Coverity Scan to CI
This job requires two CI variables to be set:

  - COVERITY_SCAN_PROJECT_NAME: project name, which is associated with
    the BIND branch for which this job is executed, e.g. "bind-master",

  - COVERITY_SCAN_TOKEN: project token.
2020-02-12 09:45:27 +00:00