diff --git a/benchmarks.md b/benchmarks.md index 3816026..c94ef1b 100644 --- a/benchmarks.md +++ b/benchmarks.md @@ -3,13 +3,10 @@ Collection of hints and links which can be useful for people new to DNS benchmarking. - ### Step 0. Prerequisites -See DNS-OARC 42, DNS Benchmarking 101: Essentials and Common Pitfalls -- [slides](https://indico.dns-oarc.net/event/48/contributions/1033/attachments/991/1943/pspacek.pdf) -- [video recording](https://www.youtube.com/watch?v=ZrMLeyu4gTU&list=PLCAxS3rufJ1fHh8oe9UQ0IwlVfHq30_hh&index=16) +See DNS-OARC 42, DNS Benchmarking 101: Essentials and Common Pitfalls [slides](https://indico.dns-oarc.net/event/48/contributions/1033/attachments/991/1943/pspacek.pdf), [video recording](https://www.youtube.com/watch?v=ZrMLeyu4gTU&list=PLCAxS3rufJ1fHh8oe9UQ0IwlVfHq30_hh&index=16) -- Test design must be different for any combination of +- Test design **must be** different for any combination of - resolver - authoritative server - normal traffic @@ -22,7 +19,7 @@ See DNS-OARC 42, DNS Benchmarking 101: Essentials and Common Pitfalls - user-space: https://github.com/DNS-OARC/dumdumd/ - XDP (UDP-only): https://gitlab.nic.cz/knot/xdp-utils/ -- Usual tuning tips apply: +- Usual tuning tips apply also to an echo server - beware of [NUMA domains](https://en.wikipedia.org/wiki/Non-uniform_memory_access) in hardware - you _might_ want to restrict yourself to an _inside_ of one NUMA domain - commands [numactl](https://man.archlinux.org/man/numactl.8), [taskset](https://man.archlinux.org/man/taskset.1) - pick the NUMA domain so it is directly connected to network card in use - tool [lstopo](https://man.archlinux.org/man/lstopo-no-graphics.1.en) can help with that - find out how many network card queues is available - command [ethtool -l devicename](https://man.archlinux.org/man/ethtool.8) @@ -30,8 +27,16 @@ See DNS-OARC 42, DNS Benchmarking 101: Essentials and Common Pitfalls - these are just a starting point, experiment! ### Resolvers -- An example with explanation how and why we test: https://www.isc.org/blogs/bind-resolver-performance-july-2021/ +- An example with explanation why and how we test resolvers: https://www.isc.org/blogs/bind-resolver-performance-july-2021/ - **:warning: Results cannot be generalized to other data sets** or setups with any certainty - Why? Because every single query (and it's timing) changes _state_ of the system under test. Not to speak of dependency on authoritative server performance ... -- Introduction to DNS Shotgun [slides](https://ripe79.ripe.net/wp-content/uploads/presentations/45-benchmarking.pdf), [video](https://ripe79.ripe.net/archives/video/198) \ No newline at end of file +- Introduction to DNS Shotgun test tool [slides](https://ripe79.ripe.net/wp-content/uploads/presentations/45-benchmarking.pdf), [video](https://ripe79.ripe.net/archives/video/198) + +- To run resolver test with real data set, using DNS Shotgun, in our Gitlab CI: + - Go to https://gitlab.isc.org/isc-projects/bind9-shotgun-ci/-/pipelines/new and fill in the form: + - `SHOTGUN_TEST_VERSION`: versions to test, specify multiple versions e.g. like `["main", "another_branch_for_testing"]` (tags and commit hashes also work) + - `SHOTGUN_TRAFFIC_MULTIPLIER`: Load _factor_. For normal resolver play in range keep 10-20. + - `SHOTGUN_ROUNDS`: How many times individual test should be repeated. Use at least 3 if you want sensible results. + - `SHOTGUN_DURATION`: Test duration in seconds, normally most cache misses happen in first minute. Use e.g. 300 if you are also interested in hot-cache behavior. + - Click `[Run pipeline]` button at the bottom of the form \ No newline at end of file