mir/bind
2
0
Fork 0
mirror of https://gitlab.isc.org/isc-projects/bind9 synced 2025-08-29 13:38:26 +00:00
Code Issues Releases Wiki Activity

Update benchmarks: add Shotgun CI

Petr Špaček 2024-09-12 10:25:19 +00:00
parent dd963b237e
commit e2a5db6978
1 changed files with 13 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
benchmarks.md

@ -3,13 +3,10 @@
Collection of hints and links which can be useful for people new to DNS benchmarking. Collection of hints and links which can be useful for people new to DNS benchmarking.
### Step 0. Prerequisites ### Step 0. Prerequisites
See DNS-OARC 42, DNS Benchmarking 101: Essentials and Common Pitfalls See DNS-OARC 42, DNS Benchmarking 101: Essentials and Common Pitfalls [slides](https://indico.dns-oarc.net/event/48/contributions/1033/attachments/991/1943/pspacek.pdf), [video recording](https://www.youtube.com/watch?v=ZrMLeyu4gTU&list=PLCAxS3rufJ1fHh8oe9UQ0IwlVfHq30_hh&index=16)
- [slides](https://indico.dns-oarc.net/event/48/contributions/1033/attachments/991/1943/pspacek.pdf)
- [video recording](https://www.youtube.com/watch?v=ZrMLeyu4gTU&list=PLCAxS3rufJ1fHh8oe9UQ0IwlVfHq30_hh&index=16)
- Test design must be different for any combination of - Test design **must be** different for any combination of
- resolver - resolver
- authoritative server - authoritative server
- normal traffic - normal traffic
@ -22,7 +19,7 @@ See DNS-OARC 42, DNS Benchmarking 101: Essentials and Common Pitfalls
- user-space: https://github.com/DNS-OARC/dumdumd/ - user-space: https://github.com/DNS-OARC/dumdumd/
- XDP (UDP-only): https://gitlab.nic.cz/knot/xdp-utils/ - XDP (UDP-only): https://gitlab.nic.cz/knot/xdp-utils/
- Usual tuning tips apply: - Usual tuning tips apply also to an echo server
- beware of [NUMA domains](https://en.wikipedia.org/wiki/Non-uniform_memory_access) in hardware - you _might_ want to restrict yourself to an _inside_ of one NUMA domain - commands [numactl](https://man.archlinux.org/man/numactl.8), [taskset](https://man.archlinux.org/man/taskset.1) - beware of [NUMA domains](https://en.wikipedia.org/wiki/Non-uniform_memory_access) in hardware - you _might_ want to restrict yourself to an _inside_ of one NUMA domain - commands [numactl](https://man.archlinux.org/man/numactl.8), [taskset](https://man.archlinux.org/man/taskset.1)
- pick the NUMA domain so it is directly connected to network card in use - tool [lstopo](https://man.archlinux.org/man/lstopo-no-graphics.1.en) can help with that - pick the NUMA domain so it is directly connected to network card in use - tool [lstopo](https://man.archlinux.org/man/lstopo-no-graphics.1.en) can help with that
- find out how many network card queues is available - command [ethtool -l devicename](https://man.archlinux.org/man/ethtool.8) - find out how many network card queues is available - command [ethtool -l devicename](https://man.archlinux.org/man/ethtool.8)
@ -30,8 +27,16 @@ See DNS-OARC 42, DNS Benchmarking 101: Essentials and Common Pitfalls
- these are just a starting point, experiment! - these are just a starting point, experiment!
### Resolvers ### Resolvers
- An example with explanation how and why we test: https://www.isc.org/blogs/bind-resolver-performance-july-2021/ - An example with explanation why and how we test resolvers: https://www.isc.org/blogs/bind-resolver-performance-july-2021/
- **:warning: Results cannot be generalized to other data sets** or setups with any certainty - **:warning: Results cannot be generalized to other data sets** or setups with any certainty
- Why? Because every single query (and it's timing) changes _state_ of the system under test. Not to speak of dependency on authoritative server performance ... - Why? Because every single query (and it's timing) changes _state_ of the system under test. Not to speak of dependency on authoritative server performance ...
- Introduction to DNS Shotgun [slides](https://ripe79.ripe.net/wp-content/uploads/presentations/45-benchmarking.pdf), [video](https://ripe79.ripe.net/archives/video/198) - Introduction to DNS Shotgun test tool [slides](https://ripe79.ripe.net/wp-content/uploads/presentations/45-benchmarking.pdf), [video](https://ripe79.ripe.net/archives/video/198)
- To run resolver test with real data set, using DNS Shotgun, in our Gitlab CI:
- Go to https://gitlab.isc.org/isc-projects/bind9-shotgun-ci/-/pipelines/new and fill in the form:
- `SHOTGUN_TEST_VERSION`: versions to test, specify multiple versions e.g. like `["main", "another_branch_for_testing"]` (tags and commit hashes also work)
- `SHOTGUN_TRAFFIC_MULTIPLIER`: Load _factor_. For normal resolver play in range keep 10-20.
- `SHOTGUN_ROUNDS`: How many times individual test should be repeated. Use at least 3 if you want sensible results.
- `SHOTGUN_DURATION`: Test duration in seconds, normally most cache misses happen in first minute. Use e.g. 300 if you are also interested in hot-cache behavior.
- Click `[Run pipeline]` button at the bottom of the form