4

pytest howto

Petr Špaček edited this page 2025-07-10 12:36:23 +00:00

Table of Contents

• Debugging
• Magic variables
• Runtime data
• Hypothesis library

title

title
pytest how-to for BIND system tests

See /bin/tests/system/README first

Debugging

• See "Test Artifacts" chapter in README first, for real!
• If test hangs on startup or exits one minute after start - most probably servers did not start. That's typically a config error.
  • pytest --log-level DEBUG should show which server does not respond to TCP queries
• See also pytest.log.txt in directory with artifacts.

Magic variables

Pytest has framework for providing 'context' to tests in a consistent manner. Their term for it is 'fixture' and it manifests itself as a function parameter with name which is nowhere to be found. See https://docs.pytest.org/en/7.1.x/explanation/fixtures.html

Runtime data

As a very first step, Pytest "collects" tests: Effectively it imports all Python files without executing test functions and does filtering (e.g. for -k command line parameter), and it executes setup.sh and other code only for tests which were selected/passed all conditions. For this reason you cannot access any files/data generated by setup.sh from outside of test function body. This limitation applies also to arguments of @decorators around test functions.

Hypothesis library

• --hypothesis-show-statistics to see how many examples the library generated and how many got actually executed
• --hypothesis-seed=0 to get de-randomized behavior for debugging