mir/apparmor
2
0
Fork 0
mirror of https://gitlab.com/apparmor/apparmor synced 2025-08-22 10:07:12 +00:00
Code Issues Releases Wiki Activity
apparmor/tests/unit/parser-tst-parser-sanity/task.yaml

8 lines
313 B
YAML
Raw Normal View History

Allow running tests with spread Spread is a full-system, or integration test suite runner initially developed to test snapd. Over time it has spread to other projects where it provides a structured way to organize, run and debug complex full-system interactions. Spread is documented on https://github.com/canonical/spread and is used in production since late 2016. Spread has a notion of backends which are responsible for allocating and discarding test machines. For the purpose of running AppArmor regression tests, I've combined spread with my own tool, image garden. The tool provides off-the-shelf images, constructed on-the-fly from freely available images, and makes them easily available to spread. The reason for doing it this way is so that using non-free cloud systems is not required and anyone can repeat the test process locally, on their own computer. Vanilla spread is somewhat limited to x86-64 systems but the way I've used it here makes it equally possible to test x86_64 *and* aarch64 systems. I've done most of the development on an ARM single-board-computer running on my desk. Spread requires a top-level spread.yaml file and a collection of task.yaml files that describe individual tasks (for us, those are just tests). Tasks have no implied dependency except that to reach a given task, spread will run all the _prepare_ statements leading to that task, starting from the project, test suite and then task. With proper care one can then run a specific individual test with a one-line command, for example: ``` spread -v garden:ubuntu-cloud-24.04:tests/regression/apparmor:at_secure ``` This will prepare a fresh ubuntu-cloud-24.04 system (matching the CPU architecture of the host), copy the project tree into the test machine, install all the build dependencies, build all the parts of apparmor and then run one specific variant of the regression test, namely the at_secure program. Importantly the same test can also run on, say debian-cloud-13 (Debian Trixie), but also, if you have a Google cloud account, on Google Compute Engine or in one of the other backends either built into spread or available as a fork of spread or as a helper for ad-hoc backend. Spread can also create more than one worker per system and distribute the tests to all of the available instances. In no way are we locking ourselves out of the ability to run our test suite on our target of choice. Spread has other useful switches, such as: - `-reuse` for keeping machines around until discarded with -discard - `-resend` for re-sending updated copy of the project (useful for -reuse) - `-debug` for starting an interactive shell on any failure - `-shell` for starting an interactive shell instead of the `execute` phase This first patch contains just the spread elements, assuming that both spread and image-garden are externally installed. A GitLab continuous integration installing everything required and running a subset of tests will follow shortly. I've expanded the initial selection of systems to allow running all the tests on several versions of Ubuntu, Debian and openSUSE, mainly as a sanity check but also to showcase how practical spread is at covering real-world systems. A number of systems and tests are currently failing: - garden:debian-cloud-12:tests/regression/apparmor:attach_disconnected - garden:debian-cloud-12:tests/regression/apparmor:deleted - garden:debian-cloud-12:tests/regression/apparmor:unix_fd_server - garden:debian-cloud-12:tests/regression/apparmor:unix_socket_pathname - garden:debian-cloud-13:tests/regression/apparmor:attach_disconnected - garden:debian-cloud-13:tests/regression/apparmor:deleted - garden:debian-cloud-13:tests/regression/apparmor:unix_fd_server - garden:debian-cloud-13:tests/regression/apparmor:unix_socket_pathname - garden:opensuse-cloud-15.6:tests/regression/apparmor:attach_disconnected - garden:opensuse-cloud-15.6:tests/regression/apparmor:deleted - garden:opensuse-cloud-15.6:tests/regression/apparmor:e2e - garden:opensuse-cloud-15.6:tests/regression/apparmor:unix_fd_server - garden:opensuse-cloud-15.6:tests/regression/apparmor:unix_socket_pathname - garden:opensuse-cloud-15.6:tests/regression/apparmor:xattrs_profile - garden:opensuse-cloud-tumbleweed:tests/regression/apparmor:attach_disconnected - garden:opensuse-cloud-tumbleweed:tests/regression/apparmor:deleted - garden:opensuse-cloud-tumbleweed:tests/regression/apparmor:unix_fd_server - garden:opensuse-cloud-tumbleweed:tests/regression/apparmor:unix_socket_pathname - garden:ubuntu-cloud-22.04:tests/regression/apparmor:attach_disconnected In addition, only on openSUSE, I've skipped the entire test suite of the utils directory, as it requires python3 ttk themes, which I cannot find in packaged form. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2024-11-25 13:54:03 +01:00
summary: Run apparmor_parser parser_sanity test from parser/tst
# This test is particularly slow. Those values are aimed at running fast enough
# on a Raspberry Pi 5, representing a slower-ish computer.
warn-timeout: 20m
kill-timeout: 30m
execute: |
make -C "$SPREAD_PATH/parser/tst" -j"$(nproc)" parser_sanity
Reference in New Issue Copy Permalink