Merge Allow running tests with spread

Spread is a full-system, or integration test suite runner initially developed to test snapd. Over time it has spread to other projects where it provides a structured way to organize, run and debug complex full-system interactions. Spread is documented on https://github.com/canonical/spread and is used in production since late 2016. Spread has a notion of backends which are responsible for allocating and discarding test machines. For the purpose of running AppArmor regression tests, I've combined spread with my own tool, image garden. The tool provides off-the-shelf images, constructed on-the-fly from freely available images, and makes them easily available to spread. The reason for doing it this way is so that using non-free cloud systems is not required and anyone can repeat the test process locally, on their own computer. Vanilla spread is somewhat limited to x86-64 systems but the way I've used it here makes it equally possible to test x86_64 *and* aarch64 systems. I've done most of the development on an ARM single-board-computer running on my desk. Spread requires a top-level spread.yaml file and a collection of task.yaml files that describe individual tasks (for us, those are just tests). Tasks have no implied dependency except that to reach a given task, spread will run all the _prepare_ statements leading to that task, starting from the project, test suite and then task. With proper care one can then run a specific individual test with a one-line command, for example: ``` spread -v garden:ubuntu-cloud-24.04:tests/regression/apparmor:at_secure ``` This will prepare a fresh ubuntu-cloud-24.04 system (matching the CPU architecture of the host), copy the project tree into the test machine, install all the build dependencies, build all the parts of apparmor and then run one specific variant of the regression test, namely the at_secure program. Importantly the same test can also run on, say debian-cloud-13 (Debian Trixie), but also, if you have a Google cloud account, on Google Compute Engine or in one of the other backends either built into spread or available as a fork of spread or as a helper for ad-hoc backend. Spread can also create more than one worker per system and distribute the tests to all of the available instances. In no way are we locking ourselves out of the ability to run our test suite on our target of choice. Spread has other useful switches, such as: - `-reuse` for keeping machines around until discarded with -discard - `-resend` for re-sending updated copy of the project (useful for -reuse) - `-debug` for starting an interactive shell on any failure - `-shell` for starting an interactive shell instead of the `execute` phase This first patch contains just the spread elements, assuming that both spread and image-garden are externally installed. A GitLab continuous integration installing everything required and running a subset of tests will follow shortly. I've expanded the initial selection of systems to allow running all the tests on several versions of Ubuntu, Debian and openSUSE, mainly as a sanity check but also to showcase how practical spread is at covering real-world systems. A number of tests are currently failing: - garden:debian-cloud-12:tests/regression/apparmor:attach_disconnected - garden:debian-cloud-12:tests/regression/apparmor:deleted - garden:debian-cloud-12:tests/regression/apparmor:unix_fd_server - garden:debian-cloud-12:tests/regression/apparmor:unix_socket_pathname - garden:debian-cloud-13:tests/regression/apparmor:attach_disconnected - garden:debian-cloud-13:tests/regression/apparmor:deleted - garden:debian-cloud-13:tests/regression/apparmor:unix_fd_server - garden:debian-cloud-13:tests/regression/apparmor:unix_socket_pathname - garden:opensuse-cloud-15.6:tests/regression/apparmor:attach_disconnected - garden:opensuse-cloud-15.6:tests/regression/apparmor:deleted - garden:opensuse-cloud-15.6:tests/regression/apparmor:e2e - garden:opensuse-cloud-15.6:tests/regression/apparmor:unix_fd_server - garden:opensuse-cloud-15.6:tests/regression/apparmor:unix_socket_pathname - garden:opensuse-cloud-15.6:tests/regression/apparmor:xattrs_profile In addition, only on openSUSE, I've skipped the entire test suite of the utils directory, as it requires python3 ttk themes, which I cannot find in packaged form. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com> MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1432 Approved-by: Georgia Garcia <georgia.garcia@canonical.com> Merged-by: Georgia Garcia <georgia.garcia@canonical.com>
2025-08-30 13:58:22 +00:00 · 2024-12-05 14:02:31 +00:00
parent e1d8bf1888 d27377a62f
commit d9304c7653
18 changed files with 519 additions and 0 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -306,3 +306,15 @@ tests/regression/apparmor/xattrs_profile
 tests/regression/apparmor/coredump
 **/__pycache__/
 *.orig
+
+# Patterns related to spread integration tests
+*.img
+*.iso
+*.lock
+*.log
+*.qcow2
+*.run
+.spread-reuse.yaml
+.spread-reuse.*.yaml
+spread-artifacts/
+spread-logs/
--- a/.image-garden.mk
+++ b/.image-garden.mk
@@ -0,0 +1,68 @@
+# This file is read by image-garden when spread is allocating test machines.
+# All the package installation happens through cloud-init profiles defined
+# below.
+
+# This is the cloud-init user-data profile for all Debian systems. Note that it
+# is an extension of the default profile necessary for operation of
+# image-garden.
+define DEBIAN_CLOUD_INIT_USER_DATA_TEMPLATE
+$(CLOUD_INIT_USER_DATA_TEMPLATE)
+packages:
+- attr
+- autoconf
+- autoconf-archive
+- automake
+- bison
+- build-essential
+- dejagnu
+- flake8
+- flex
+- gettext
+- libdbus-1-dev
+- libtool
+- liburing-dev
+- pkg-config
+- python3-all-dev
+- python3-gi
+- python3-notify2
+- python3-psutil
+- python3-setuptools
+- python3-tk
+- python3-ttkthemes
+- swig
+endef
+
+# Ubuntu shares cloud-init profile with Debian.
+UBUNTU_CLOUD_INIT_USER_DATA_TEMPLATE=$(DEBIAN_CLOUD_INIT_USER_DATA_TEMPLATE)
+
+# This is the cloud-init user-data profile for openSUSE Tumbleweed.
+define OPENSUSE_tumbleweed_CLOUD_INIT_USER_DATA_TEMPLATE
+$(CLOUD_INIT_USER_DATA_TEMPLATE)
+packages:
+- attr
+- autoconf
+- autoconf-archive
+- automake
+- bison
+- dbus-1-devel
+- dejagnu
+- flex
+- gcc
+- gcc-c++
+- gettext
+- gobject-introspection
+- libtool
+- liburing2-devel
+- make
+- pkg-config
+- python3-flake8
+- python3-notify2
+- python3-psutil
+- python3-setuptools
+- python3-setuptools
+- python3-tk
+- python311
+- python3-devel
+- python311-devel
+- swig
+endef
--- a/README.md
+++ b/README.md
@@ -206,6 +206,30 @@ usage and how to update and add tests. Below is a quick overview of their
 location and how to run them.


+Using spread with local virtual machines
+----------------------------------------
+
+It may be convenient to use the spread tool to provision and run the test suite
+in an ephemeral virtual machine. This allows testing in isolation from the
+host, as well as testing across different commonly used distributions and their
+real kernels.
+
+```sh
+sudo apt install git golang whois ovmf genisoimage qemu-utils qemu-system
+go install github.com/snapcore/spread/cmd/spread@latest
+git clone https://gitlab.com/zygoon/image-garden
+make -C image-garden
+sudo make -C image-garden install
+image-garden make ubuntu-cloud-24.10.x86_64.run
+cd $APPARMOR_PATH
+git clean -xdf
+~/go/bin/spread -artifacts ./spread-artifacts -v ubuntu-cloud-24.10
+# or ~/go/bin/spread -v garden:ubuntu-cloud-24.04:tests/regression/apparmor:at_secure
+```
+
+Running the `run_spread.sh` script, with `spread` on `PATH` will run all the
+tests across several supported systems (Debian, Ubuntu and openSUSE).
+
 Regression tests
 ----------------
 For details on structure and adding tests, see
--- a/run-spread.sh
+++ b/run-spread.sh
@@ -0,0 +1,29 @@
+#!/bin/sh
+# Run integration tests with spread sequentially on all the systems, using
+# multiple workers per system. This mode is suitable to run on a single
+# quad-core CPU with 8GB of RAM and no desktop session.
+set -xeu
+
+if test -z "$(command -v spread)"; then
+	echo "You need to install spread from https://github.com/snapcore/spread with the Go compiler and the command: go install github.com/snapcore/spread/cmd/spread@latest" >&2
+	exit 1
+fi
+
+if test -z "$(command -v image-garden)"; then
+	echo "You need to install image-garden from https://gitlab.com/zygoon/image-garden: make install prefix=/usr/local" >&2
+	exit 1
+fi
+
+rm -rf spread-logs spread-artifacts
+mkdir -p spread-logs
+for system in \
+	opensuse-cloud-tumbleweed \
+	debian-cloud-12 \
+	debian-cloud-13 \
+	ubuntu-cloud-22.04 \
+	ubuntu-cloud-24.04 \
+	ubuntu-cloud-24.10; do
+	if ! spread -artifacts ./spread-artifacts -v "$system" | tee spread-logs/"$system".log; then
+		echo "Spread exited with code $?" >spread-logs/"$system".failed
+	fi
+done
--- a/spread.yaml
+++ b/spread.yaml
@@ -0,0 +1,138 @@
+project: apparmor
+backends:
+    garden:
+        # The garden backend relies on https://gitlab.com/zygoon/image-garden
+        # TODO: Switch to a released version for better stability.
+        type: adhoc
+        # Use just enough RAM to link the parser on a virtual system with two
+        # cores. Using more cores may easily consume more memory, due to make
+        # -j$(nproc, used below than a small CI/CD system is typically granted.
+        # It is better to have more workers than to have one big worker with
+        # lots of resources.
+        allocate: ADDRESS "$(QEMU_MEM_OPTION="-m 1536" QEMU_SMP_OPTION="-smp 2" image-garden allocate "$SPREAD_SYSTEM".$(uname -m))"
+        discard: image-garden discard "$SPREAD_SYSTEM_ADDRESS"
+        systems:
+            # All systems except for the one Ubuntu system are marked as
+            # manual. This way we don't accidentally spin up everything when
+            # someone runs spread without knowing better.
+            - opensuse-cloud-tumbleweed:
+                  username: opensuse
+                  password: opensuse
+                  workers: 4
+                  manual: true
+            - debian-cloud-12:
+                  username: debian
+                  password: debian
+                  workers: 4
+                  manual: true
+            - debian-cloud-13:
+                  username: debian
+                  password: debian
+                  workers: 4
+                  manual: true
+            - ubuntu-cloud-22.04:
+                  username: ubuntu
+                  password: ubuntu
+                  workers: 4
+                  manual: true
+            - ubuntu-cloud-24.04:
+                  username: ubuntu
+                  password: ubuntu
+                  workers: 4
+                  manual: true
+            - ubuntu-cloud-24.10:
+                  username: ubuntu
+                  password: ubuntu
+                  workers: 4
+
+exclude:
+    - .git
+    - "*.o"
+    # Files related to spread and image-garden.
+    - "*.qcow2"
+    - "*.iso"
+    - "*.img"
+    - "*.log"
+    - "*.run"
+    - "*.lock"
+    - spread-logs
+    - spread-artifacts
+
+# Copy the project to this path on the test system.
+# This is also available as $SPREAD_PATH.
+path: /tmp/apparmor
+
+prepare: |
+    # Configure libapparmor but only if a makefile is not already present.
+    # This makes repeated iteration with -reuse much faster, as the chain of
+    # invocations of make below are efficient if nothing needs to be done.
+    if [ ! -f "$SPREAD_PATH"/libraries/libapparmor/Makefile ]; then
+      (
+        cd $SPREAD_PATH/libraries/libapparmor
+        sh ./autogen.sh && sh ./configure --prefix=/usr --with-perl --with-python
+      )
+    fi
+    # Build libapparmor.
+    make -C $SPREAD_PATH/libraries/libapparmor -j"$(nproc)"
+    # Build apparmor_parser.
+    make -C $SPREAD_PATH/parser -j"$(nproc)"
+    # Build binary utilities (aa-exec and firends).
+    make -C $SPREAD_PATH/binutils -j"$(nproc)"
+    # Build python utilities.
+    make -C $SPREAD_PATH/utils -j"$(nproc)"
+
+# In case of failure, include the kernel version in the log.
+debug-each: |
+    uname -a
+
+suites:
+    utils/:
+        summary: Unit tests for the Python utilities.
+        prepare: |
+            # Generate apparmor profiles that the tests rely on.
+            make -C "$SPREAD_PATH"/parser/tst gen_xtrans gen_dbus
+            # Spread does not support programmatically generated test variants.
+            # Ensure that the list baked into utils/test/task.yaml contains all
+            # the files matching utils/test/test-*.py
+            fail=0
+            for V in $SPREAD_PATH/utils/test/test-*.py; do
+              Vdash="$(basename "$V" | sed -e 's,^test-,,' -e 's,\.py$,,')"
+              Vunder="$(basename "$V" | sed -e 's,^test-,,' -e 's,\.py$,,' -e 's,-,_,g')"
+              if ! grep -xF '    TEST/'"$Vunder"': '"$Vdash" "$SPREAD_PATH"/utils/test/task.yaml; then
+                echo "utils/test/task.yaml: missing test variant: TEST/$Vunder: $Vdash" >&2
+                fail=1
+              fi
+            done
+            if [ "$fail" -ne 0 ]; then
+              echo "exiting due to missing variants listed above" >&2
+              exit 1
+            fi
+    tests/unit/:
+        summary: Unit tests that do not exercise the kernel layer.
+    tests/regression/:
+        summary: Regression tests for parser-kernel interaction.
+        prepare: |
+            # Spread does not support programmatically generated test variants.
+            # Ensure that the list baked into tests/regression/apparmor/task.yaml
+            # contains all the tests defined in tests/regression/apparmor/Makefile.
+            echo '$(foreach t,$(TESTS),$(info TEST/$t))' | \
+              make -n -f "$SPREAD_PATH"/tests/regression/apparmor/Makefile -f /dev/stdin | \
+              grep -F TEST/ | \
+              cut -d / -f 2 | \
+              tee apparmor-regression-tests.txt
+            fail=0
+            while read -r V; do
+              if ! grep -xF '    TEST/'"$V"': 1' "$SPREAD_PATH"/tests/regression/apparmor/task.yaml; then
+                echo "tests/regression/task.yaml: missing test variant: TEST/$V" >&2
+                fail=1
+              fi
+            done <apparmor-regression-tests.txt
+            if [ "$fail" -ne 0 ]; then
+              echo "exiting due to missing variants listed above" >&2
+              exit 1
+            fi
+
+            # Build all the apparmor regression test programs.
+            make -C "$SPREAD_PATH"/tests/regression/apparmor -j"$(nproc)"
+        restore: |
+            rm -f apparmor-regression-tests.txt
--- a/tests/regression/apparmor/task.yaml
+++ b/tests/regression/apparmor/task.yaml
@@ -0,0 +1,129 @@
+summary: run all of the apparmor regression test suite
+# See spread.yaml for the code which verifies that nothing is missing from this list of variants
+environment:
+    TEST/aa_exec: 1
+    TEST/aa_policy_cache: 1
+    TEST/access: 1
+    TEST/at_secure: 1
+    TEST/attach_disconnected: 1
+    TEST/capabilities: 1
+    TEST/changehat: 1
+    TEST/changehat_fork: 1
+    TEST/changehat_misc: 1
+    TEST/changeprofile: 1
+    TEST/chdir: 1
+    TEST/clone: 1
+    TEST/complain: 1
+    TEST/coredump: 1
+    TEST/dbus_eavesdrop: 1
+    TEST/dbus_message: 1
+    TEST/dbus_service: 1
+    TEST/dbus_unrequested_reply: 1
+    TEST/deleted: 1
+    TEST/e2e: 1
+    TEST/environ: 1
+    TEST/exec: 1
+    TEST/exec_qual: 1
+    TEST/exec_stack: 1
+    TEST/fchdir: 1
+    TEST/fd_inheritance: 1
+    TEST/fork: 1
+    TEST/i18n: 1
+    TEST/introspect: 1
+    TEST/io_uring: 1
+    TEST/link: 1
+    TEST/link_subset: 1
+    TEST/longpath: 1
+    TEST/mkdir: 1
+    TEST/mmap: 1
+    TEST/mount: 1
+    TEST/mult_mount: 1
+    TEST/named_pipe: 1
+    TEST/namespaces: 1
+    TEST/net_raw: 1
+    TEST/nfs: 1
+    TEST/nnp: 1
+    TEST/onexec: 1
+    TEST/open: 1
+    TEST/openat: 1
+    TEST/pipe: 1
+    TEST/pivot_root: 1
+    TEST/posix_ipc: 1
+    TEST/ptrace: 1
+    TEST/pwrite: 1
+    TEST/query_label: 1
+    TEST/readdir: 1
+    TEST/regex: 1
+    TEST/rename: 1
+    TEST/rw: 1
+    TEST/sd_flags: 1
+    TEST/setattr: 1
+    TEST/socketpair: 1
+    TEST/stackonexec: 1
+    TEST/stackprofile: 1
+    TEST/swap: 1
+    TEST/symlink: 1
+    TEST/syscall: 1
+    TEST/sysv_ipc: 1
+    TEST/tcp: 1
+    TEST/unix_fd_server: 1
+    TEST/unix_socket_abstract: 1
+    TEST/unix_socket_autobind: 1
+    TEST/unix_socket_pathname: 1
+    TEST/unix_socket_unnamed: 1
+    TEST/unlink: 1
+    TEST/userns: 1
+    TEST/xattrs: 1
+    TEST/xattrs_profile: 1
+    # Some tests are currently failing. Those are listed below. For each variant
+    # listed above, the XFAIL variable contains a list of spread systems where this
+    # test is expected to fail.
+    #
+    # Error: unix_fd_server passed. Test 'ATTACH_DISCONNECTED (attach_disconnected.path rule at /)' was expected to 'fail'
+    XFAIL/attach_disconnected: opensuse-cloud-tumbleweed debian-cloud-12 debian-cloud-13 ubuntu-cloud-22.04
+    # Error: unix_fd_server failed. Test 'fd passing; unconfined client' was expected to 'pass'. Reason for failure 'FAIL - bind failed: Permission denied'
+    # Error: unix_fd_server failed. Test 'fd passing; confined client w/ rw' was expected to 'pass'. Reason for failure 'FAIL - bind failed: Permission denied'
+    XFAIL/deleted: opensuse-cloud-tumbleweed debian-cloud-12 debian-cloud-13
+    # Error: unix_fd_server failed. Test 'fd passing; confined -> unconfined' was expected to 'pass'. Reason for failure 'FAIL - bind failed: Permission denied'
+    # Error: unix_fd_server failed. Test 'fd passing; unconfined -> confined' was expected to 'pass'. Reason for failure 'FAIL CLIENT - connect Permission denied'
+    # Error: unix_fd_server failed. Test 'fd passing; unconfined -> confined (no perm)' was expected to 'pass'. Reason for failure 'FAIL CLIENT - connect Permission denied'
+    # Error: unix_fd_server failed. Test 'fd passing; confined -> confined' was expected to 'pass'. Reason for failure 'FAIL - bind failed: Permission denied'
+    XFAIL/unix_fd_server: opensuse-cloud-tumbleweed debian-cloud-12 debian-cloud-13
+    # Error: unix_socket failed. Test 'AF_UNIX pathname socket (stream); confined server w/ access (rw)' was expected to 'pass'. Reason for failure 'FAIL - setsockopt (SO_RCVTIMEO): Permission denied'
+    # Error: unix_socket passed. Test 'AF_UNIX pathname socket (stream); confined server w/ a missing af_unix access (create)' was expected to 'fail'
+    # Error: unix_socket failed. Test 'AF_UNIX pathname socket (stream); confined client w/ access (rw)' was expected to 'pass'. Reason for failure 'FAIL - setsockopt (SO_RCVTIMEO): Permission denied'
+    # xpass: AF_UNIX pathname socket (dgram); confined server w/ access (rw)
+    # Error: unix_socket passed. Test 'AF_UNIX pathname socket (dgram); confined server w/ a missing af_unix access (create)' was expected to 'fail'
+    # xpass: AF_UNIX pathname socket (dgram); confined client w/ access (rw)
+    # Error: unix_socket failed. Test 'AF_UNIX pathname socket (seqpacket); confined server w/ access (rw)' was expected to 'pass'. Reason for failure 'FAIL - setsockopt (SO_RCVTIMEO): Permission denied'
+    # Error: unix_socket passed. Test 'AF_UNIX pathname socket (seqpacket); confined server w/ a missing af_unix access (create)' was expected to 'fail'
+    # Error: unix_socket failed. Test 'AF_UNIX pathname socket (seqpacket); confined client w/ access (rw)' was expected to 'pass'. Reason for failure 'FAIL - setsockopt (SO_RCVTIMEO): Permission denied'
+    XFAIL/unix_socket_pathname: opensuse-cloud-tumbleweed debian-cloud-12 debian-cloud-13
+artifacts:
+    - bash.log
+    - bash.err
+execute: |
+    # Run the shell script that is named after the spread variant we are running
+    # now. The makefile runs them all sequentially via the "alltests" goal. Here
+    # we can parallelize it through spread and also have a way to run precisely
+    # the test we want, especially for debugging.
+    if ! bash -x "$SPREAD_VARIANT".sh >bash.log 2>bash.err; then
+      for xfail in ${XFAIL:-}; do
+        if [ "$SPREAD_SYSTEM" = "$xfail" ]; then
+          echo "Ignoring expected failure of $SPREAD_TASK on $SPREAD_SYSTEM"
+          exit 0
+        fi
+      done
+
+      echo "Test $SPREAD_VARIANT has unexpectedly failed"
+      echo "Test execution logs are in the files bash.{log.err} and are collected as artifacts"
+      exit 1
+    else
+      for xfail in ${XFAIL:-}; do
+        if [ "$SPREAD_SYSTEM" = "$xfail" ]; then
+          echo "Test $SPREAD_VARIANT has unexpectedly passed"
+          echo "Test execution logs are in the files bash.{log.err} and are collected as artifacts"
+          exit 1
+        fi
+      done
+    fi
--- a/tests/unit/binutils/task.yaml
+++ b/tests/unit/binutils/task.yaml
@@ -0,0 +1,3 @@
+summary: Run unit tests of AppArmor binary utilities
+execute: |
+    make -C "$SPREAD_PATH"/binutils check
--- a/tests/unit/libapparmor/task.yaml
+++ b/tests/unit/libapparmor/task.yaml
@@ -0,0 +1,3 @@
+summary: Run unit tests of libapparmor
+execute: |
+    make -C "$SPREAD_PATH"/libraries/libapparmor check
--- a/tests/unit/parser-tst-caching/task.yaml
+++ b/tests/unit/parser-tst-caching/task.yaml
@@ -0,0 +1,3 @@
+summary: Run apparmor_parser caching test from parser/tst
+execute: |
+    make -C "$SPREAD_PATH"/parser/tst -j"$(nproc)" caching
--- a/tests/unit/parser-tst-dirtest/task.yaml
+++ b/tests/unit/parser-tst-dirtest/task.yaml
@@ -0,0 +1,3 @@
+summary: Run apparmor_parser dirtest test from parser/tst
+execute: |
+    make -C "$SPREAD_PATH"/parser/tst -j"$(nproc)" dirtest
--- a/tests/unit/parser-tst-equality/task.yaml
+++ b/tests/unit/parser-tst-equality/task.yaml
@@ -0,0 +1,11 @@
+summary: Run apparmor_parser tests from parser/tst
+# This test is particularly slow. Those values are aimed at running fast enough
+# on a Raspberry Pi 5, representing a slower-ish computer.
+#
+# TODO: figure out how to make this test more parallelizable.
+warn-timeout: 20m
+kill-timeout: 30m
+# Start this task sooner rather than later.
+priority: 100
+execute: |
+    make -C "$SPREAD_PATH"/parser/tst -j"$(nproc)" equality
--- a/tests/unit/parser-tst-error-output/task.yaml
+++ b/tests/unit/parser-tst-error-output/task.yaml
@@ -0,0 +1,3 @@
+summary: Run apparmor_parser error_output test from parser/tst
+execute: |
+    make -C "$SPREAD_PATH"/parser/tst -j"$(nproc)" error_output
--- a/tests/unit/parser-tst-minimize/task.yaml
+++ b/tests/unit/parser-tst-minimize/task.yaml
@@ -0,0 +1,3 @@
+summary: Run apparmor_parser minimize test from parser/tst
+execute: |
+    make -C "$SPREAD_PATH"/parser/tst -j"$(nproc)" minimize
--- a/tests/unit/parser-tst-parser-sanity/task.yaml
+++ b/tests/unit/parser-tst-parser-sanity/task.yaml
@@ -0,0 +1,11 @@
+summary: Run apparmor_parser parser_sanity test from parser/tst
+# This test is particularly slow. Those values are aimed at running fast enough
+# on a Raspberry Pi 5, representing a slower-ish computer.
+#
+# TODO: figure out how to make this test more parallelizable.
+warn-timeout: 20m
+kill-timeout: 30m
+# Start this task sooner rather than later.
+priority: 100
+execute: |
+    make -C "$SPREAD_PATH"/parser/tst -j"$(nproc)" parser_sanity
--- a/tests/unit/parser/task.yaml
+++ b/tests/unit/parser/task.yaml
@@ -0,0 +1,16 @@
+summary: Run apparmor_parser unit tests from parser/
+details: |
+    The parser has a number of different tests. Those are all represented as
+    spread task variants so that they are directly visible and runnable.
+environment:
+    TEST/tst_regex: 1
+    TEST/tst_misc: 1
+    TEST/tst_symtab: 1
+    TEST/tst_variable: 1
+    TEST/tst_lib: 1
+prepare: |
+    # The test relies on make to build a binary.
+    make -C "$SPREAD_PATH"/parser -j"$(nproc)" "$SPREAD_VARIANT"
+execute: |
+    cd "$SPREAD_PATH"/parser
+    ./"$SPREAD_VARIANT"
--- a/utils/test/Makefile
+++ b/utils/test/Makefile
@@ -84,6 +84,16 @@ clean:
 check: __libapparmor __parser
 	export PYTHONPATH=$(PYTHONPATH) LD_LIBRARY_PATH=$(LD_LIBRARY_PATH) LC_ALL=C __AA_CONFDIR=$(CONFDIR) __AA_BASEDIR=$(BASEDIR) __AA_PARSER=$(PARSER) ; $(foreach test, $(wildcard test-*.py), echo ; echo === $(test) === ; $(call pyalldo, $(test)))

+# FIXME: passing -v to test-$*.py below makes some tests fail.
+#
+# This is a static pattern rule that allows running individual tests via the
+# check-one-test-NAME target, where NAME matches an exciting file test-NAME.py.
+# The way it is coded allows make <tab><tab> to complete it, making it easier
+# to use interactively.
+check-one-test-%: PYTHON ?= python3
+$(foreach t,$(wildcard test-*.py),check-one-test-$(patsubst test-%.py,%,$t)): check-one-test-%: __libapparmor __parser
+	PYTHONPATH=$(PYTHONPATH) LD_LIBRARY_PATH=$(LD_LIBRARY_PATH) LC_ALL=C __AA_CONFDIR=$(CONFDIR) __AA_BASEDIR=$(BASEDIR) __AA_PARSER=$(PARSER) $(PYTHON) test-$*.py
+
 .coverage: $(wildcard ../aa-* ../apparmor/*.py test-*.py) __libapparmor __parser
 	export PYTHONPATH=$(PYTHONPATH) LD_LIBRARY_PATH=$(LD_LIBRARY_PATH) LC_ALL=C __AA_CONFDIR=$(CONFDIR) __AA_BASEDIR=$(BASEDIR) __AA_PARSER=$(PARSER) ; $(COVERAGE_IGNORE_FAILURES_CMD) ; $(foreach test, $(wildcard test-*.py), echo ; echo === $(test) === ; $(PYTHON) -m coverage run --branch -p $(test); )
 	$(PYTHON) -m coverage combine
--- a/utils/test/task.yaml
+++ b/utils/test/task.yaml
@@ -0,0 +1,50 @@
+summary: Run "make check" broken down by test name
+systems:
+ - -opensuse-cloud-15.6
+environment:
+    # FIXME: Spread disallows dashes in variant names, most likely as a bug.
+    # As such we map the all_under_score variant name to actual test name.
+    TEST/aa_cli_bootstrap: aa-cli-bootstrap
+    TEST/aa_decode: aa-decode
+    TEST/aa_easyprof: aa-easyprof
+    TEST/aa_notify: aa-notify
+    TEST/aa: aa
+    TEST/aare: aare
+    TEST/abi: abi
+    TEST/alias: alias
+    TEST/all: all
+    TEST/baserule: baserule
+    TEST/boolean: boolean
+    TEST/capability: capability
+    TEST/change_profile: change_profile
+    TEST/common: common
+    TEST/config: config
+    TEST/dbus: dbus
+    TEST/example: example
+    TEST/file: file
+    TEST/include: include
+    TEST/io_uring: io_uring
+    TEST/libapparmor_test_multi: libapparmor-test_multi
+    TEST/logparser: logparser
+    TEST/logprof: logprof
+    TEST/minitools: minitools
+    TEST/mount: mount
+    TEST/mqueue: mqueue
+    TEST/network: network
+    TEST/notify: notify
+    TEST/parser_simple_tests: parser-simple-tests
+    TEST/pivot_root: pivot_root
+    TEST/profile_list: profile-list
+    TEST/profile_storage: profile-storage
+    TEST/profiles: profiles
+    TEST/ptrace: ptrace
+    TEST/regex_matches: regex_matches
+    TEST/rlimit: rlimit
+    TEST/severity: severity
+    TEST/signal: signal
+    TEST/translations: translations
+    TEST/unix: unix
+    TEST/userns: userns
+    TEST/variable: variable
+execute: |
+    make --warn-undefined-variables check-one-test-"$TEST"
--- a/utils/vim/task.yaml
+++ b/utils/vim/task.yaml
@@ -0,0 +1,3 @@
+summary: Run "make check"
+execute: |
+    make --warn-undefined-variables check