mir/apparmor
2
0
Fork 0
mirror of https://gitlab.com/apparmor/apparmor synced 2025-08-23 02:27:12 +00:00
Code Issues Releases Wiki Activity
apparmor/utils/test/test-parser-simple-tests.py

582 lines
21 KiB
Python
Raw Normal View History

Switch utils to python3 As discussed a while ago, switch the utils (including their tests) to use python3 by default. While on it, drop usage of "env" to always get the system python3 instead of a random one that happens to live somewhere in $PATH. In practise, this patch doesn't change much - AFAIK openSUSE, Debian and Ubuntu already patch aa-* to use python3. Also add a note to README to officially deprecate Python 2.x. (I won't break Python 2.x support intentionally - unless some future change gives me a very good reason to finally drop Python 2.x support.) Acked-by: Seth Arnold <seth.arnold@canonical.com> (since 2016-08-23, but the commit had to wait for the FileRule series because it touches test-file.py)
2016-10-01 20:57:09 +02:00
#! /usr/bin/python3
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# ------------------------------------------------------------------
#
# Copyright (C) 2015 Christian Boltz <apparmor@cboltz.de>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of version 2 of the GNU General Public
# License published by the Free Software Foundation.
#
# ------------------------------------------------------------------
Order imports and module-level dunder name assignments.
2022-08-07 20:32:07 -04:00
import os
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
import unittest
Order imports and module-level dunder name assignments.
2022-08-07 20:32:07 -04:00
import apparmor.aa as apparmor
from apparmor.common import AppArmorException, is_skippable_file, open_file_read
from common_test import AATest, setup_aa, setup_all_loops
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# This testcase will parse all parser/tst/simple_tests with parse_profile_data(),
# except the files listed in one of the arrays below.
#
# Files listed in skip_startswith will be completely skipped.
# Files listed in the other arrays will be checked, but with the opposite of the expected result.
# XXX tests listed here will be *** SKIPPED *** XXX
skip_startswith = (
# the tools don't check for conflicting x permissions (yet?)
'generated_x/conflict-',
'generated_x/ambiguous-',
'generated_x/dominate-',
# 'safe' and 'unsafe' keywords
'generated_perms_safe/',
# Pux and Cux (which actually mean PUx and CUx) get rejected by the tools
'generated_x/exact-',
parser: add the ability to specify a priority prefix to rules This enables adding a priority to a rules in policy, finishing out the priority work done to plumb priority support through the internals in the previous patch. Rules have a default priority of 0. The priority prefix can be added before the other currently support rule prefixes, ie. [priority prefix][audit qualifier][rule mode][owner] If present a numerical priority can be assigned to the rule, where the greater the number the higher the priority. Eg. priority=1 audit file r /etc/passwd, priority=-1 deny file w /etc/**, Rule priority allows the rule with the highest priority to completely override lower priority rules where they overlap. Within a given priority level rules will accumulate in standard apparmor fashion. Eg. given priority=1 w /*c, priority=0 r /a*, priority=-1 k /*b*, /abc, /bc, /ac .. will have permissions of w /ab, /abb, /aaa, .. will have permissions of r /b, /bcb, /bab, .. will have permissions of k User specified rule priorities are currently capped at the arbitrary values of 1000, and -1000. Notes: * not all rule types support the priority prefix. Rukes like - network - capability - rlimits need to be reworked need to be reworked to properly preserve the policy rule structure. * this patch does not support priority on rule blocks * this patch does not support using a variable in the priority value. Signed-off-by: John Johansen <john.johansen@canonical.com>
2024-05-11 23:33:42 -07:00
# don't handle rule priorities yet
'file/priority/',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
)
# testcases that should raise an exception, but don't
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
exception_not_raised = (
Add most abi/bad_*.sd tests to "exception not raised" list Interestingly, abi/bad_6.sd is detected as invalid, and therefore not added to the list.
2018-10-13 20:23:57 +02:00
# most abi/bad_* aren't detected as bad by the basic implementation in the tools
'abi/bad_10.sd',
'abi/bad_11.sd',
'abi/bad_12.sd',
# interesting[tm] profile name
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'change_hat/bad_parsing.sd',
utils: Handle the safe/unsafe change_profile exec modes https://launchpad.net/bugs/1584069 This patch adds support for the safe and unsafe exec modes for change_profile rules. The logic is pretty simple at this point because the kernel's default for exec modes changed in newer versions. Therefore, this patch simply retains any specified exec mode in parsed rules. If an exec mode is not specified in a rule, there is no attempt to force the usage of "safe" because older kernels do not support it. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Seth Arnold <seth.arnold@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2016-07-20 17:24:11 -05:00
Adding userspace support for mount rules in aa-genprof/aa-logprof
2024-02-29 17:59:50 +00:00
'dbus/bad_regex_04.sd',
'dbus/bad_regex_05.sd',
'dbus/bad_regex_06.sd',
'file/bad_re_brace_1.sd',
'file/bad_re_brace_2.sd',
'file/bad_re_brace_3.sd',
utils: Handle the safe/unsafe change_profile exec modes https://launchpad.net/bugs/1584069 This patch adds support for the safe and unsafe exec modes for change_profile rules. The logic is pretty simple at this point because the kernel's default for exec modes changed in newer versions. Therefore, this patch simply retains any specified exec mode in parsed rules. If an exec mode is not specified in a rule, there is no attempt to force the usage of "safe" because older kernels do not support it. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Seth Arnold <seth.arnold@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2016-07-20 17:24:11 -05:00
# The tools don't detect conflicting change_profile exec modes
'change_profile/onx_conflict_unsafe1.sd',
'change_profile/onx_conflict_unsafe2.sd',
Ignore test failures about duplicated conditionals in dbus rules Since r3634, the tools allow any order of dbus conditionals. Quoting the r3634 patch description: This patch eases the restriction on the ordering at the expense of the utils no longer being able to detect and reject a single attribute that is repeated multiple times. In that situation, only the last occurrence of the attribute will be honored by the utils. It seems nobody tested with all test profiles generated ;-) so we have to add some exceptions to the "does not raise an exception" list now. Acked-by <timeout> for trunk and 2.11
2017-04-20 13:05:53 +02:00
# duplicated conditionals aren't detected by the tools
'generated_dbus/duplicated-conditionals-45127.sd',
'generated_dbus/duplicated-conditionals-45131.sd',
'generated_dbus/duplicated-conditionals-45124.sd',
'generated_dbus/duplicated-conditionals-45130.sd',
'generated_dbus/duplicated-conditionals-45125.sd',
'generated_dbus/duplicated-conditionals-45128.sd',
'generated_dbus/duplicated-conditionals-45129.sd',
utils: Don't enforce ordering of dbus rule attributes https://launchpad.net/bugs/1628286 The utils were enforcing that the dbus rule attributes were strictly ordered in the following fashion: bus -> path -> interface -> member -> peer However, the parser has always accepted the attributes in any order. If the system contained a profile which did not use the strict ordering enforced by the utils, the utils would refuse to operate at all. This patch eases the restriction on the ordering at the expense of the utils no longer being able to detect and reject a single attribute that is repeated multiple times. In that situation, only the last occurrence of the attribute will be honored by the utils. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2017-02-28 23:04:24 +00:00
'dbus/bad_modifier_2.sd',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'dbus/bad_regex_01.sd',
'dbus/bad_regex_02.sd',
'dbus/bad_regex_03.sd',
'dbus/bad_regex_04.sd',
'dbus/bad_regex_05.sd',
'dbus/bad_regex_06.sd',
[14/38] Use FileRule and FileRuleset Change aa.py to use FileRule and FileRuleset for parsing and saving profiles. In detail, this means: - add 'file' to the list of rule classes to enable it at various places - store file rules in aa[profile][hat]['file'] (not 'path' as before) to be consistent with the FileRule name - drop the no longer needed delete_path_duplicates() - this is now handled by FileRuleset like in all other rule classes. (same change in cleanprofile.py) - replace usage of RE_PROFILE_BARE_FILE_ENTRY and RE_PROFILE_PATH_ENTRY with FileRule.match() - drop write_path_rules() and write_paths() and replace them with the new write_file() function. - adjust several code sections to use write_file() and 'file' instead of 'path' FileRule doesn't drop optional keywords ('allow' and 'file'), therefore adjust cleanprof_test.out to the changed behaviour. (If someone insists on dropping optional keywords in aa-cleanprof, that's something for a future patch.) Also adjust the list of known failures in test-parser-simple-tests.py - switching to FileRule avoids several test failures (and introduces a few new ones ;-) IMPORTANT: This patch introduces a "brain split" which means - parsing and writing the profile and aa-cleanprof use the new location (aa[profile][hat]['file']) - aa-logprof and aa-genprof still save data to the old location (aa[profile][hat]['allow']['path']) and probably ask superfluous questions because there are no rules existing in the old location TL;DR: don't try aa-logprof or aa-genprof with only this patch applied. I know this isn't ideal, but still better than an even bigger and totally unreadable patch ;-) Acked-by: Steve Beattie <steve@nxnw.org>
2016-10-01 19:54:48 +02:00
'file/bad_re_brace_1.sd',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'file/bad_re_brace_2.sd',
'file/bad_re_brace_3.sd',
Adding userspace support for mount rules in aa-genprof/aa-logprof
2024-02-29 17:59:50 +00:00
# We do not check that options are compatible
parse tests: add parse tests for missing mount options add simple parsing tests for nostrictatime, lazytime, nolazytime Signed-off-by: John Johansen <john.johansen@canonical.com>
2023-03-31 02:59:48 -07:00
'mount/bad_opt_29.sd',
'mount/bad_opt_30.sd',
'mount/bad_opt_31.sd',
Adding userspace support for mount rules in aa-genprof/aa-logprof
2024-02-29 17:59:50 +00:00
'mount/bad_1.sd',
'mount/bad_2.sd',
mount: accept fstype and options in any order Note: If multiple fstype= or options= are given, this is not detected as an error (to keep the regex simpler). When writing back such a rule, only one fstype and options will "survive". Adjust the exclude list in test-parser-simple-tests.py accordingly: - several valid mount rules no longer fail - two invalid mount rules which so far accidentally raised an exception because of the fstype/options order no longer raise this exception (conflicting mount options, which are the real reason why these rules are invalid, are not detected in the tools) Fixes: https://gitlab.com/apparmor/apparmor/-/issues/501
2025-04-05 19:18:09 +02:00
'mount/bad_3.sd',
'mount/bad_4.sd',
Adding userspace support for mount rules in aa-genprof/aa-logprof
2024-02-29 17:59:50 +00:00
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'profile/flags/flags_bad10.sd',
'profile/flags/flags_bad11.sd',
'profile/flags/flags_bad12.sd',
'profile/flags/flags_bad13.sd',
'profile/flags/flags_bad15.sd',
'profile/flags/flags_bad18.sd',
'profile/flags/flags_bad19.sd',
'profile/flags/flags_bad2.sd',
'profile/flags/flags_bad3.sd',
'profile/flags/flags_bad4.sd',
'profile/flags/flags_bad5.sd',
'profile/flags/flags_bad6.sd',
'profile/flags/flags_bad7.sd',
'profile/flags/flags_bad8.sd',
'profile/flags/flags_bad_debug_1.sd',
'profile/flags/flags_bad_debug_2.sd',
'profile/flags/flags_bad_debug_3.sd',
parser: support enforce, kill and unconfined profile modes The enforce profile mode is the default but specifying it explicitly has not been supported. Allow enforce to be specified as a mode. If no mode is specified the default is still enforce. The kernel has supported kill and unconfined profile modes for a long time now. And support to the parser so that profiles can make use of these modes. MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/440 Fixes: https://gitlab.com/apparmor/apparmor/-/issues/7 Signed-off-by: John Johansen <john.johansen@canonical.com> Acked-by: Steve Beattie <sbeattie@ubuntu.com>
2018-07-04 09:26:27 -07:00
# detection of conflicting flags not supported
'profile/flags/flags_bad30.sd',
'profile/flags/flags_bad31.sd',
'profile/flags/flags_bad32.sd',
'profile/flags/flags_bad33.sd',
'profile/flags/flags_bad34.sd',
'profile/flags/flags_bad35.sd',
'profile/flags/flags_bad36.sd',
'profile/flags/flags_bad37.sd',
'profile/flags/flags_bad38.sd',
'profile/flags/flags_bad39.sd',
'profile/flags/flags_bad40.sd',
'profile/flags/flags_bad41.sd',
'profile/flags/flags_bad42.sd',
'profile/flags/flags_bad43.sd',
'profile/flags/flags_bad44.sd',
'profile/flags/flags_bad45.sd',
'profile/flags/flags_bad46.sd',
parser: add support for prompt profile mode Add support for the prompt profile mode. Signed-off-by: John Johansen <john.johansen@canonical.com>
2019-12-05 14:20:24 -08:00
'profile/flags/flags_bad47.sd',
'profile/flags/flags_bad48.sd',
'profile/flags/flags_bad49.sd',
'profile/flags/flags_bad50.sd',
'profile/flags/flags_bad51.sd',
'profile/flags/flags_bad52.sd',
'profile/flags/flags_bad53.sd',
'profile/flags/flags_bad54.sd',
'profile/flags/flags_bad55.sd',
'profile/flags/flags_bad56.sd',
parser: add interruptible flag Allow indicating that prompt upcalls to userspace can be interrupted Signed-off-by: John Johansen <john.johansen@canonical.com>
2023-08-14 12:30:01 -07:00
'profile/flags/flags_bad64.sd',
'profile/flags/flags_bad65.sd',
'profile/flags/flags_bad66.sd',
parser: add kill.signal=XXX flag support Add a flag that allows setting the signal used to kill the process. This should not be normally used but can be very useful when debugging applications, interaction with apparmor. Signed-off-by: John Johansen <john.johansen@canonical.com>
2023-08-21 11:51:42 -07:00
'profile/flags/flags_bad67.sd',
'profile/flags/flags_bad68.sd',
'profile/flags/flags_bad69.sd',
parser: Add support for a default_allow mode Add support for a default_allow mode that facillitates writing profiles in that allow everything by default. This is not normally recomended but fascilitates creating basic profiles while working to transition policy away from unconfined. This mode is being added specifically to replace the use of the unconfined flag in these transitional profiles as the use of unconfined in policy is confusing and does not reflect the semantics of what is being done. Generally the goal for policy should be to remove all default_allow profiles once the policy is fully developed. Note: this patch only adds parsing of default_allow mode. Currently it sets the unconfined flag to achieve default allow but this prevents deny rules from being applied. Once dominance is fixed a subsequent patch will transition default_allow away from using the unconfined flag. Signed-off-by: John Johansen <john.johansen@canonical.com>
2023-10-10 02:22:29 -07:00
'profile/flags/flags_bad70.sd',
'profile/flags/flags_bad71.sd',
'profile/flags/flags_bad72.sd',
'profile/flags/flags_bad73.sd',
'profile/flags/flags_bad74.sd',
'profile/flags/flags_bad75.sd',
'profile/flags/flags_bad76.sd',
'profile/flags/flags_bad77.sd',
'profile/flags/flags_bad78.sd',
'profile/flags/flags_bad79.sd',
'profile/flags/flags_bad80.sd',
'profile/flags/flags_bad81.sd',
'profile/flags/flags_bad82.sd',
'profile/flags/flags_bad83.sd',
'profile/flags/flags_bad84.sd',
'profile/flags/flags_bad85.sd',
'profile/flags/flags_bad86.sd',
parser: add error=EXX flag support Add a flag that allows setting the error code AppArmor will send when an operation is denied. This should not be used normally. Signed-off-by: Georgia Garcia <georgia.garcia@canonical.com>
2024-04-15 16:32:16 -03:00
# flags=(error=EXX)
'profile/flags/flags_bad87.sd',
'profile/flags/flags_bad88.sd',
'profile/flags/flags_bad89.sd',
parser: add support for attach_disconnected.path Add support for specifying the path prefix used when attach disconnected is specified. The kernel supports prepending a different value than / when a path is disconnected. Expose through a profile flag. Signed-off-by: John Johansen <john.johansen@canonical.com>
2020-10-20 03:53:06 -07:00
'profile/flags/flags_bad_disconnected_path1.sd',
'profile/flags/flags_bad_disconnected_path2.sd',
'profile/flags/flags_bad_disconnected_path3.sd',
'profile/flags/flags_bad_disconnected_path4.sd',
'profile/flags/flags_bad_disconnected_path5.sd',
parser: add attach_disconnected.ipc parser tests Signed-off-by: Georgia Garcia <georgia.garcia@canonical.com>
2025-03-12 16:31:02 -03:00
'profile/flags/flags_bad_disconnected_ipc1.sd',
'profile/flags/flags_bad_disconnected_ipc2.sd',
'profile/flags/flags_bad_disconnected_ipc3.sd',
'profile/flags/flags_bad_disconnected_ipc4.sd',
'profile/flags/flags_bad_disconnected_ipc5.sd',
'profile/flags/flags_bad_disconnected_ipc6.sd',
'profile/flags/flags_bad_disconnected_ipc7.sd',
'profile/flags/flags_bad_disconnected_ipc8.sd',
Add simple_tests/profile/profile_ns_bad8.sd to utils test exception list parser/tst/simple_tests/profile/profile_ns_bad8.sd was added in r3376 (trunk) / r3312 (2.10 branch) and contains the profile name ':ns/t' which misses the terminating ':' for the namespace. Unfortunately the tools don't understand namespaces yet and just use the full profile name. This also means this test doesn't fail as expected when tested against the utils code. This patch adds profile_ns_bad8.sd to the exception list of test-parser-simple-tests.py. Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.10.
2016-02-19 00:22:59 +01:00
'profile/profile_ns_bad8.sd', # 'profile :ns/t' without terminating ':'
Use PtraceRule Change aa.py to use PtraceRule and PtraceRuleset in profile_storage(), parse_profile_data() and write_ptrace(). This also means we can drop the now unused parse_ptrace_rule() and write_ptrace_rules() functions. Raw_Ptrace_Rule in rules.py is now also unused and can be dropped. Also adjust logparser.py to include the peer in the result, and shorten the list of known-failing tests in test-parser-simple-tests.py. Acked-by: John Johansen <john.johansen@canonical.com>
2015-12-27 01:20:37 +01:00
'ptrace/bad_10.sd', # peer with invalid regex
Change aa.py to use SignalRule and SignalRuleset This means: - import the classes instead of RE_PROFILE_SIGNAL - simplify signal rule parsing a lot - drop the (now unused) functions parse_signal_rule() and write_signal_rules() - change write_signal() to use the SignalRuleset class Also drop the now unused Raw_Signal_Rule from rules.py. Finally, drop most parser signal tests from the "known wrong results" blacklist in test-parser-simple-tests.py because those tests succeed with SignalRule. Acked-by: Kshitij Gupta <kgupta8592@gmail.com>
2015-11-24 00:09:37 +01:00
'signal/bad_21.sd', # invalid regex
Adding userspace support for unix mediation
2024-03-29 13:09:06 +00:00
# Invalid regexes
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'unix/bad_regex_01.sd',
'unix/bad_regex_02.sd',
'unix/bad_regex_03.sd',
'unix/bad_regex_04.sd',
Adding userspace support for unix mediation
2024-03-29 13:09:06 +00:00
'unix/bad_modifier_2.sd', # We do not check for duplicated keywords
'unix/bad_bind_2.sd', # We do not check bind coherency
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'vars/vars_bad_3.sd',
'vars/vars_bad_4.sd',
'vars/vars_bad_5.sd',
'vars/vars_dbus_bad_01.sd',
'vars/vars_dbus_bad_02.sd',
'vars/vars_dbus_bad_03.sd',
utils: Don't enforce ordering of dbus rule attributes https://launchpad.net/bugs/1628286 The utils were enforcing that the dbus rule attributes were strictly ordered in the following fashion: bus -> path -> interface -> member -> peer However, the parser has always accepted the attributes in any order. If the system contained a profile which did not use the strict ordering enforced by the utils, the utils would refuse to operate at all. This patch eases the restriction on the ordering at the expense of the utils no longer being able to detect and reject a single attribute that is repeated multiple times. In that situation, only the last occurrence of the attribute will be honored by the utils. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2017-02-28 23:04:24 +00:00
'vars/vars_dbus_bad_04.sd',
'vars/vars_dbus_bad_05.sd',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'vars/vars_dbus_bad_06.sd',
'vars/vars_dbus_bad_07.sd',
'vars/vars_file_evaluation_8.sd',
# profile name in var doesn't start with /
'vars/vars_profile_name_bad_1.sd',
# profile name is undefined variable
'vars/vars_profile_name_bad_2.sd',
'vars/vars_profile_name_23.sd',
# attachment in var doesn't start with /
'vars/vars_profile_name_07.sd',
'vars/vars_profile_name_08.sd',
'vars/vars_profile_name_12.sd',
'vars/vars_profile_name_13.sd',
'vars/vars_profile_name_15.sd',
'vars/vars_profile_name_22.sd',
'vars/vars_recursion_1.sd',
'vars/vars_recursion_2.sd',
'vars/vars_recursion_3.sd',
'vars/vars_recursion_4.sd',
'vars/vars_simple_assignment_3.sd',
'vars/vars_simple_assignment_8.sd',
'vars/vars_simple_assignment_9.sd',
'xtrans/simple_bad_conflicting_x_10.sd',
'xtrans/simple_bad_conflicting_x_11.sd',
'xtrans/simple_bad_conflicting_x_12.sd',
'xtrans/simple_bad_conflicting_x_13.sd',
'xtrans/simple_bad_conflicting_x_2.sd',
'xtrans/simple_bad_conflicting_x_4.sd',
'xtrans/simple_bad_conflicting_x_6.sd',
'xtrans/simple_bad_conflicting_x_8.sd',
'xtrans/x-conflict.sd',
NetworkRule: Add support for fine-grained mediation rules
2024-04-23 20:07:19 +00:00
'network/perms/bad_modifier_2.sd',
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
)
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# testcases with lines that don't match any regex and end up as "unknown line"
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
unknown_line = (
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# 'other' keyword
'file/allow/ok_other_1.sd',
'file/allow/ok_other_2.sd',
'file/ok_other_1.sd',
'file/ok_other_2.sd',
'file/ok_other_3.sd',
[14/38] Use FileRule and FileRuleset Change aa.py to use FileRule and FileRuleset for parsing and saving profiles. In detail, this means: - add 'file' to the list of rule classes to enable it at various places - store file rules in aa[profile][hat]['file'] (not 'path' as before) to be consistent with the FileRule name - drop the no longer needed delete_path_duplicates() - this is now handled by FileRuleset like in all other rule classes. (same change in cleanprofile.py) - replace usage of RE_PROFILE_BARE_FILE_ENTRY and RE_PROFILE_PATH_ENTRY with FileRule.match() - drop write_path_rules() and write_paths() and replace them with the new write_file() function. - adjust several code sections to use write_file() and 'file' instead of 'path' FileRule doesn't drop optional keywords ('allow' and 'file'), therefore adjust cleanprof_test.out to the changed behaviour. (If someone insists on dropping optional keywords in aa-cleanprof, that's something for a future patch.) Also adjust the list of known failures in test-parser-simple-tests.py - switching to FileRule avoids several test failures (and introduces a few new ones ;-) IMPORTANT: This patch introduces a "brain split" which means - parsing and writing the profile and aa-cleanprof use the new location (aa[profile][hat]['file']) - aa-logprof and aa-genprof still save data to the old location (aa[profile][hat]['allow']['path']) and probably ask superfluous questions because there are no rules existing in the old location TL;DR: don't try aa-logprof or aa-genprof with only this patch applied. I know this isn't ideal, but still better than an even bigger and totally unreadable patch ;-) Acked-by: Steve Beattie <steve@nxnw.org>
2016-10-01 19:54:48 +02:00
# 'unsafe' keyword
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'file/file/front_perms_ok_1.sd',
'file/front_perms_ok_1.sd',
'xtrans/simple_ok_cx_1.sd',
# permissions before path and owner / audit {...} blocks
'file/file/owner/ok_1.sd',
'file/owner/ok_1.sd',
'profile/entry_mods_audit_ok1.sd',
# namespace
'profile/profile_ns_named_ok1.sd', # profile keyword?
'profile/profile_ns_named_ok2.sd', # profile keyword?
'profile/profile_ns_named_ok3.sd', # profile keyword?
'profile/profile_ns_ok1.sd',
'profile/profile_ns_ok2.sd',
'profile/profile_ns_ok3.sd', # profile keyword?
'profile/re_named_ok4.sd', # profile keyword
'profile/re_ok4.sd',
tools: Mark profiles with multiple rules in one line as known-failing The tools don't support having multiple rules in one line (they expect \n after each rule), therefore mark some of the bare_include_tests as known failures.
2018-01-23 22:40:07 +01:00
# multiple rules in one line
'bare_include_tests/ok_14.sd',
'bare_include_tests/ok_19.sd',
'bare_include_tests/ok_64.sd',
'bare_include_tests/ok_69.sd',
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include with quoted relative path
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_11.sd',
'bare_include_tests/ok_12.sd',
'bare_include_tests/ok_13.sd',
'bare_include_tests/ok_15.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include with unquoted relative path
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_16.sd',
'bare_include_tests/ok_17.sd',
'bare_include_tests/ok_18.sd',
'bare_include_tests/ok_20.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include with quoted relative path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_26.sd',
'bare_include_tests/ok_27.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include with quoted magic path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_28.sd',
'bare_include_tests/ok_29.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include with magic path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_30.sd',
'bare_include_tests/ok_31.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with quoted relative path
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_61.sd',
'bare_include_tests/ok_62.sd',
'bare_include_tests/ok_63.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with unquoted relative path
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_65.sd',
'bare_include_tests/ok_66.sd',
'bare_include_tests/ok_67.sd',
'bare_include_tests/ok_68.sd',
'bare_include_tests/ok_70.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with quoted relative path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_76.sd',
'bare_include_tests/ok_77.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with quoted magic path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_78.sd',
'bare_include_tests/ok_79.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with unquoted magic path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_80.sd',
'bare_include_tests/ok_81.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with quoted relative path, non-existing include file
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_82.sd',
'bare_include_tests/ok_84.sd',
'bare_include_tests/ok_85.sd',
'bare_include_tests/ok_86.sd',
Adding userspace support for mount rules in aa-genprof/aa-logprof
2024-02-29 17:59:50 +00:00
Adding userspace support for unix mediation
2024-03-29 13:09:06 +00:00
# Unsupported \\" in unix AARE
'unix/ok_regex_03.sd',
'unix/ok_regex_09.sd',
'unix/ok_regex_13.sd',
'unix/ok_regex_19.sd',
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
)
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# testcases with various unexpected failures
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
syntax_failure = (
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# missing profile keywords
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'profile/re_named_ok2.sd',
# Syntax Errors caused by boolean conditions (parse_profile_data() gets confused by the closing '}')
'conditional/defined_1.sd',
'conditional/defined_2.sd',
'conditional/else_1.sd',
'conditional/else_2.sd',
'conditional/else_3.sd',
'conditional/else_if_1.sd',
'conditional/else_if_2.sd',
'conditional/else_if_3.sd',
'conditional/else_if_5.sd',
'conditional/ok_1.sd',
'conditional/ok_2.sd',
'conditional/ok_3.sd',
'conditional/ok_4.sd',
'conditional/ok_5.sd',
'conditional/ok_6.sd',
'conditional/ok_7.sd',
'conditional/ok_8.sd',
'conditional/ok_9.sd',
'conditional/stress_1.sd',
# unexpected uppercase vs. lowercase in *x rules
'file/ok_5.sd', # Invalid mode UX
'file/ok_2.sd', # Invalid mode RWM
'file/ok_4.sd', # Invalid mode iX
test-parser-simple-tests.py: No longer skip testing generated_perms_leading profiles FileRule understands leading permissions, so the reason to skip those (generated) test profiles in test-parser-simple-tests.py is gone. However, the gen-xtrans.pl script generates profiles with a not-so-valid mix of uppercase and lowercase, for example "Pux" and "Cux". The parser accepts this, but the tools complain about such rules. Therefore add the affected profiles to the exception list. In total, this means we now test 319 of the 380 generated_perms_leading test profiles. The patch also moves some lines around to get the \-escaped profiles out of the mixed uppercase/lowercase exec rule section. Acked-by: Seth Arnold <seth.arnold@canonical.com>
2017-03-03 13:14:03 +01:00
'xtrans/simple_ok_pix_1.sd', # Invalid mode pIx
'xtrans/simple_ok_pux_1.sd', # Invalid mode rPux
# unexpected uppercase vs. lowercase in *x rules - generated_perms_leading directory
'generated_perms_leading/exact-re-Puxtarget.sd',
'generated_perms_leading/dominate-ownerCuxtarget2.sd',
'generated_perms_leading/ambiguous-Cux.sd',
'generated_perms_leading/dominate-ownerPux.sd',
'generated_perms_leading/exact-re-ownerPux.sd',
'generated_perms_leading/overlap-ownerCuxtarget.sd',
'generated_perms_leading/exact-re-ownerCuxtarget.sd',
'generated_perms_leading/dominate-Puxtarget2.sd',
'generated_perms_leading/dominate-ownerCuxtarget.sd',
'generated_perms_leading/dominate-ownerPuxtarget.sd',
'generated_perms_leading/ambiguous-Pux.sd',
'generated_perms_leading/ambiguous-Cuxtarget2.sd',
'generated_perms_leading/exact-Puxtarget2.sd',
'generated_perms_leading/ambiguous-ownerCux.sd',
'generated_perms_leading/exact-ownerPux.sd',
'generated_perms_leading/ambiguous-ownerPuxtarget.sd',
'generated_perms_leading/exact-re-ownerPuxtarget.sd',
'generated_perms_leading/exact-re-Cuxtarget.sd',
'generated_perms_leading/exact-re-Puxtarget2.sd',
'generated_perms_leading/dominate-Cux.sd',
'generated_perms_leading/exact-re-ownerCuxtarget2.sd',
'generated_perms_leading/ambiguous-ownerCuxtarget.sd',
'generated_perms_leading/exact-re-Cuxtarget2.sd',
'generated_perms_leading/ambiguous-Puxtarget.sd',
'generated_perms_leading/overlap-Puxtarget.sd',
'generated_perms_leading/ambiguous-Puxtarget2.sd',
'generated_perms_leading/overlap-Puxtarget2.sd',
'generated_perms_leading/exact-Puxtarget.sd',
'generated_perms_leading/overlap-ownerPuxtarget.sd',
'generated_perms_leading/exact-ownerCuxtarget.sd',
'generated_perms_leading/exact-re-ownerCux.sd',
'generated_perms_leading/exact-ownerPuxtarget2.sd',
'generated_perms_leading/exact-ownerCux.sd',
'generated_perms_leading/overlap-Cuxtarget2.sd',
'generated_perms_leading/ambiguous-Cuxtarget.sd',
'generated_perms_leading/ambiguous-ownerPuxtarget2.sd',
'generated_perms_leading/dominate-ownerCux.sd',
'generated_perms_leading/exact-Pux.sd',
'generated_perms_leading/exact-Cuxtarget.sd',
'generated_perms_leading/overlap-ownerCuxtarget2.sd',
'generated_perms_leading/overlap-Pux.sd',
'generated_perms_leading/overlap-ownerPux.sd',
'generated_perms_leading/ambiguous-ownerCuxtarget2.sd',
'generated_perms_leading/exact-re-Cux.sd',
'generated_perms_leading/exact-re-Pux.sd',
'generated_perms_leading/overlap-Cuxtarget.sd',
'generated_perms_leading/exact-re-ownerPuxtarget2.sd',
'generated_perms_leading/exact-Cuxtarget2.sd',
'generated_perms_leading/exact-Cux.sd',
'generated_perms_leading/overlap-Cux.sd',
'generated_perms_leading/overlap-ownerCux.sd',
'generated_perms_leading/exact-ownerPuxtarget.sd',
'generated_perms_leading/dominate-Pux.sd',
'generated_perms_leading/exact-ownerCuxtarget2.sd',
'generated_perms_leading/dominate-Puxtarget.sd',
'generated_perms_leading/ambiguous-ownerPux.sd',
'generated_perms_leading/overlap-ownerPuxtarget2.sd',
'generated_perms_leading/dominate-Cuxtarget2.sd',
'generated_perms_leading/dominate-Cuxtarget.sd',
'generated_perms_leading/dominate-ownerPuxtarget2.sd',
Quote trailing backslash in test case This fixes an error with Python 3.11: ``` test/test-parser-simple-tests.py:420:21: E502 the backslash is redundant between brackets ``` Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2024-11-25 13:54:03 +01:00
# escaping with "\"
[14/38] Use FileRule and FileRuleset Change aa.py to use FileRule and FileRuleset for parsing and saving profiles. In detail, this means: - add 'file' to the list of rule classes to enable it at various places - store file rules in aa[profile][hat]['file'] (not 'path' as before) to be consistent with the FileRule name - drop the no longer needed delete_path_duplicates() - this is now handled by FileRuleset like in all other rule classes. (same change in cleanprofile.py) - replace usage of RE_PROFILE_BARE_FILE_ENTRY and RE_PROFILE_PATH_ENTRY with FileRule.match() - drop write_path_rules() and write_paths() and replace them with the new write_file() function. - adjust several code sections to use write_file() and 'file' instead of 'path' FileRule doesn't drop optional keywords ('allow' and 'file'), therefore adjust cleanprof_test.out to the changed behaviour. (If someone insists on dropping optional keywords in aa-cleanprof, that's something for a future patch.) Also adjust the list of known failures in test-parser-simple-tests.py - switching to FileRule avoids several test failures (and introduces a few new ones ;-) IMPORTANT: This patch introduces a "brain split" which means - parsing and writing the profile and aa-cleanprof use the new location (aa[profile][hat]['file']) - aa-logprof and aa-genprof still save data to the old location (aa[profile][hat]['allow']['path']) and probably ask superfluous questions because there are no rules existing in the old location TL;DR: don't try aa-logprof or aa-genprof with only this patch applied. I know this isn't ideal, but still better than an even bigger and totally unreadable patch ;-) Acked-by: Steve Beattie <steve@nxnw.org>
2016-10-01 19:54:48 +02:00
'file/ok_embedded_spaces_4.sd', # \-escaped space
'file/file/ok_embedded_spaces_4.sd', # \-escaped space
'file/ok_quoted_4.sd', # quoted string including \"
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
Check for mount rules with multiple 'options' ... and adjust the tools to raise an exception if such a rule is found. While this is not nice, it's better than the previous behaviour where only the last 'options' was kept, and the others were lost when writing the rule.
2025-04-06 13:48:42 +02:00
# mount rules with multiple 'options' are not supported by the tools yet, and when writing them, only the last 'options' would survive. Therefore MountRule intentionally raises an exception when parsing such a rule.
'mount/ok_opt_87.sd',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# misc
util: enhance AARE file path validation Fixes https://bugs.launchpad.net/apparmor/+bug/2106033 Improve the validation of AARE file paths by introducing a new regex that supports paths starting with '{' (e.g. '{/,/org/freedesktop/DBus}'). These paths are notably used in snap.lxd.* profiles. Signed-off-by: Maxime Bélair <maxime.belair@canonical.com>
2025-04-02 17:24:41 +02:00
'vars/vars_dbus_12.sd', # AARE starting with {{ are not handled
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'vars/vars_simple_assignment_12.sd', # Redefining existing variable @{BAR} ('\' not handled)
'bare_include_tests/ok_2.sd', # two #include<...> in one line
parser: add network conditional parser tests
2023-08-30 14:50:33 -03:00
parser: add port range support on network policy Signed-off-by: Georgia Garcia <georgia.garcia@canonical.com>
2024-09-05 11:32:15 -03:00
# network port range
'network/network_ok_17.sd',
Add more tests for network port range
2024-09-10 22:58:29 +02:00
'network/network_ok_45.sd',
'network/network_ok_46.sd',
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
)
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
Fix most PEP 8 whitespace, indentation, and major line length violations.
2022-08-07 12:26:24 -04:00
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
class TestParseParserTests(AATest):
tests = [] # filled by parse_test_profiles()
def _run_test(self, params, expected):
with open_file_read(params['file']) as f_in:
data = f_in.readlines()
if params['disabled']:
# skip disabled testcases
return
if params['tools_wrong']:
# if the tools are marked as being wrong about a profile, expect the opposite result
# this makes sure we notice any behaviour change, especially not being wrong anymore
expected = not expected
Check variable errors when parsing simple_tests ... by calling active_profiles.get_all_merged_variables() Also remove vars/vars_bad_add_assignment_1.sd from the exception_not_raised list again - now it raises an exception as expected.
2020-05-23 22:50:20 +02:00
# make sure the profile is known in active_profiles.files
apparmor.active_profiles.init_file(params['file'])
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
if expected:
Add in_preamble parameter to profile loading/parsing functions in_preamble keeps track of the current parsing position. It's True while parsing the preamble of a profile file, and when loading an include file that is included in the preamble (typically tunables/*). While inside a profile or parsing abstractions/*, it is False. This commit only hands the information around and keeps in_preamble updated, but it doesn't really get used yet. Also adjust the tests to hand over the additional parameter to parse_profile_data().
2021-02-22 22:22:25 +01:00
apparmor.parse_profile_data(data, params['file'], 0, True)
Drop profile_dir parameter from ProfileList get_all_merged_variables() This parameter is superfluous and unused since some commits. Also adjust all callers.
2020-06-01 23:24:22 +02:00
apparmor.active_profiles.get_all_merged_variables(params['file'], apparmor.include_list_recursive(apparmor.active_profiles.files[params['file']]))
Check variable errors when parsing simple_tests ... by calling active_profiles.get_all_merged_variables() Also remove vars/vars_bad_add_assignment_1.sd from the exception_not_raised list again - now it raises an exception as expected.
2020-05-23 22:50:20 +02:00
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
else:
with self.assertRaises(AppArmorException):
Add in_preamble parameter to profile loading/parsing functions in_preamble keeps track of the current parsing position. It's True while parsing the preamble of a profile file, and when loading an include file that is included in the preamble (typically tunables/*). While inside a profile or parsing abstractions/*, it is False. This commit only hands the information around and keeps in_preamble updated, but it doesn't really get used yet. Also adjust the tests to hand over the additional parameter to parse_profile_data().
2021-02-22 22:22:25 +01:00
apparmor.parse_profile_data(data, params['file'], 0, True)
Drop profile_dir parameter from ProfileList get_all_merged_variables() This parameter is superfluous and unused since some commits. Also adjust all callers.
2020-06-01 23:24:22 +02:00
apparmor.active_profiles.get_all_merged_variables(params['file'], apparmor.include_list_recursive(apparmor.active_profiles.files[params['file']]))
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
Fix most PEP 8 whitespace, indentation, and major line length violations.
2022-08-07 12:26:24 -04:00
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
def parse_test_profiles(file_with_path):
Use triple double-quoted strings for docstrings.
2022-08-07 14:57:30 -04:00
"""parse the test-related headers of a profile (for example EXRESULT) and add the profile to the set of tests"""
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
exresult = None
exresult_found = False
description = None
todo = False
disabled = False
with open_file_read(file_with_path) as f_in:
data = f_in.readlines()
relfile = os.path.relpath(file_with_path, apparmor.profile_dir)
for line in data:
if line.startswith('#=EXRESULT '):
exresult = line.split()[1]
if exresult == 'PASS':
Set (instead of compare) exresult Interestingly this accidentally worked because `if exresult` is true for both a non-empty string ("PASS") as well as a real `True` value. Found by Mark Grassi as part of https://gitlab.com/apparmor/apparmor/-/merge_requests/906
2022-08-14 12:33:56 +02:00
exresult = True
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
exresult_found = True
elif exresult == 'FAIL':
exresult = False
exresult_found = True
else:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception('{} contains unknown EXRESULT {}'.format(file_with_path, exresult))
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
elif line.upper().startswith('#=DESCRIPTION '):
description = line.split()[1]
elif line.rstrip() == '#=TODO':
todo = True
elif line.rstrip() == '#=DISABLED':
disabled = True
if not exresult_found:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception(file_with_path + ' does not contain EXRESULT')
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
if not description:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception(file_with_path + ' does not contain description')
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
tools_wrong = False
if relfile in exception_not_raised:
if exresult:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception(file_with_path + " listed in exception_not_raised, but has EXRESULT PASS")
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
tools_wrong = 'EXCEPTION_NOT_RAISED'
elif relfile.startswith(skip_startswith):
return 1 # XXX *** SKIP *** those tests
elif relfile in unknown_line:
if not exresult:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception(file_with_path + " listed in unknown_line, but has EXRESULT FAIL")
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
tools_wrong = 'UNKNOWN_LINE'
elif relfile in syntax_failure:
if not exresult:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception(file_with_path + " listed in syntax_failure, but has EXRESULT FAIL")
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
tools_wrong = 'SYNTAX_FAILURE'
params = {
'file': file_with_path,
'relfile': relfile,
'todo': todo,
'disabled': disabled,
'tools_wrong': tools_wrong,
'exresult': exresult,
}
TestParseParserTests.tests.append((params, exresult))
return 0
def find_and_setup_test_profiles(profile_dir):
Use triple double-quoted strings for docstrings.
2022-08-07 14:57:30 -04:00
"""find all profiles in the given profile_dir, excluding
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
- skippable files
- include directories
- files in the main directory (readme, todo etc.)
Use triple double-quoted strings for docstrings.
2022-08-07 14:57:30 -04:00
"""
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
skipped = 0
profile_dir = os.path.abspath(profile_dir)
apparmor.profile_dir = profile_dir
utils test: fix typo in simple_tests output Signed-off-by: Steve Beattie <steve.beattie@canonical.com>
2020-05-05 11:54:49 -07:00
print('Searching for parser simple_tests... (this will take a while)')
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
for root, dirs, files in os.walk(profile_dir):
relpath = os.path.relpath(root, profile_dir)
if relpath == '.':
# include files are checked as part of the profiles that include them (also, they don't contain EXRESULT)
dirs.remove('includes')
dirs.remove('include_tests')
dirs.remove('includes-preamble')
for file in files:
file_with_path = os.path.join(root, file)
import is_skippable_file from apparmor.common ... instead of indirectly using it via apparmor.aa
2021-08-24 22:47:39 +02:00
if not is_skippable_file(file) and relpath != '.':
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
skipped += parse_test_profiles(file_with_path)
if skipped:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
print('Skipping {} test profiles listed in skip_startswith.'.format(skipped))
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
print('Running {} parser simple_tests...'.format(len(TestParseParserTests.tests)))
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
utils: Require apparmor.aa users to call init_aa() Introduce an apparmor.aa.init_aa() method and move the initialization code of the apparmor.aa module into it. Note that this change will break any external users of apparmor.aa because global variables that were previously initialized when importing apparmor.aa will not be initialized unless a call to the new apparmor.aa.init_aa() method is made. The main purpose of this change is to allow the utils tests to be able to set a non-default location for configuration files. Instead of hard-coding the location of logprof.conf and other utils related configuration files to /etc/apparmor/, this patch allows it to be configured by calling apparmor.aa.init_aa(confdir=PATH). This allows for the make check target to use the in-tree config file, profiles, and parser by default. A helper method, setup_aa(), is added to common_test.py that checks for an environment variable containing a non-default configuration directory path prior to calling apparmor.aa.init_aa(). All test scripts that use apparmor.aa are updated to call setup_aa(). Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Suggested-by: Christian Boltz <apparmor@cboltz.de> Acked-by: Seth Arnold <seth.arnold@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2017-03-02 21:21:53 +00:00
setup_aa(apparmor)
Check variable errors when parsing simple_tests ... by calling active_profiles.get_all_merged_variables() Also remove vars/vars_bad_add_assignment_1.sd from the exception_not_raised list again - now it raises an exception as expected.
2020-05-23 22:50:20 +02:00
profile_dir = os.path.abspath('../../parser/tst/simple_tests/')
find_and_setup_test_profiles(profile_dir)
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
setup_all_loops(__name__)
if __name__ == '__main__':
make utils tests less verbose Given the big number of tests, printing a dot for each test (instead of multiple lines) is enough ;-)
2018-04-08 20:18:30 +02:00
unittest.main(verbosity=1)
Reference in New Issue Copy Permalink