This patch rewrites the caching test in python, using python's unittest
framework. It has been used with python 2.7 and python 3.3; python2.6
may have issues. It covers the tests in the existing caching.sh
test script (with the exception of the test that checks for when the
parser in $PATH is newer), as well as adding additional tests that
more extensively cover using a cache in an alternate location from
basedir. It also adds simple tests for the --create-cache-dir option
(along with that option's interaction with the alt-cache option).
(Some further work to be done is listed under TODO.)
Patch history:
v1: - initial version
v2: - create template base class
- add keep_on_fail() decorator to keep temporary test files
around after a test fails
- don't dump raw cache file to failure output in
test_cache_writing_updates_cache_file()
- push run_cmd into template class
- create run_cmd_check wrapper to run_cmd that adds an assertion
check based on whether return code matches the expected rc
(the valgrind tests only want to verify that the rc is not a
specific set of values, hence the separate wrapper function)
- similarly, add a check to run_cmd_check for verifying the output
contains a specific string, also simplifying many of the caching
tests.
- create testlib.write_file() to simplify writing file
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Christian Boltz <apparmor@cboltz.de>
This patch adds a test wrapper that runs valgrind on the parser over the
simple_tests tree (or other directory tree if passed on the command
line). An alternate parser location can also be passed on the command
line.
Like the libapparmor python bindings test, this test uses a bit of magic
to generate tests that doesn't work with auto-detecting test utilities
like nose.
Running valgrind on the parser over all 69000+ testcases takes several
hours, so while this patch includes a make target 'make valgrind', it
does not add it to the set of tests run when 'make check' is called.
Perhaps a 'make extra-tests' target is in order.
Patch history:
v1: - initial version.
v2: - add some valgrind suppressions for overaggressive 4 byte reads
past the end of allocated storage (not completed).
v3: - add ability to dump valgrind suppressions to stdout, to use
diagnosis runs of valgrind for determining whether a given
failure is a false positive or not.
- correctly return 0 on a successful run and an error code if one
or more test cases fail.
- point LD_LIBRARY_PATH at the in-tree libapparmor build.
- split out some utility functions into testlib.py, for possible
use by other to be written test scripts
v4: - convert optparse to argparse
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Tyler Hicks <tyhicks@canonical.com> (for v2 version)
