We have a problem after commit 212e4c771a1b ("test: make zdtm.py
python2/python3 compatible") when running tests on python2:
https://ci.openvz.org/job/CRIU/job/CRIU-virtuozzo/job/criu-dev/3804/console
Traceback (most recent call last):
File "./test/zdtm.py", line 2249, in <module>
opts['action'](opts)
File "./test/zdtm.py", line 2001, in run_tests
launcher.run_test(t, tdesc, run_flavs)
File "./test/zdtm.py", line 1680, in run_test
self.wait()
File "./test/zdtm.py", line 1737, in wait
self.__wait_one(0)
File "./test/zdtm.py", line 1725, in __wait_one
print(open(sub['log']).read())
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in
position 258: ordinal not in range(128)
print does not like "‘" symbol in zdtm_static_cgroup04.log:
...
rmdir: failed to remove ‘cgclean.sKFHLm/zdtmtst/special_prop_check’: No
such file or directory
Small reproducer:
[snorch@snorch ~]$ cat test_ascii.py
from __future__ import absolute_import, division, print_function, unicode_literals
from builtins import (str, open, range, zip, int, input)
f = open('./zdtm_static_cgroup04.log')
s = f.read()
print(s)
[snorch@snorch ~]$ python test_ascii.py | grep ""
Traceback (most recent call last):
File "test_ascii.py", line 6, in <module>
print(s)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in
position 258: ordinal not in range(128)
So just ignore these quote symbol when printing logs.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
We alwasy exclude vsyscall vma-s for x32 tests, but
this part was broken by python3 patches.
1: Old maps lost: set([])
1: New maps appeared: set([u"ffffffffff600000-ffffffffff601000 [u'r-xp']"])
Reported-by: Mr Jenkins
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
I am regularly running zdtm.py after updating the CRIU rpm to test if
the new rpm still works. Until know I usually did:
-criu_bin = "../criu/criu"
-crit_bin = "../crit/crit"
+criu_bin = "/usr/sbin/criu"
+crit_bin = "/usr/bin/crit"
This commit adds two arguments to zdtm.py:
--criu-bin CRIU_BIN Path to criu binary
--crit-bin CRIT_BIN Path to crit binary
It still defaults to the old values from above, but can now easily be
changed to the CRIU binary provided by the packaging system.
This change was more complicated than expected which is probably related
to the fact that zdtm restarts itself in namespaces.
v2:
- rebase
v3:
- fix errors reported by 'make lint'
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Send pre-dump notify to 36
Traceback (most recent call last):
File "zdtm.py", line 2161, in <module>
do_run_test(tinfo[0], tinfo[1], tinfo[2], tinfo[3])
File "zdtm.py", line 1549, in do_run_test
cr(cr_api, t, opts)
File "zdtm.py", line 1264, in cr
test.pre_dump_notify()
File "zdtm.py", line 490, in pre_dump_notify
fdin.write(struct.pack("i", 0))
TypeError: write() argument 1 must be unicode, not str
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Here is an example of ldd output:
/musl # ldd /usr/lib/libxtables.so.12
ldd (0x7fae8e06c000)
libc.musl-x86_64.so.1 => ldd (0x7fae8e06c000)
We need to skip the "ldd (0x7fae8e06c000)" line.
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
As python3 imports are working a bit differently the structure of the
imported files needs to change. Instead of having symlinks to criu.py
and rpc_pb2.py which does not match the import structure just create
symlink to ../lib/py (like crit) to make zdtm usable again with the
python3 compatibility.
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
iptables 1.6.2 fails without /run, because it tries to create
the /run/xtables.lock file
Test output: ================================
Fatal: can't open lock file /run/xtables.lock: No such file or directory
23:29:06.098: 4: ERR: netns-nf.c:21: Can't set input rule (errno = 2 (No such file or directory))
23:29:06.098: 3: ERR: test.c:315: Test exited unexpectedly with code 255
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
If pre-dump-notify flag is set, zdtm sends a notify to the test after
pre-dump was finished and waits for the test to send back a reply that
test did all it's work and now is ready for a next pre-dump/dump.
How it can be used:
while (!test_wait_pre_dump()) {
/* Do something after predump */
test_wait_pre_dump_ack();
}
/* Do something after restore */
Internally we open two pipes for the test one for receiving notify (with
two open ends) and one for replying to it (only write end open). Fds of
pipes are dupped to predefined numbers and zdtm opens these fds through
/proc/<test-pid>/fd/{100,101} and communicates with the test.
v9: switch to two way interface to remove race then operation we try to
run after predump may be yet unfinished at the time of next dump.
Suggested-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Currently zdtm doesn't detect when restore failed, if it is executed
with strace. With this patch, fake-restore.sh creates a test file, and
zdtm is able to distinguish when restore failed.
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
The get() method requires a key and now we are using an index. That
will never work correctly as it is now.
Acked-by: Adrian Reber <adrian@lisas.de>
Reported-by: Adrian Reber <adrian@lisas.de>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
The --lazy-migrate option allows testing of lazy migration when running ns
or uns flavor.
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
$ make lint
flake8 --config=scripts/flake8.cfg test/zdtm.py
test/zdtm.py:323:19: F841 local variable 'e' is assigned to but never used
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Start test
./mxcsr --pidfile=mxcsr.pid --outfile=mxcsr.out
Run criu dump
Unable to kill 44: [Errno 3] No such process <--------------- this one
Run criu restore
Run criu dump
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Reviewed-by: Dmitry Safonov <0x7f454c46@gmail.com>
The idea of the check-only option is that criu dump and criu
restore is executed with this option to check whether c/r is
possible for a set of processes. This has to work faster than
without the check-only option.
Now we run criu restore --check-only for images which have
been generated by criu dump without --check-only, it is obviously wrong.
Cc: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
If the check-only option is set, dump and restore is executed twice,
and we need to set separate logs for both cases.
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
If the restore was exexuted with the check-only option,
after restoring all resources tasks waits children and
exits with the 0 code.
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Creating a test for verifying configuration parsing feature. The
test is created by reusing already present inotify_irmap test.
Because of addition of default configuration files, --no-default-config
option is added to zdtm.py to not break the test suite on systems with
these files present.
Signed-off-by: Veronika Kabatova <vkabatov@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
This adds the option '--check-only' to zdtm.py. If specified each test
cases is first dumped with the '--check-only' option enabled before the
real dump. Also during restore the test case is first restored with
--check-only before doing the real restore.
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
If calling gone() without ever calling getpid() before leads to
backtrace. Just call getpid() to avoid that.
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
In preparation for the zdtm option '--check-only' a new helper function
reset_pid() is added which writes to ns_last_pid to avoid PID collisions
during check-only restore and the real restore.
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Currently, one feature is supported. Add possibility
for a test to depend on several features.
v2: Delete excess "if" as suggested by Andrey Vagin.
Rename variables to decrise patch size.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
This can help to investigate logs from Mr Jenkins.
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
v2: defining crit_bin and using it for Popen() // Mike
Acked-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
flake8 was updated recently and now it shows a few new warnings:
[root@fc24 criu]# make lint
flake8 --config=scripts/flake8.cfg test/zdtm.py
test/zdtm.py:181:4: E722 do not use bare except'
test/zdtm.py:304:2: E722 do not use bare except'
test/zdtm.py:325:3: E722 do not use bare except'
test/zdtm.py:445:3: E722 do not use bare except'
test/zdtm.py:573:4: E722 do not use bare except'
test/zdtm.py:1369:2: E722 do not use bare except'
test/zdtm.py:1385:3: E722 do not use bare except'
test/zdtm.py:1396:2: E722 do not use bare except'
test/zdtm.py:1420:3: E722 do not use bare except'
test/zdtm.py:1820:2: E741 ambiguous variable name 'l'
make: *** [Makefile:369: lint] Error 1
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
test/zdtm.py:1535:30: E711 comparison to None should be 'if cond is not None:'
test/zdtm.py:1538:31: E711 comparison to None should be 'if cond is not None:'
test/zdtm.py:1601:31: E711 comparison to None should be 'if cond is not None:'
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
We found a good plugin for Jenkins to analize test results,
but it can handle only the junit format.
v2: work without the junit python module
v3: install junit-xml, because zdtm.py is executed
with the --report option from jenkins scripts.
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Raise an exception for kernels that do not have userfaultfd. For the
kernels that have userfaultfd but do not provide non-cooperative events
(4.3 - 4.11) just print a warning.
Fixes: #363
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
This allows skipping tests that are not yet run with --remote-lazy-pages,
but can be run with --lazy-pages
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
When running with --lazy-pages or --remote-lazy-pages, the daemons should
run in the background, rather than complete before t.stop() is called.
Many tests try to verify things are ok after test_waitsig() and that's
exactly the place where they access memory and cause page faults.
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
It works faster and allows to check exit codes.
travis-ci: success for series starting with [1/2] page-server: don't return a server pid from page-server
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
travis-ci: success for crtools: close a signal descriptor after passing a preparation stage (rev6)
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Currently we are waiting for lazy-pages daemon to finish as a part of
.restore method, which may cause filling test process memory before the
test process resumed it's execution after call to test_waitsig(). In such
case, no page faults occur, but rather all the memory is copied in
handle_remaining_pages method in uffd.c.
Let's move wait(<lazy-pages-pid>,..) after call to test.stop().
travis-ci: success for lazy-pages: add non-#PF events handling
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Lazy migration requires both dumped and restored processes to coexist at
the same time. This breaks some basic assumptions in the zdtm design.
Simulation of lazy migration with the page server allows testing most of
the involved code paths without major intervention into zdtm
infrastructure.
travis-ci: success for lazy-pages: improve testability (rev2)
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
The kernel support for lazy pages (userfaultfd) lacks many important
features which effectively prevents success in certain tests.
Allow skipping such test with somewhat informative message
travis-ci: success for lazy-pages: improve testability (rev2)
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Right now the zdtm.py hacks around core code and waits for
a second for the socket to appear. Let's better make proper
--daemon mode for lazy-pages daemon and pidfile generation.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Acked-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
