Two modes of pre-dump algorithm:
1) splicing memory by parasite
--pre-dump-mode=splice (default)
2) using process_vm_readv syscall
--pre-dump-mode=read
Signed-off-by: Abhishek Dubey <dubeyabhishek777@gmail.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
As discussed on the mailing list, current .py files formatting does not
conform to the world standard, so we should better reformat it. For this
the yapf tool is used. The command I used was
yapf -i $(find -name *.py)
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
In zdtm.py, the page size is hardcoded as 4096, but on ppc64le, is is equal
to 64K and all test fail with errors like this:
ERROR: bad page counts, stats = 13 real = 208(0)
Signed-off-by: Andrei Vagin <avagin@gmail.com>
It is inspired by the discussion about inotify fix:
https://github.com/checkpoint-restore/criu/pull/728#issuecomment-506929427
From one point of view, warnings might be important to understand why we
detect some visible change in the environment after c/r-ing the process,
and if this change is expected or not. So we should add "Warn" messages
to the output.
From over point, these warnings if they are expected, can spoil our
final logs with a lot of unnecessary details, so add changes in previous
patches to silence the most noisy of these warnings.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
After dump command -- verify that the amount of bytes counted in
stats-dump matches the real sizes of pages-*.img files.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
The built-in bool() function returns a boolean value by converting the
input using standard truth testing procedure.
https://docs.python.org/3/library/functions.html#bool
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
Suppress the false positive fail in criu-live-migration job:
https://ci.openvz.org/job/CRIU/job/criu-live-migration/job/criu-dev/1796/
[criu]# ./test/zdtm.py run -t zdtm/static/overmounted_file -f uns --lazy-migrate
=== Run 1/1 ================ zdtm/static/overmounted_file
=================== Run zdtm/static/overmounted_file in uns ====================
Start test
Test is SUID
./overmounted_file --pidfile=overmounted_file.pid --outfile=overmounted_file.out --dirname=overmounted_file.test
Run criu dump
Test zdtm/static/overmounted_file FAIL at criu dump exited with 1 ######
Send the 9 signal to 49
Wait for zdtm/static/overmounted_file(49) to die for 0.100000
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
Currently, we rely on the fact that nobody else is using random and
parent and child processes will get the same random values.
Now, this test fails on the fedora rawhide. There will be nothing wrong
if we will generate test messages before forking the test process.
Signed-off-by: Andrei Vagin <avagin@gmail.com>
safe_dump produces only standard YAML tags and
cannot represent an arbitrary Python object.
Reported-by: Mr Jenkins
Signed-off-by: Andrei Vagin <avagin@gmail.com>
Currently, we create only one external resource to check how it is
handled by criu. It is better to create more than one.
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
CPython currently uses a reference-counting scheme with (optional)
delayed detection of cyclically linked garbage, which collects most
objects as soon as they become unreachable, but is not guaranteed to
collect garbage containing circular references.
Some objects contain references to "external" resources such as open
files. It is understood that these resources are freed when the object
is garbage-collected, but since garbage collection is not guaranteed to
happen, such objects also provide an explicit way to release the
external resource, usually a close() method.
Programs are strongly recommended to explicitly close such objects.
Reference: https://docs.python.org/3.6/reference/datamodel.html
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
When --remote-lazy-pages is used the address option was not specified.
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
When zdtm.py is executed with `list` sub-command the 'criu_bin'
option is not defined and criu.available() fails.
$ python test/zdtm.py list
Traceback (most recent call last):
File "test/zdtm.py", line 2243, in <module>
criu.available()
File "test/zdtm.py", line 1185, in available
if not os.access(opts['criu_bin'], os.X_OK):
KeyError: u'criu_bin'
However, we don't need to check the existence of criu_bin
unless we use the `run` action.
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
Acked-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
After running criu test, docker on the node becomes unusable, as it is
confused by our leftover cgroups. Surely docker should be fixed to
ignore custom cgroups (https://github.com/moby/moby/issues/37601), but
we should not leave them after test also.
v2: rmdir the holder only if it exists, remove racy wait and remove
wrongly added cleanup method in class criu
v3: bring back missed semicolon
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
We already check (root, mountpoint) pairs preserve, do the same for
(root, mountpoint, shared, slave) fours.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Creating a test for verifying configuration parsing feature. The
test is created by reusing already present inotify_irmap test.
Because of addition of default configuration files, --no-default-config
option is added to zdtm.py to not break the test suite on systems with
these files present.
Signed-off-by: Veronika Kabatova <vkabatov@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Send pre-dump notify to 36
Traceback (most recent call last):
File "zdtm.py", line 2161, in <module>
do_run_test(tinfo[0], tinfo[1], tinfo[2], tinfo[3])
File "zdtm.py", line 1549, in do_run_test
cr(cr_api, t, opts)
File "zdtm.py", line 1264, in cr
test.pre_dump_notify()
File "zdtm.py", line 490, in pre_dump_notify
fdin.write(struct.pack("i", 0))
TypeError: write() argument 1 must be unicode, not str
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
If pre-dump-notify flag is set, zdtm sends a notify to the test after
pre-dump was finished and waits for the test to send back a reply that
test did all it's work and now is ready for a next pre-dump/dump.
How it can be used:
while (!test_wait_pre_dump()) {
/* Do something after predump */
test_wait_pre_dump_ack();
}
/* Do something after restore */
Internally we open two pipes for the test one for receiving notify (with
two open ends) and one for replying to it (only write end open). Fds of
pipes are dupped to predefined numbers and zdtm opens these fds through
/proc/<test-pid>/fd/{100,101} and communicates with the test.
v9: switch to two way interface to remove race then operation we try to
run after predump may be yet unfinished at the time of next dump.
Suggested-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
We have a problem after commit 212e4c771a1b ("test: make zdtm.py
python2/python3 compatible") when running tests on python2:
https://ci.openvz.org/job/CRIU/job/CRIU-virtuozzo/job/criu-dev/3804/console
Traceback (most recent call last):
File "./test/zdtm.py", line 2249, in <module>
opts['action'](opts)
File "./test/zdtm.py", line 2001, in run_tests
launcher.run_test(t, tdesc, run_flavs)
File "./test/zdtm.py", line 1680, in run_test
self.wait()
File "./test/zdtm.py", line 1737, in wait
self.__wait_one(0)
File "./test/zdtm.py", line 1725, in __wait_one
print(open(sub['log']).read())
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in
position 258: ordinal not in range(128)
print does not like "‘" symbol in zdtm_static_cgroup04.log:
...
rmdir: failed to remove ‘cgclean.sKFHLm/zdtmtst/special_prop_check’: No
such file or directory
Small reproducer:
[snorch@snorch ~]$ cat test_ascii.py
from __future__ import absolute_import, division, print_function, unicode_literals
from builtins import (str, open, range, zip, int, input)
f = open('./zdtm_static_cgroup04.log')
s = f.read()
print(s)
[snorch@snorch ~]$ python test_ascii.py | grep ""
Traceback (most recent call last):
File "test_ascii.py", line 6, in <module>
print(s)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in
position 258: ordinal not in range(128)
So just ignore these quote symbol when printing logs.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
We alwasy exclude vsyscall vma-s for x32 tests, but
this part was broken by python3 patches.
1: Old maps lost: set([])
1: New maps appeared: set([u"ffffffffff600000-ffffffffff601000 [u'r-xp']"])
Reported-by: Mr Jenkins
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
I am regularly running zdtm.py after updating the CRIU rpm to test if
the new rpm still works. Until know I usually did:
-criu_bin = "../criu/criu"
-crit_bin = "../crit/crit"
+criu_bin = "/usr/sbin/criu"
+crit_bin = "/usr/bin/crit"
This commit adds two arguments to zdtm.py:
--criu-bin CRIU_BIN Path to criu binary
--crit-bin CRIT_BIN Path to crit binary
It still defaults to the old values from above, but can now easily be
changed to the CRIU binary provided by the packaging system.
This change was more complicated than expected which is probably related
to the fact that zdtm restarts itself in namespaces.
v2:
- rebase
v3:
- fix errors reported by 'make lint'
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Send pre-dump notify to 36
Traceback (most recent call last):
File "zdtm.py", line 2161, in <module>
do_run_test(tinfo[0], tinfo[1], tinfo[2], tinfo[3])
File "zdtm.py", line 1549, in do_run_test
cr(cr_api, t, opts)
File "zdtm.py", line 1264, in cr
test.pre_dump_notify()
File "zdtm.py", line 490, in pre_dump_notify
fdin.write(struct.pack("i", 0))
TypeError: write() argument 1 must be unicode, not str
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Here is an example of ldd output:
/musl # ldd /usr/lib/libxtables.so.12
ldd (0x7fae8e06c000)
libc.musl-x86_64.so.1 => ldd (0x7fae8e06c000)
We need to skip the "ldd (0x7fae8e06c000)" line.
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
As python3 imports are working a bit differently the structure of the
imported files needs to change. Instead of having symlinks to criu.py
and rpc_pb2.py which does not match the import structure just create
symlink to ../lib/py (like crit) to make zdtm usable again with the
python3 compatibility.
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
iptables 1.6.2 fails without /run, because it tries to create
the /run/xtables.lock file
Test output: ================================
Fatal: can't open lock file /run/xtables.lock: No such file or directory
23:29:06.098: 4: ERR: netns-nf.c:21: Can't set input rule (errno = 2 (No such file or directory))
23:29:06.098: 3: ERR: test.c:315: Test exited unexpectedly with code 255
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Currently zdtm doesn't detect when restore failed, if it is executed
with strace. With this patch, fake-restore.sh creates a test file, and
zdtm is able to distinguish when restore failed.
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
The get() method requires a key and now we are using an index. That
will never work correctly as it is now.
Acked-by: Adrian Reber <adrian@lisas.de>
Reported-by: Adrian Reber <adrian@lisas.de>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
