2
0
mirror of https://github.com/checkpoint-restore/criu synced 2025-08-29 13:28:27 +00:00

10586 Commits

Author SHA1 Message Date
Radostin Stoyanov
f70605ef1e criu-ns: update script name in help message
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-09-03 10:31:00 -07:00
Radostin Stoyanov
f472e2590e Documentation: Add man page for criu-ns
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-09-03 10:31:00 -07:00
Radostin Stoyanov
8891e51cd4 make: install criu-ns
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-09-03 10:31:00 -07:00
Zeyad Yasser
4a9bcd8844 zdtm: prioritize /lib/* dependencies in some tests
Prioritize /lib/* because iptables fails to search /usr/lib64/*
first on archlinux.

This change of 'deps' order prioritizes the default library location.

This affects:
	- zdtm/static/netns-nf
	- zdtm/static/netns-nft-ipt
	- zdtm/static/socket-tcp-closed-last-ack
	- zdtm/static/socket-tcp-reseted
	- zdtm/static/socket-tcp-syn-sent

Signed-off-by: Zeyad Yasser <zeyady98@gmail.com>
2021-09-03 10:31:00 -07:00
Zeyad Yasser
00ca2b519e scripts/build: add a docker file for archlinux
Signed-off-by: Zeyad Yasser <zeyady98@gmail.com>
2021-09-03 10:31:00 -07:00
Zeyad Yasser
694eafa1f6 protobuf: remove leading underscores from protobuf structs
Fixes: #1560

The latest protobuf-c compiler breaks CRIU because they removed
leading underscores from structs in 1.4.0.

This replaces those definitions with the standard generated structs.

v2: remove struct _VmaEntry, struct _CredsEntry and struct _CoreEntry

Signed-off-by: Zeyad Yasser <zeyady98@gmail.com>
2021-09-03 10:31:00 -07:00
Andrei Vagin
efb9fccd41 cgroup: cgroup_contains has to update the mask for cgroupv2
Fixes: #1524
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
ac27562f09 ci: add msgque test case to crit-recode
crit-recode on msgque tests was broken in Jenkins. Run it also in GitHub
CI.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
7e86519fe3 lib: fix crit-recode msgque errors in Jenkins
b'test/dump/zdtm/static/msgque/63/1/ipcns-msg-11.img'  encode fails: expected bytes-like object, not str
b'test/dump/zdtm/static/msgque/63/1/ipcns-msg-11.img' pretty  encode fails: expected bytes-like object, not str
FAIL

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Radostin Stoyanov
5034885974 ci/openj9: run mrproper before make
Make sure to remove all files created from previous local build
before compiling in the container.

Reported-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-09-03 10:31:00 -07:00
Pavel Tikhomirov
7ff785e1d4 zdtm: make --sbs also stop on each pre-dump/snap iteration
This is useful to investigate problems on pre-dump iterations. After
this patch test output with "--pre=2 --sbs" would have new usefull stop
points.

While on it let's remove confusion in sbs stop point naming. "Pause at
pre-dump" actually has nothing to do with pre-dump, let's better use
"before " instead of "at pre-", similar let's use "after " instead of
"at post-".

Result would look like:

========================== Run zdtm/static/env00 in h ==========================
Start test
./env00 --pidfile=env00.pid --outfile=env00.out --envname=ENV_00_TEST
Pause before pre-dump 0. Press Enter to continue.
Run criu pre-dump
Pause before pre-dump 1. Press Enter to continue.
Run criu pre-dump
Pause before dump. Press Enter to continue.
Run criu dump
Pause before restore. Press Enter to continue.
Run criu restore
Pause after restore. Press Enter to continue.

v2: improve sbs step naming; rename "iter" to more meaningfull
"pre-dump"/"snap".

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2021-09-03 10:31:00 -07:00
Liu Hua
07316d15a6 restore: cleanup cgroup properly in error path
Cgroup yard is setup in crtools_prepare_shared. But it is not cleaned up
properly in some code path, which leads cgroup mountpoint leaking.

Signed-off-by: Liu Hua <weldonliu@tencent.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
8f2b8c7be0 scripts: run lint also on criu-ns
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
bd648cc8d9 ci: also test tcp stream crit recoding
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
fa9acb9dc7 lib: fix broken crit-recode test
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
0ca36c95ee ci: combine cross compile container definitions
The cross compile container definitions for each architecture were
almost the same files except for the architecture.

This moves the architecture to variables so that all cross compile
setups can use the same container definition.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Pavel Tikhomirov
2ebb1c7419 crit: fix error on memfd files parsing
Else we get error:

[root@fedora criu]# crit/crit x test/dump/zdtm/static/memfd00/56/1/ mems
...
Traceback (most recent call last):
  File "/home/snorch/devel/ms/criu/crit/crit", line 6, in <module>
    cli.main()
  File "/home/snorch/devel/ms/criu/crit/pycriu/cli.py", line 430, in main
    opts["func"](opts)
  File "/home/snorch/devel/ms/criu/crit/pycriu/cli.py", line 361, in explore
    explorers[opts['what']](opts)
  File "/home/snorch/devel/ms/criu/crit/pycriu/cli.py", line 283, in explore_mems
    fn = ' ' + get_file_str(opts, {
  File "/home/snorch/devel/ms/criu/crit/pycriu/cli.py", line 214, in get_file_str
    f = ft['get'](opts, ft, fd['id'])
  File "/home/snorch/devel/ms/criu/crit/pycriu/cli.py", line 165, in ftype_reg
    rf = ftype_find_in_image(opts, ft, fid, 'reg-files.img')
  File "/home/snorch/devel/ms/criu/crit/pycriu/cli.py", line 154, in ftype_find_in_image
    return f[ft['field']]
KeyError: 'reg'

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2021-09-03 10:31:00 -07:00
Zeyad Yasser
f57e45df56 cr-service: move pidfd_store initialization to cr-service
Moved pidfd store hashtable initialization to cr-service.c since
pidfd_store is only relevant in RPC mode.

Signed-off-by: Zeyad Yasser <zeyady98@gmail.com>
2021-09-03 10:31:00 -07:00
Zeyad Yasser
f7cd254004 pidfd_store: tidy up interface and hide unneeded details
There is no need to expose the internals of pidfd_store like
hash table entries and how to use them for pid reuse detection.

v2: fixup some spacing issues
v3: fix hash memory leak after xmalloc in init_pidfd_store_hash()

Signed-off-by: Zeyad Yasser <zeyady98@gmail.com>
2021-09-03 10:31:00 -07:00
Zeyad Yasser
083f0822ed pidfd_store: move pidfd_store to a separate file
Signed-off-by: Zeyad Yasser <zeyady98@gmail.com>
2021-09-03 10:31:00 -07:00
Zeyad Yasser
d55f34ed78 test/ci: sync netns_lock test and its --post-start hook
The --post-start hook creates a netns which the test should enter
at the beginning of the test.

The test randomly failed in CI tests, it is most likely caused by
a race condition.

I suspect this flow is root cause:
	1. --post-start hook starts just after the test (in parallel)
	2. --post-start hook calls ip netns add to create the test netns
	3. ip creates the netns file
	4. netns_lock test opens that file and uses it in setns
	5. ip mounts the netns to the file

Of course test fails at step 4 because the netns is not yet mounted
to the file.

I made the test wait for SYNCFILE to be created by the --post-start
hook before it tries to open the netns file and call setns.

Signed-off-by: Zeyad Yasser <zeyady98@gmail.com>
2021-09-03 10:31:00 -07:00
Zeyad Yasser
b290df9a65 test/jenkins: fix netns_lock test multiple iterations failure
netns_lock is highly dependent on the order of the hooks, and
iterations causes the --pre-dump hook to be called multiple
times which expectedly caused the test to fail.

The server loop accommodates for multiple iterations.

https://ci.kernoops.org/job/CRIU/job/CRIU-iter/job/criu-dev/431/testReport/(root)/criu/zdtm_static_netns_lock/

Signed-off-by: Zeyad Yasser <zeyady98@gmail.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
75feb9635e ci: fix mips64el-cross test
The mips64el-cross test target started to show following error:

error: listing the stack pointer register '$29' in a clobber list is deprecated [-Werror=deprecated]

This fixes it in three different places by removing $29' from the
clobber list. This is only compile tested as we have no mips hardware
for testing.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Younes Manton
f3cb156604 Keep inherit-fd strings alive until task restore
If inherit-fd is read from a config file its buffer will be freed
after the config file is parsed but before task restore, which is
when we need to use the mapping. Therefore, when adding an
inherit-fd mapping to the opts list, copy the key string to a new
buffer.

Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2021-09-03 10:31:00 -07:00
fu.lin
d3ce492cc2 pycrit: fix the broken of cli the crit show xxx.img
It will broken when the cli `crit show ipcns-shm-9.img` is executed, msg:
  {
      "magic": "IPCNS_SHM",
      "entries": [
          {
              "desc": {
                  "key": 0,
                  "uid": 0,
                  "gid": 0,
                  "cuid": 0,
                  "cgid": 0,
                  "mode": 438,
                  "id": 0
              },
              "size": 1048576,
              "in_pagemaps": true,
              "extra": Traceback (most recent call last):
    File "/usr/bin/crit", line 6, in <module>
      cli.main()
    File "/usr/lib/python3/dist-packages/pycriu/cli.py", line 412, in main
      opts["func"](opts)
    File "/usr/lib/python3/dist-packages/pycriu/cli.py", line 45, in decode
      json.dump(img, f, indent=indent)
    File "/usr/lib/python3.9/json/__init__.py", line 179, in dump
      for chunk in iterable:
    File "/usr/lib/python3.9/json/encoder.py", line 431, in _iterencode
      yield from _iterencode_dict(o, _current_indent_level)
    File "/usr/lib/python3.9/json/encoder.py", line 405, in _iterencode_dict
      yield from chunks
    File "/usr/lib/python3.9/json/encoder.py", line 325, in _iterencode_list
      yield from chunks
    File "/usr/lib/python3.9/json/encoder.py", line 405, in _iterencode_dict
      yield from chunks
    File "/usr/lib/python3.9/json/encoder.py", line 438, in _iterencode
      o = _default(o)
    File "/usr/lib/python3.9/json/encoder.py", line 179, in default
      raise TypeError(f'Object of type {o.__class__.__name__} '
  TypeError: Object of type bytes is not JSON serializable

This is caused by `img['magic'][0]['extra']` which is bytes. I find
other load condtions, fix them at the same time.

Signed-off-by: fu.lin <fulin10@huawei.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
093fb0c878 Add test for new --lsm-mount-context option
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
64dd64e504 Enable changing of mount context on restore
This change is motivated by checkpointing and restoring container in
Pods.

When restoring a container into a new Pod the SELinux label of the
existing Pod needs to be used and not the SELinux label saved during
checkpointing.

The option --lsm-profile already enables changing of process SELinux
labels on restore. If there are, however, tmpfs checkpointed they
will be mounted during restore with the same context as during
checkpointing. This can look like the following example:

 context="system_u:object_r:container_file_t:s0:c82,c137"

On restore we want to change this context to match the mount label of
the Pod this container is restored into. Changing of the mount label
is now possible with the new option --mount-context:

 criu restore --mount-context "system_u:object_r:container_file_t:s0:c204,c495"

This will lead to mount options being changed to

 context="system_u:object_r:container_file_t:s0:c204,c495"

Now the restored container can access all the files in the container
again.

This has been tested in combination with runc and CRI-O.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
5be71273f6 Remove unnecessary whitespace
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Zeyad Yasser
fc7705a13f zdtm: add network namespace locking test
When criu dumps a process in a network namespace it locks
the network so that no packets from peer enters the stack,
otherwise RST will be sent by a kernel causing the connection
to fail.

In netns_lock.c we try to enter the netns created by post-start
hook so that criu locks the network namespace between dump and
restore.

A TCP server is started in post-start hook inside the test netns
and runs in the background detached from its parent so that
it stays alive for the duration of the test.

Other hooks (pre-dump, pre-restore, post-restore) try to
connect to the server.

Pre-dump and post-restore hooks should be able to connect
successfully.

Pre-restore hook client with SOCCR_MARK should also connect
successfully.

Pre-restore hook client without SOCCR_MARK should not be able
to connect but also should not get connection refused as all
packets are dropped in the namespace so the kernel shouldn't
send an RST packet as a result. Instead we check that the
connect operation causes a timeout.

This test would be useful when testing that the network is
locked using different ways (using iptables currently and
other methods later).

v2:
	- check that packets with SOCCR_MARK are allowed to
	  pass when the netns is locked.

v3:
	- fix pre-restore hook skipping non SOCCR_MARK
	  connection test due to early exit in SOCCR_MARK
	  variant.

Signed-off-by: Zeyad Yasser <zeyady98@gmail.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
0cf79a3608 test: remove exec test
criu exec is deprecated for some time now and criu just exits with an
error if running 'criu exec'. This removes the test for that non-working
subcommand.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
1a197d4d86 criu: add unit testing for config file parser
This tries to add a unit test for the configuration file parser.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
45bde968a2 test: add tests for configuration file parsing
This adds a test run to ensure known (but fixed) configuration file
parser errors are not crashing CRIU anymore.

Based on missing test code coverage this script also tests code paths of
the option handling which have not been tested until now.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
f695e6e107 config: make configuration file parser more robust
Trying to see how robust the configuration parser I was able to crash
CRIU pretty quickly. This fixes a few crashes in the existing
configuration file parser.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
381d2e88fb criu: add cleanup_free attribute
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Nicolas Viennot
031a8d7905 bfd: loop through read()/write() when the action is incomplete
The callers of bread() and bwrite() assume the operation reads/writes
the complete length of the passed buffer.
We must loop when invoking the read()/write() APIs.

Fixes #1504

Signed-off-by: Nicolas Viennot <Nicolas.Viennot@twosigma.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
24bc083653 ci: disable some tests on CentOS 7
Now that we are running CI on an actual CentOS 7 kernel different
tests are no longer working as they require newer kernels.

This commit disables a few tests only on CentOS 7.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
63ca464bcb ci: remove old workarounds
This commit removes a couple of workaround for old kernels and
distributions which we no longer use in CI.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
6ef01d3e6a ci: switch CentOS 7 test to Cirrus CI
On Cirrus CI we can run tests on the orignal CentOS 7 kernel.

The kernel is rather old, but on GitHub Actions we a 5.8 kernel
and a containerized CentOS 7 user space not much is working
correctly anymore. With this commit CentOS 7 based tests are
no longer running on GitHub Actions but on Cirrus CI.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
1fbe876242 ci: disable -x during print_env()
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Adrian Reber
b4c7267b0e zdtm: allow ignore taint via environment variable
With this change tainted kernels can be ignored with setting
ZDTM_IGNORE_TAINT=1. This is just to simplify the CI script to not
require to change every call of zdtm. Setting the variable once should
be enough.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Radostin Stoyanov
a92833818b scripts/vagrant: Use vagrant 2.2.16
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-09-03 10:31:00 -07:00
Radostin Stoyanov
eda3ac2ff3 scripts/vagrant: Use Fedora 34
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-09-03 10:31:00 -07:00
Mike Frysinger
87ea13f6b7 add PKG_CONFIG default in a few more places
These files use $PKG_CONFIG before they include the common files that
setup a default, so set early defaults in them too.

Signed-off-by: Mike Frysinger <vapier@chromium.org>
2021-09-03 10:31:00 -07:00
Valery Ivanov
6db0f95dbf crtools: improve error handling on signal setting
Signed-off-by: Valery Ivanov <ivalery111@gmail.com>
2021-09-03 10:31:00 -07:00
Mike Frysinger
2967bed64e build: respect $PKG_CONFIG settings
The build needs to respect $PKG_CONFIG env var like other standard
build systems and the the upstream pkg-config project itself.  This
allows the package builder to point it to the right tool when doing
a cross-compile build.  Otherwise the host pkg-config tool is used
which won't have access to the packages in the cross sysroot.

Signed-off-by: Mike Frysinger <vapier@chromium.org>
2021-09-03 10:31:00 -07:00
Radostin Stoyanov
81a68ad3b2 docker-test: use latest containerd release
This patch improves the changes from 19be9ced9.

To use the newer version of containerd, we need to make sure that the
containerd service has been restarted after install. Instead of
hard-coding a version number, we can use github API to get the latest
release. In addition, the tar file contains all binary files in a
'./bin' sub-folder. Thus, it should be extracted in '/usr'.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-09-03 10:31:00 -07:00
Radostin Stoyanov
638e53c950 zdtm/tun_ns: add per-test dependencies
The tun_ns test was introduced with [1] and [2], however, these commits
didn't add per-test dependencies required for the test.

Per-test dependencies are listed in the .desc file as 'deps': [<list>]

These dependencies are made available inside the test namespace and without
the ip dependency, the tests fails on Fedora 34 with

   Error: ipv4: FIB table does not exist.

[1] https://github.com/checkpoint-restore/criu/commit/7e355e7
[2] https://github.com/checkpoint-restore/criu/commit/3ba0893

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-09-03 10:31:00 -07:00
Adrian Reber
9d9ec73dd7 test: skip time namespaced tests on <= 5
Although CentOS 8 comes with 4.18 kernel it has time namespace patches
backported but not all the required once. This disables time namespaced
tests on everything older than 5.11.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-09-03 10:31:00 -07:00
Radostin Stoyanov
e42083aa8b ci: update docker test matrix
Ubuntu 18.04 still has a problem with overlayfs and breaks CRIU
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1857257

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-09-03 10:31:00 -07:00
Christian Brauner
ebc74668ff cr_options: handle the case where __dest == __src in SET_CHAR_OPTS
The SET_CHAR_OPT(__dest, __src) macro is essentially:

free(opts.__dest);
opts.__dest = xstrdup(__src);

So if __dest == __src the string that get's copied is freed. This e.g.
is the case in criu/lsm.c

int lsm_check_opts(void)
{
	char *aux;

	if (!opts.lsm_supplied)
		return 0;

	aux = strchr(opts.lsm_profile, ':');
	if (aux == NULL) {
		pr_err("invalid argument %s for --lsm-profile\n", opts.lsm_profile);
		return -1;
	}

	*aux = '\0';
	aux++;

	if (strcmp(opts.lsm_profile, "apparmor") == 0) {
		if (kdat.lsm != LSMTYPE__APPARMOR) {
			pr_err("apparmor LSM specified but apparmor not supported by kernel\n");
			return -1;
		}

		SET_CHAR_OPTS(lsm_profile, aux);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

	} else if (strcmp(opts.lsm_profile, "selinux") == 0) {
		if (kdat.lsm != LSMTYPE__SELINUX) {
			pr_err("selinux LSM specified but selinux not supported by kernel\n");
			return -1;
		}

		SET_CHAR_OPTS(lsm_profile, aux);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

	} else if (strcmp(opts.lsm_profile, "none") == 0) {
		xfree(opts.lsm_profile);
		opts.lsm_profile = NULL;
	} else {
		pr_err("unknown lsm %s\n", opts.lsm_profile);
		return -1;
	}

	return 0;
}

Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
2021-09-03 10:31:00 -07:00