2
0
mirror of https://github.com/checkpoint-restore/criu synced 2025-08-29 13:28:27 +00:00

11541 Commits

Author SHA1 Message Date
Michał Mirosław
5a723937a2 compel: Log the status word with "Task is still running" errors.
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
664598dc74 files-reg: Debug "open file on overmounted mount" error.
Log the mount and file that were the cause of failing a dump.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
a5fe99d2c5 cgroup: Propagate error on cgroup mount failure.
This makes the error to mount cgroup hierarchy a bit less noisy:

Error (criu/cgroup.c:623): cg: Unable to mount cgroup2 : Invalid argument'

Instead of

Error (criu/cgroup.c:623): cg: Unable to mount cgroup2 : Invalid argument'
Error (criu/cgroup.c:715): cg: failed walking /proc/self/fd/-1/zdtmtst for empty cgroups: No such file or directory'

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
056cac474e zdtm: drop python 2 compatibility
This patch removes the code for Python 2 compatibility introduced
with commit e65c7b5 (zdtm: Replace imp module with importlib).

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
002f2372a9 lib/py: drop python 2 compatibility
This patch removes code introduced for compatibility with
Python 2 in commits:

  bf80fee (lib: correctly handle stdin/stdout (Python 3))

  b82f222 (lib: fix crit-recode fix for Python 2)

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
d388b91de2 test/others: drop setup_swrk() py2 compatibility
This patch removes the code introduced for compatibility
with Python 2 in commits:

4c1ee3e227045fc1dc07b10ac7a538a68299693b
test/other: Resolve Py3 compatibility issues

6b615ca15277fc14b52a09b4eb18314b7c6cbe75
test/others: Reuse setup_swrk()

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
ee9983d4a9 test/criu-ns: drop python 2 compatibility
This patch is replacing the set_blocking() function with
os.set_blocking(). This function was introduced for compatibility with
Python 2 in commit 8094df8di (criu-ns: Add tests for criu-ns script).

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
ede018176c make: remove checks for python 2 binary
This commit removes the checks for the Python 2 binary in the makefile
and makes sure that ZDTM tests always use python3. Since support for
Python 2 has been dropped, these checks are no longer needed.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
642fd99bfd remove python-future dependency
This commit removes the dependency on the __future__ module, which was
used to enable Python 3 features in Python 2 code. With support for
Python 2 being dropped, it is no longer necessary to maintain backward
compatibility.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
376f3d1800 crit: add requirements.txt for pip>=20.1
When building with pip version 20.0.2 or older, the pip install
command creates a temporary directory and copies all files from
./crit. This results in the following error message:

    ModuleNotFoundError: No module named 'pycriu'

This error appears because the symlink 'pycriu' uses a relative path
that becomes invalid '../lib/py/'.

The '--no-build-isolation' option for pip install is needed to enable
the use of pre-installed dependencies (e.g., protobuf) during build.

The '--ignore-installed' option for pip is needed to avoid an error when
crit is already installed. For example, crit is installed in the GitHub
CI environment as part of the criu OBS package as a dependency for
podman.

Distributions such as Arch Linux have adopted an externally managed
python installation in compliance with PEP 668 [1] that prevents pip
from breaking the system by either installing packages to the system or
locally in the home folder. The '--break-system-packages' [2] option
allows pip to modify an externally managed Python installation.

[1] https://peps.python.org/pep-0668/
[2] https://pip.pypa.io/en/stable/cli/pip_uninstall/

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
f5d06571c5 crit: drop python 2 support
This patch reverts changes introduced with the following commits:

4feb07020dedbf845fc00268d8ca02f4645641cd
crit: enable python2 or python3 based crit

b78c4e071a42ebe34aac82fa0711df07ed375e2b
test: fix crit test and extend it

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
7825f4ebfa coredump: drop python 2 support
This patch reverts changes introduced for Python 2 compatibility
in commits:

  1c866db (Add new files for running criu-coredump via python 2 or 3)

  3180d35 (Add support for python3 in criu-coredump).

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
05f0535de4 ci: clean up CentOS 7 related tweaks
We have disabled CentOS 7 tests in CI. This patch reverts the
changes introduced in the following commit:

24bc083653f7d2b984653194e921b1ff32292b3b
ci: disable some tests on CentOS 7

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
362d8fa5c2 ci: disable CentOS 7 test in Cirrus CI
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Abhishek Guleri
94c9e47872 readme: refactor asciinema link for video playback
Instead of opening the image directly, the commit refactors the
asciinema image embedded link to redirect users to the corresponding
video.

Signed-off-by: Abhishek Guleri <abhishekguleri24@gmail.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
7bda5e656c zdtm: Update netns purpose comment in zdtm_ct.
With the parasite socket clash now guaranteed not to happen,
the comment becomes obsolete. netns is steel needed though, so
update the comment to point at the requirement.

Change-Id: I3cfb253cd5c53b91b955fcb001530b4aee5129f4
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
9ad59f58ff util: Make CRIU run_id machine-level unique.
Instead of relying on chance of CLOCK_MONOTONIC reading being unique,
use pid namespace ID that combined with the process ID will make it
unique on the machine level.

If pidns is not enabled on a kernel we'll get ENOENT, but then CRIU's
pid will already be unique. If there is some other error, log it but
continue, as the socket clash (if it happens) will result in a failed
run anyway.

Fixes: 45e048d77a6a (2022-03-31 "criu: generate unique socket names")
Fixes: 408a7d82d644 (2022-02-12 "util: add an unique ID of the current criu run")
Change-Id: I111c006e1b5b1db8932232684c976a84f4256e49
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
eecc53d05a kerndat: Don't fail on NETLINK/nsid support missing.
If not dumping netns nor connections, nsid support is not used. Don't
fail the run as if the support is needed, the dumping process will fail
later.

Change-Id: I39a086756f6d520c73bb6b21eaf6d9fb49a18879
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
bf8446ae5f kerndat: unexport kerndat_nsid()
kerndat_nsid() is not used outside kerndat.c. Make it static.

Change-Id: I52e518ecb7c627cc1866e373411b2be3f71a2c9d
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
f2011e1c76 util: Downgrade ignored errors to warnings.
If the error is ignored it is not important enough - make it a warning
instead.

From: Mian Luo <mianl@google.com>
Change-Id: If2641c3d4e0a4d57fdf04e4570c49be55f526535
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
2aa9cb9333 rpc: Support setting images_dir by path.
Google's RPC client process is in a different pidns and has more privileges --
CRIU can't open its /proc/<pid>/fd/<fd>.  For images_dir_fd to be useful here
it would need to refer to a passed or CRIU's fd.

From: Michał Cłapiński <mclapinski@google.com>
Change-Id: Icbfb5af6844b21939a15f6fbb5b02264c12341b1
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
439b522433 rpc: Support gathering external file list after freezing process tree.
New 'query-ext-files' action for `criu dump` is sent after
freezing the process tree. This allows to defer gathering
the external file list when the process tree is in a stable
state and avoids race with the process creating and deleting
files.

Change-Id: Iae32149dc3992dea086f513ada52cf6863beaa1f
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
82a0db036e docker/podman: test c/r with action-script
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
28d0056388 action-scripts: allow shell scripts in rpc mode
Container runtimes commonly use CRIU with RPC. However, this prevents
the use of action-scripts set in a CRIU configuration file due to the
explicit scripts mode introduced with the following commit:

  ac78f13bdfaee260dd4234f054bf4c5d2a373783
  actions: Introduce explicit scripts mode

This patch enables container checkpoint/restore with action-scripts
specified via configuration file.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Michał Mirosław
3e0a8ffd6d build: libnfnetlink: Remove nla_get_s32().
nla_get_s32() was added to libnl 3.2.7 in 2015. Remove CRIU's definition
as it breaks build when statically linking the binary.

From: Uros Prestor <urosp@google.com>
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
f043cb22af build: Guard against libbsd's version of __aligned.
When trying to build CRIU with libbsd enabled the compilation fails due
to duplicate definition of __aligned macro. Other such definitions are
already wrapped with #ifndef make __aligned definition consistent and
make it easier in the future to use the libbsd features if needed.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
c5142104a2 build: Use make-provided AR for building libzdtmtst.
Make $(AR) used also for libzdtmtst build.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
88f8fdda82 build: Fix LIBS vs LDFLAGS order.
$LDFLAGS can contain `-Ldir`s that are required by '-lib's in $LIBS.
Reverse the order so that `-L` options make effect.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
cbbe6c683d build: Debug system feature tests.
`make` without `-s` option will normally show the commands executed. In
the case of detecting build environment features current makefile will
cause detected features to be seen as 'echo #define' commands, but not
detected ones will be silent. Change it so that all tried features can
be seen (outside of make's silent mode) regardless of detection result.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
1a1fa439c7 build: Remove HAS_MEMFD test
The test for HAS_MEMFD is empty and noit used. Remove it.

Fixes: 5ee1ac1f28e6 ("criu: remove FEATURE_TEST_MEMFD")
Change-Id: I43b8f0cfd50ce9bdf93dafb647377318df1deae8
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
2bf10c8d27 restore: remove unused secbits field.
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
e5c9bc4d08 kerndat: Make socket feature probing work on IPv6-only host.
Try IPv6 if IPv4 sockets are not supported.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
eea0d6edee pipes: Plug pipe fd leak in "Unable to set a pipe size" error case.
From: Piotr Figiel <figiel@google.com>
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
ffa1e47fd8 sockets: Increase the size of sockets hashmap to 16K.
During dump, CRIU stores the structs representing sockets in a statically sized
hashmap of size 32. We have some (admittedly crazy) tasks that use tens of
thousands of sockets, and seem to spend most of the dump time iterating over
the linked lists of the map.

16K is chosen arbitrarily, so that it reduces the lengths of the chains to few
elements on average, while not introducing significant memory overhead.

From: Radosław Burny <rburny@google.com>
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
304a309aed test/thp_disable: fix lint
The fail() macro provides a new line character at the end of the
message. This patch fixes the following lint check that currently
fails in CI:

$ git --no-pager grep -E '^\s*\<(pr_perror|fail)\>.*\\n"'
test/zdtm/static/thp_disable.c:		fail("prctl(GET_THP_DISABLE) returned unexpected value: %d != 1\n", ret);
test/zdtm/static/thp_disable.c:		fail("Flags changed %lx -> %lx\n", orig_flags, new_flags);
test/zdtm/static/thp_disable.c:		fail("Madvs changed %lx -> %lx\n", orig_madv, new_madv);
test/zdtm/static/thp_disable.c:		fail("post-migration prctl(GET_THP_DISABLE) returned unexpected value: %d != 1\n", ret);
test/zdtm/static/thp_disable.c:		fail("Flags changed %lx -> %lx\n", orig_flags, new_flags);
test/zdtm/static/thp_disable.c:		fail("Madvs changed %lx -> %lx\n", orig_madv, new_madv);

Fixes: #2193

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Michał Mirosław
c97cc6a6ce Allow skipping iptables/nftables invocation.
Make it possible to skip network lock to enable uses that break connections
anyway to work without iptables/nftables being present.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
d0ac547b3d zdtm: sock_opts00: Improve error messages.
Make it clear that the option numbers are indexes not the option
identifiers ("names"). Also show the value change that prompted test
failure.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
6bc00fcb84 zdtm: Implement test sharding.
Allow to split test suite into predictable sets to parallelize runs on
multiple machines or VMs.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
1fb5c410c8 zdtm: Allow --keep-going for single test.
We don't want test framework to change its behaviour on whether we
run a single or multiple tests in a run. When we shard the test suite
it can result in some shards having a single test to run and
unexpectedly change the test output format.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
ed88e3241c zdtm: Add timeouts for test commands.
Extend ability to limit time taken to all CRIU invocations.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
516fade932 zdtm: Allow overriding /tmp.
Use $TMPDIR for tests_root as the host's /tmp might not have enough
features or space.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Cłapiński
4b764a9dce Allow passing --log_to_stderr via RPC.
Signed-off-by: Michał Cłapiński <mclapinski@google.com>
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Cłapiński
1e5ebec39d Allow passing --display_stats via RPC.
Signed-off-by: Michał Cłapiński <mclapinski@google.com>
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
0e88a91ff0 Allow passing --leave_stopped by RPC.
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Haorong Lu
4455444eeb compel/test: Return 0 in case of error in fdspy
This commit revises the error handling in the fdspy test. Previously,
a failure case could have been incorrectly reported as successful because
of a specific check `pass != 0`, leading to potential false positives
when `check_pipe_ends()` returned `-1` due to a read/write pipe error.

To improve this, we've adjusted the error handling to return `0` in case
of any error. As such, the final success condition remains unchanged. This
approach will help accurately differentiate between successful and failed
cases, ensuring the output "All OK" is printed for success, and "Something
went WRONG" for any failure.

Fixes: 5364ca3 ("compel/test: Fix warn_unused_result")

Signed-off-by: Haorong Lu <ancientmodern4@gmail.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
4c1409b8f6 Fill FPU init state if it's not provided by kernel.
Apparently Skylake uses init-optimization when saving FPU state, and ptrace()
returns XSTATE_BV[0] = 0 meaning FPU was not used by a task (in init state).
Since CRIU restore uses sigreturn to restore registers, FPU state is always
restored. Fill the state with default values on dump to make restore happy.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
93ad8d40de zdtm: thp_disable: Verify MADV_NOHUGEPAGE before migration
Add a sanity check for THP_DISABLE. This discovered a broken commit
in Google's kernel tree.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
01238d2706 zdtm: thp_disable: Verify prctl(THP_DISABLE) migration
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
a0c78a7902 zdtm: thp_disable: Output a single failure message
While at it, don't carry over stale errno to the fail() message.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
2364c963c6 Log if prctl(SET_THP_DISABLE) doesn't work as expected.
If prctl(SET_THP_DISABLE) is not used due to bad semantics, log it
for easier debugging.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00