Makefile knows that a binary depends on .o file,
and the .o file depends on .c file, and adds the
per-target variables even in the indirect case.
Therefore, .o suffix is not needed here.
Note that it not hurts to leave it as is, so this commit
is done only for the sake of keeping things simple.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
This test name was added by commit 1ce1cca, but there is no test.
We do not use these lists since zdtm.py was introduced, therefore
the error went unnoticed. I found it while running "make dep".
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
This commit expands the success with lib/Makefile to the rest of
ztdm/Makefiles. In particular, it moves the common part to Makefile.inc
and let {lib,static,transition}/Makefile use it.
This results in:
- dependencies being handled properly;
- reduction of makefiles length due to unification;
- possibly more proper dependencies and builds.
While at it, let's also:
- make rules for criu-rtc.so silent;
- set default CC=gcc;
- remove duplicate -m32 from CFLAGS.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
In GNU make, 'override' means change a variable even if it was set
from the make command line (as in "make VAR=value"). As we do not
intend to modify CFLAGS/LDFLAGS from the command line, 'override'
can be dropped.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
When optimizing our Travis runs, I noticed that test/zdtm/lib is built 3
times in a row -- first time the binaries, 2nd and 3rd time the deps,
in parallel (sic!).
I took a look at Makefile and found this, and some more problems,
like:
- dependencies are generated but not included
- cleandep does not depend on clean
- groups.c dependencies are not generated (or used)
- "make all" does not make groups binary
- overcomplicated and unused groups.{pid,out,cleanout} targets
- extra -f flag to rm
- etc.
This patch (hopefully) fixes all of the above. In addition, it provides
a few enhancements:
- enables "silent make" rules, so now lib/ is in line with the rest of zdtm
- adds "dep" target (not used, just for convenience)
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Commit 299e4b4 ("travis: enable ccache for docker/qemu builds") combined
multiple RUN statements in Dockerfiles into a single one, which is good
for performance (as there is an intermediate cache created after each
statement). Unfortunately, it partially did so by combining statements
with use of semicolons, meaning we are ignoring non-zero exit codes.
As a result, failure from make are ignored and Travis builds are all
green.
To fix, replace all occurences of semicolon with &&.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Since commit 93db22b ("compel: no -r for ARM ldflags") and
commit ce043d3 ("compel: make plugins .a archives") the generated
parasite/restorer blobs for 32-bit ARM are no longer relocatable,
and so their ELF header has e_type of ET_EXEC (rather than ET_REL).
As a result, we have "Unsupported header detected" message and
a failure from compel hgen.
This commit allows e_type == ET_EXEC in case NO_RELOCS is defined.
I am not sure if it's OK to still allow ET_REL -- maybe Dmitry
can chime in on that.
Cc: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Commit d9486bd720 ("arm/pie/build: do not produce relocatable parasite
object") removed -r from LDFLAGS used to compile criu pie. This
functionality somehow never made it to criu-dev, and was also lost
in master then compel was ported to it.
Make it work with compel.
Unfortunately it was not as simple as I initially thought, as -r flag
to ld was built into nmk. This patch removes it, and adds it to all
places that need intermediate linking.
Cc: Dmitry Safonov <dsafonov@virtuozzo.com>
Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
The objective is to only do parasite code linking once -- when we link
parasite objects with compel plugin(s). So, let's use ar (rather than
ld) here. This way we'll have a single ld invocation with the proper
flags (from compel ldflags) etc.
There are two tricks in doing it:
1. The order of objects while linking is important. Therefore, compel
plugins should be the last to add to ld command line.
2. Somehow ld doesn't want to include parasite-head.o in the output
(probably because no one else references it), so we have to force
it in with the modification to our linker scripts.
NB: compel makefiles are still a big mess, but I'll get there.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
When performing static linking, the order of objects is important
(when the linker finds an undefined reference, it looks forward,
never back). Therefore, sorting objects breaks things.
Required for the next patch.
Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
1. Remove .FORCE, it's not used.
2. Consolidate CFLAGS stripping into a single line.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
The way criu/pie/Makefile is currently written, ld is run twice:
1. link $(NAME-obj-y) objects to NAME.built-in.o
2. link NAME.built-in.o, pie.lib.a, and compel plugins
to NAME.built-in.bin.o (with compel ldflags and linker script)
There is absolutely no need for such two-stage linking, but it was OK.
It is not OK now, as "compel ldflags" for ARM doesn't need -r, and we
can't run the first stage with -r and the second stage without it.
So, let's simplify linking using a single ld invocation. This is my
third attempt in doing it, I think I nailed it this time -- it is now
clean and (relatively) simple.
While at it:
- fix compel linker script dependency (it was not working);
- rearrange the Makefile so variables goes first, then rules;
- remove a comment about mount implementation in restorer.
NOTE that compel is called with ./ prefix so the file paths it prints
are also prefixed with ./, which is needed for objectify macro to ignore
those.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
1. Remove useless LDFLAGS -- dynamic linker is not used here.
2. Remove $(iquotes) -- there are none defined here.
3. Consolidate filter-out statements into a single one.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
This is an addition to commit 3a7e027 ("criu: pie -- Switch to use
compel shipped lds scripts").
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
There are too many directories in the include path.
Let's remove this one, and refer to the blobs with pie/ prefix.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
This is a missing hunk from commit 85b04c8 "Makefiles: nuke $(SRC_DIR)"
which has somehow escaped from my attention.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
It can be passed as link to real object.
Signed-off-by: Stanislav Kinsburskiy <skinsbursky@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Usually we print the PID we wait for, let's do the same here.
Probably just a typo.
Cc: Kirill Tkhai <ktkhai@virtuozzo.com>
Reviewed-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
I guess this is a followup to commit 59e80d4.
1 s/determinate/figure out/ (there is no "determinate" verb in English)
2 moved 'with' to the end
Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
These helpers are valuable and can be used outside.
Signed-off-by: Stanislav Kinsburskiy <skinsbursky@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Before dump:
438 437 0:51 /mtest /zdtm/static/mnt_ext_manual.test rw,relatime - tmpfs zdtm_auto_ext_mnt rw
439 437 0:51 /mtest /zdtm/static/mnt_ext_manual_private_shared_bind.test rw,relatime shared:144 - tmpfs zdtm_auto_ext_mnt rw
440 437 0:51 /mtest /zdtm/static/mnt_ext_manual_bind.test rw,relatime shared:144 - tmpfs zdtm_auto_ext_mnt rw
441 437 0:51 /mtest /zdtm/static/mnt_ext_manual_slave_shared_bind.test rw,relatime shared:145 master:144 - tmpfs zdtm_auto_ext_mnt rw
442 437 0:51 /mtest /zdtm/static/mnt_ext_manual_slave_bind.test rw,relatime master:145 - tmpfs zdtm_auto_ext_mnt rw
>From log: mount mnt_ext_manual_slave_bind is restored as slave of
mnt_ext_manual_slave_shared_bind, which is restored as slave of
mnt_ext_manual_private_shared_bind, which is restored as shared of
mnt_ext_manual_bind, which is restored as private bind of external,
that is exactly the same way mnt_is_external() looks for external.
v2: fix test to cover all cases in mnt_is_external()
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
If container has external bindmount given to criu through
--ext-mount-map option by admin, container user can bindmount
subdirs of these external bindmount to somewhere else inside
container creating secondary external bindmounts. Criu we will
fail to restore them as having unreachable sharing. But we can
restore secondary external bindmounts bindmounting them from
primary external bindmount.
v2: s/external_bind/mnt_is_external/, make mnt_is_external bool,
do mnt_is_external without recursion
v3: add debug message on propagate_mount when bind is set
v5: remove hunk of v4 which goes to previous patch, so same as v3
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
need it to check if we can bindmount from external mount
note: when migrating from criu with patch to criu without,
external mount mapping won't work, we do not support it.
v2: s/real_root/ext_real_root/
v4: add comment
v5: use ext_key field for mapping, put NO_ROOT_MOUNT in root
for old externals for which we do not have it.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
"return" and "exit" are mixed in this function, and this is wrong.
Must be "return" only, because its callers don't want exit.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
mi->external is always false in these places
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
"make uninstall" is supposed to remove all the files that
"make install" (with the same arguments) have created.
This is a test to check that.
PS ideally, "make uninstall" should also remove any empty directories,
but let's not care about it for now.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
As we compile-test non-x86_64 architectures under qemu emulation,
it works pretty slow.
Dmitry Safonov suggested, and Andrey Vagin initially implemented
supporting ccache for such builds. This patch is based heavily
on Andrey's work -- all the bugs added are purely mine though.
Performance results: in an ideal environment (two builds of the same
code, one with cold (empty) ccache, another with the hot one)
I saw compile time improvements of 4x to 5x, and total test run time
improvement up to 2x to 2.5x. In layman terms, the complete test run
that was taking more than 50 minutes now takes about 25!
Notes on handling .ccache directory:
1. Before running docker build, .ccache directory (saved in between
runs by Travis) is moved to criu source code root, from where it
is copied by docker together with criu sources.
2. In Dockerfile, .ccache gets moved to /tmp, and CCACHE_DIR
is set accordingly.
3. After running docker build, .ccache is copied out from docker
container back to the host (from where it is saved by Travis).
Ccache envorinment notes:
1. CCACHE_NOCOMPRESS is needed because otherwise tons of time is spent
on compression/decompression (big performance hit under qemu).
2. CCACHE_CPP2 is required with clang, see detailed explanation at
http://petereisentraut.blogspot.com/2011/09/ccache-and-clang-part-2.html
The logic of setting CCACHE_CPP2 in Dockerfile is somewhat fancy;
unfortunately I was not able to come up with a simpler approach.
Misc:
1. Travis runs "ccache -s" after the build is completed. A call to
"ccache -s" is called to Dockerfile before make, so one can
compare before/after numbers.
2. make invocations are surrounded by "date" calls so one can get the
compilation times.
Changes in v2:
- consolidate Dockerfile statements (ENV, RUN)
- single object make test is no longer commented out
- simplify "make mrproper" test
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Reviewed-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
For compatibility with ccache (so it can cache the compilation results),
let's compile and link separately. For this, we have to
- disable the implicit make rules
- write the explicit ones
While at it, do use the "silent make" stuff that is already here
FIXME: figure out if it helps to speed up ccache build
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
For now, it's done for local builds only (i.e. no per-arch Docker
builds yet). The reason is, it's easier to play with ccache when
the compiles are (relatively) fast.
Performance: there is 2x to 3x improvement in build speeds for
ideal cases (same code, hot cache), but the absolute savings
are negligible (example: 7 seconds instead of 23).
Note that ccache is not compatible with gcov, so we only enable it
for non-gcov build (which happens to be the one with clang).
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Unify alpine and non-alpine builds. The only difference is foreign
arch builds need some preparation -- separate that to a dependency.
Unfortunately we can't use wildcard targets ("%: ") as non-wildcard
ones are prevaling. Therefore, a somewhat ugly hack to generate
$arch: Dockerfile.$arch dependency is added.
While at it:
- rename DB_ARGS to DB_CC
- mark clean as phony
- make "all" really run all the builds we can
Nice side effect: autocompletion ("make -C scripts/build <TAB>")
now works!
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
As pointed out by Andrey, arch clang tests are using gcc regardless
of CLANG=1 set in travis environment.
Frankly, I do not understand how it worked before (while being
pretty sure it worked!), but here is a way to fix it.
Reported-by: Andrey Vagin <avagin@virtuozzo.com>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
This is generated by clang-3.8:
> compel/arch/ppc64/src/lib/infect.c:31:20: error: unused function
> '__check_code_syscall' [-Werror,-Wunused-function]
> static inline void __check_code_syscall(void)
> ^
The fix is the same as in commit 3ea2fd7.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
If using the '--keep-going' option, zdtm prints out an overview how many
tests were run, failed and skipped. This would also be useful to know if
it did not fail. This patch changes the output like this:
################## ALL TEST(S) PASSED (TOTAL 297/SKIPPED 36) ###################
or in the case of a failure it is unchanged:
################### 2 TEST(S) FAILED (TOTAL 297/SKIPPED 34) ####################
* zdtm/static/sched_policy00(ns)
* zdtm/static/cgroup02(h)
##################################### FAIL #####################################
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
decode_flav() was doing 'if i in flavors:' where 'i' was an integer but
the keys from the flavors dict are strings 'h', 'ns' and 'uns'.
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
When infecting victim we construct sigframe to
be able to self-rectore it in case if something
goes wrong. But in case is a targer been using
alternative stack for signal handling it will
be missed in sigframe since we don't fetch it.
Thus add fetching sas on infection stage and
put it into signal frame early.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
We need to leave this part of restorer in restored task in case
of inserted redirected calls. Jump trampolines from old vdso lead
here - it would be painful to land nowhere.
Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Reworked this code a little and it becomes more readable.
Drop those macroses under CONFIG_X86_64 define and just use
boolean `compat_vdso' to check whether insert 64 or 32-bit jmp.
Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
It's the same on all archs (as it's called from generic code),
no need to redeclare it. <parasite-vdso.h> is included in all
per-arch headers. Drop vdso_symtable forward declaration as
<parasite-vdso.h> includes <util-vdso.h> header which defines
the structure.
Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
When checkpointing applications with really big
memory slab (like in our vz7 test with 920G of
memory) the int type get cutted, we should use
long int instead, just like we do in other code
pieces.
Otherwise get (on vz7's criu, which s sharing the code)
| pie: 756: Daemon waits for command
| (01.193097) Wait for ack 12 on daemon socket
| (01.193112) Fetched ack: 12 12 0
| (01.193164) 988065 fdinfo 0: pos: 0 flags: 100002/0
| (01.193201) fdinfo: type: 0xb flags: 0100002/0 pos: 0 fd: 0
| (01.193279) 988065 fdinfo 1: pos: 0 flags: 100002/0
| (01.193307) fdinfo: type: 0xb flags: 0100002/0 pos: 0 fd: 1
| (01.193341) 988065 fdinfo 2: pos: 0 flags: 100002/0
| (01.193365) fdinfo: type: 0xb flags: 0100002/0 pos: 0 fd: 2
| (01.193375) ----------------------------------------
| (01.193405) Error (criu/parasite-syscall.c:243): BUG at criu/parasite-syscall.c:243
| pie: 756: Error (criu/pie/parasite.c:676): Trimmed message received (1>
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Supporting shutdown for dgram sockets (udp and udplite)
is simple -- just fetch the state from diag module and
record it in the image, then upon socket creation restore
this state.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
To restore fsnotify's watches on files we need to find paths for each
of them using handle we have in /proc/<pid>/fdinfo/<fsnotifyfd>.
These handle is valid to open the file with open_by_handle_at if
you have mount fd where the file lays. So we try open_by_handle_at
for all possible mount fds we have.
But we can not do so for 'file' bind-mounts, as the way we open
mount fd opens file instead and can hang on fifos or fail on sockets.
(see check_open_handle->open_handle->open_mount code path, imagine
lookup_mnt_sdev() found 'file' bind-mount, open_mount() failed(hanged)
in __open_mountpoint() and if irmap_lookup() also was not successful the
whole dump fails too)
So if we have file bindmount of fifo file, and we restore some
inotify on other file on other mount with same s_dev, we hang forever
on open.
So just skip non-directory mounts from inotify search we will find
path for them on other mount(e.g. non-bindmount) with same s_dev.
v2: remove isdir hashing, improve commit message
v3: make lookup_mnt_sdev handle only nondir mounts, add comment,
move more expensive notdir_mountpoint check after s_dev
v4: inverse notdir_mountpoint to be mnt_is_dir, now on error
in mnt_is_dir mount is also skipped
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
External mounts are provided by an user and CRIU doesn't
need to restore their content.
https://github.com/xemul/criu/issues/299
Reported-by: Stéphane Graber
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Currently we collect mounts to clean up a mount namespace,
but it isn't required when we are going to call pivot_root.
https://github.com/docker/docker/issues/31663
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
