Currently CRIU traces syscalls to catch a moment, when sigreturn() is
called. Now we trace recv(cmd), close(logfd), close(cmdfd), sigreturn().
We can reduce a number of steps by using hw breakpoints. A breakpoint is
set before sigreturn, so we will need to trace only it.
Signed-off-by: Andrey Vagin <avagin@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
If a file like /proc/20/mountinfo is open, but 20 is a zombie (or doesn't exist
any more), we can't read this file at all, so a link remap won't work. Instead,
we add a new remap, called the dead process remap, which forks a TASK_HELPER as
that dead pid so that the restore task can open the new /proc/20/mountinfo
instead.
This commit also adds a new stage CR_STATE_RESTORE_SHARED. Since new
TASK_HELPERS are added when loading the shared resource images, we need to wait
to start forking tasks until after these resources are loaded.
v2: fix a mutex bug
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
In order to use TASK_HELPERS to open files from dead processes, they should
persist until criu is done restoring the filesystem, which happens in the
RESTORE stage. To do this, we need to pass each helper's PIDs to the restorer
blob, so that it can wait() on them when the restore stage is done.
This commit is in preparation for the remap_dead_pid commits.
v2: wait() on helpers after restore stage is over
v3: add CR_STATE_RESTORE_FS stage
v4: CR_STATE_RESTORE_FS waits for nr_tasks + nr_helpers, not nr_threads
v5: ditch CR_STATE_RESTORE_FS in favor of passing helpers to restorer blob
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
When clone-ing kids we can set their stack on current, as
it will anyway be COW-ed later. One thing to note -- we do
need to reserve some space on the stack for glibc's arguments
and retcode allocation. 128 bytes should be enough for 16
pointers while clone has 5 arguments.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
In cr_dump_tasks() we expect restore_root_task to return < 0 if
error ocures.
Signed-off-by: Ruslan Kuprieiev <kupruser@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
We have a slight mess with how criu restores root task.
Right now we have the following options.
1) CLI
a) Usually
task calling criu
`- criu
`- root restored task
b) when --restore-detached AND root has pdeath_sig
task calling criu
`- criu
`- root restored task
2) Library/SWRK
task using lib/swrk
`- criu
`- root restored task
3) Standalone service
a) Usually
service
`- service sub task
`- root restored task
b) when root has pdeath_sig
criu service
`- criu sub task
`- root restored task
It would be better is CRIU always restored the root task as sibling,
but we have 3 constraints:
First, the case 1.a is kept for zdtm to run tests in pid namespaces
on 3.11, which in turn doesn't allow CLONE_PARENT | CLONE_NEWPID.
Second, CLI w/o --restore-detach waits for the restored task to die and
this behavior can be "expected" already.
Third, in case of standalone service tasks shouldn't become service's
children.
And I have one "plan". The p.haul project while live migrating tasks
on destination node starts a service, which uses library/swrk mode. In
this case the restored processes become p.haul service's kids which is
also not great.
That said, here's the option called --restore-child that pairs the
--restore-detach like this:
* detached AND child:
task
`- criu restore (exits at the end)
`- root task
The root task will become task's child.
This will be default to library/swrk.
This is what LXC needs.
* detach AND !child
task
`- criu restore (exits at the end)
`- root task
The root task will get re-parented to init.
This will be compatible with 1.3.
This will be default to standalone service and
to my wish with the p.haul case.
* !detach AND child
task
`- criu restore (waits for root task to die)
`- root task
This should be deprecated, so that criu restore doesn't mess
with task <-> root task signalling.
* !detach AND !child
task
`- criu restore (waits for root task to die)
`- root task
This is how plain criu restore works now.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Tycho Andersen <tycho.andersen@canonical.com>
Acked-by: Andrew Vagin <avagin@openvz.org>
root_as_sibling was used in criu_signals_setup(), but was only defined later
(when forking the root task for the first time). This meant that the
SA_NOCLDSTOP was never masked off, which meant SIGCHLD was never delivered
after ptracing the root task. Thus, when the a child of the root task died
(e.g. from cr_system), the root task sat in PTRACE_STOP, and the restore task
never PTRACE_CONT'd, resulting in a deadlock.
Instead, we only unmask SA_NOCLDSTOP right before we PTRACE_SEIZE, after the
value is defined.
v2: re-work the condition for CLONE_PARENT
v3: move unmasking of SA_NOCLDSTOP to restore_root_task
v4: keep all the comments in the original code
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Here we define new api to be used in plugins.
- Plugin should provide a descriptor with help of
CR_PLUGIN_REGISTER macro, or in case if plugin require
no init/exit functions -- with CR_PLUGIN_REGISTER_DUMMY.
- Plugin should define a plugin hook with help of
CR_PLUGIN_REGISTER_HOOK macro.
- Now init/exit functions of plugins takes @stage
argument which tells plugin which stage of criu
it's been called on dump/restore. For exit it
also takes @ret which allows plugin to know if
something went wrong and it needs to cleanup
own resources.
The idea behind is to not limit plugins authors with names
of functions they might need to use for particular hook.
Such new API deprecates olds plugins structure but to keep
backward compatibility we will provide a tiny layer of
additional code to support old plugins for at least a couple
of release cycles.
For example a trivial plugin might look like
| #include <sys/types.h>
| #include <sys/stat.h>
| #include <fcntl.h>
| #include <libgen.h>
| #include <errno.h>
|
| #include <sys/socket.h>
| #include <linux/un.h>
|
| #include <stdio.h>
| #include <stdlib.h>
| #include <string.h>
| #include <unistd.h>
|
| #include "criu-plugin.h"
| #include "criu-log.h"
|
| static int dump_ext_file(int fd, int id)
| {
| pr_info("dump_ext_file: fd %d id %d\n", fd, id);
| return 0;
| }
|
| CR_PLUGIN_REGISTER_DUMMY("trivial")
| CR_PLUGIN_REGISTER_HOOK(CR_PLUGIN_HOOK__DUMP_EXT_FILE, dump_ext_file)
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Acked-by: Andrew Vagin <avagin@parallels.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
This is really just the last bit of c32046c9; if restore_one_task() fails, we
need to do the same futex wakeup we do everywhere else in this function.
v2: use err instead of err_fini_mnt after mount has been finalized normally
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
Acked-by: Acked-by: Andrew Vagin <avagin@parallels.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Once the task restore has failed, we can just abort, no need to restore the cg
props.
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
Acked-by: Andrew Vagin <avagin@parallels.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
When in --restore-detached (i.e. root_as_sibling) mode, we ptrace(PTRACE_SEIZE)
the root task to receive its SIGCHLD in case one of its child tasks dies.
However, we don't receive a SIGCHLD if the root task itself dies, so we must
explicitly abort.
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
Acked-by: Andrew Vagin <avagin@parallels.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
fini_cgroup umounts a cgyard directory, which is mounted
in prepare_cgroup().
Reported-by: Mr Jenkins
Signed-off-by: Andrew Vagin <avagin@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
In order to save backward compatibility, criu will try to open signal*.img,
if no signals_* are found.
Signed-off-by: Ruslan Kuprieiev <kupruser@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
We need to open cores for each thread early, because we'll need them to
prepare signals later.
Signed-off-by: Ruslan Kuprieiev <kupruser@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
The task_entries is a small structure used to coordinate the
processes restore stages. Currentl we allocate one page for
it and handle one separately. No need in this complexity, actually.
The rst_mem engine is already capable to controll this small object.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
This is a position in the RM_SHREMAP memory. Since shmems are currently
the only user of it, this is validly equals zero, but it will change soon.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
It's been discovered that on 3.11 we might fail on restore
if pass @CLONE_PARENT flag into clone() call due to kernel
limitations.
Because we're treating 3.11 as a base working kernel lets
do a trick instead
- setup this flag iif pdeath_sig is present
- if CLONE_NEWPID is passed warn a user about
potential consequences.
- because we need to carry the condition in attach_to_tasks
call, introduce @root_as_sibling variable for this.
CC: Tycho Andersen <tycho.andersen@canonical.com>
CC: Pavel Emelyanov <xemul@parallels.com>
CC: Andrey Vagin <avagin@openvz.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Acked-by: Andrey Vagin <avagin@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
If criu process attaches to the root task (it happens for opts.swrk_restore
and opts.restore_detach) with ptrace, then any signal delivered to the root
would be also delivered to criu. The latter woult treat the former to die
due to this delivery and would abort the restore.
Fix it by checking that criu (current == NULL) gets ptrace notification
(si_code == CLD_TRAPPED) about signal delivered (si_status = SIGCHLD,
no other signals are allowed by the restoring tasks).
This patch fixes the following error of static/zombie00:
Execute zdtm/live/static/zombie00
./zombie00 --pidfile=zombie00.pid --outfile=zombie00.out
Dump 2207
Restore
Test: zdtm/live/static/zombie00, Result: FAIL
==================================== ERROR ====================================
Restore log: /root/git/orig/criu/test/dump/static/zombie00/2207/1/restore.log
(00.026826) Error (cr-restore.c:1085): 2207 killed by signal 17
(00.026985) Error (cr-restore.c:1706): Restoring FAILED.
================================= ERROR OVER =================================
Reported-by: Mr Jenkins
Cc: Pavel Emelyanov <xemul@parallels.com>
Cc: Tycho Andersen <tycho.andersen@canonical.com>
Signed-off-by: Andrey Vagin <avagin@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
We need to use CLONE_PARENT to prevent processes from immediately dying due to
pdeath_sig when they are restored in detached mode.
[ xemul: One more place which requires check for restore-detach
is in sigactions preparation ]
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
They are now in per-pid images, but every entry contains a
pid to which it "belongs". This belonging is fake -- it's
just a pid of a task who placed the lock, while locks really
belong to files. We even have a bug when task that locked
a file exited and "delegated" the lock to its child.
This images merge reduces the amount of image files criu
generates and may simplify the fix of mentioned above issue.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Restores 2 cgroup properties after the criu restoration of tasks.
Currently the cgroup files to be restored are static but
are easily extendable. To change the properties to be restored,
edit this list at the top of cgroup.c. If a cgroup exists during
restoration, its properties will not be overwritten.
Work based off Tycho Anderson tycho.andersen@canonical.com
Change-Id: Ida32b9773eeac1d4d6e82ad644524ed099d5f9b1
Signed-off-by: Garrison Bellack <gbellack@google.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
There is an issue where if the proccess to be killed spawns a child proccess and
moves it in a child cgroup of the one the parent process is in, the cgroup fd
was being closed in the parent process before it forked the child. Then when
move_in_cgroup() is called for the child process, the file descriptor has
already been closed causing a failure for the second call to move_in_cgroup().
Moved the fd close after the fork call.
Change-Id: I6ae88b95c5410a7f56108e28eb3133f113e868d0
Signed-off-by: Garrison Bellack <gbellack@google.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
SIGMAX is a valid value, but the 0 signal doesn't exist.
Signed-off-by: Andrey Vagin <avagin@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
The helper task doesn't change sigaction and does nothing with
parent_sigacts. paren_sigacts will contain values for the previous alive
task, so the logic about inherence should work as expected.
Reported-by: Jenkins Criuovich
Signed-off-by: Andrew Vagin <avagin@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Most of the sigactions are the same across the tasks in the image.
Nonetheless existing code always calls a syscall to restore them
and spends 64 calls per-task.
Let's restore signals before forking children and let them inherit
sigactions. Tune one only if it differs from the parent's.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Andrew Vagin <avagin@parallels.com>
The whole idea behind this code was to stop receiving CHLD from
restored tasks after resume. The comment about this is done for
scripts is wrong (we call more scripts before this) because
sigchld_handler() knows about scripts:
commit de71bc69170cfeceb24bddd431ad10b8ea607d42
exit = (siginfo->si_code == CLD_EXITED);
status = siginfo->si_status;
+
+ /* skip scripts */
+ if (!current && root_item->pid.real != pid) {
+ pid = waitpid(root_item->pid.real, &status, WNOHANG);
+ if (pid <= 0)
+ return;
+ }
And since CHLD handler makes little sence after exec, it's easier
just to reset one to default action at the end.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Andrew Vagin <avagin@parallels.com>
We tune the CHLD handler if we're restoring root task
as sibling. This tuning is better to be done with one
sigaction() call, rather than two. First, it's shorter
and the second -- it will allow us to move the whole
criu signalling setup into one helper.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Andrew Vagin <avagin@parallels.com>
We don't need pid in any of these calls actually, they are
all legacy from the old days. I plan to move the call to
prepare_sigactions, so remove the pid argument in advance.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Andrew Vagin <avagin@parallels.com>
Currently we have this:
.......
/* No longer need it */
core_entry__free_unpacked(core, NULL);
ret = prepare_itimers(pid, core, task_args);
if (ret < 0)
goto err;
.......
So we're using ptr right after free-ing it.
Signed-off-by: Ruslan Kuprieiev <kupruser@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
On ARM some PTRACE_... constants are not declared in sys/ptrace.h file.
They are in linux/ptrace.h, but on x86 this file somewhat conflicts with
the sys/ one. For now fix ARM compilation by using criu/ one and think
of it later.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Andrey validly pointed out, that restoring pdeath_sig is not
compatible with criu_restore_child() call -- after criu restore
children, it will exit and fire the pdeath_sig into restored
tree root, potentially killing it.
The fix for that could be -- when started in swrk more, criu can
restore tree not as children tasks, but as siblings, using the
CLONE_PARENT flag when fork()-ing the root task.
With this we should also take care about errors handing -- right
now criu catches the SIGCHILD from dying children tasks, and
since we plan to create them be children of the criu parent (the
library caller) we will not be able to catch them. To do so we
SEIZE the root task in advance thus causing all SIGCHLD-s go to
criu, not to its parent.
Having this done we no longer need the SUBREAPER trick in the
library call -- tasks get restored right as callers kids :)
Some thoughts for future -- using this trick we can finally make
"natural" restoration of shell jobs. I.e. -- make criu restore
some subtree right under bash, w/o leaving itself as intermediate
task and w/o re-parenting the subtree to init after restore.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Andrey Vagin <avagin@parallels.com>
The implementation is pretty straightforward. When dumping per-thread
misc data with parasite, collect one, then write in thread_core_info.
On restore wait for creds restore and put the value back (some creds
changes drop it to zero).
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Serge E. Hallyn <serge.hallyn@ubuntu.com>
New kernel 3.16 will have old vDSO zone splitted into the two vmas:
one for vdso code itself and second that named vvar for data been
referenced from vdso code.
Because I can't do 'dump' and 'restore' parts of the code separately
(otherwise test would fail) the commit is pretty big one and hard to
read so here is detailed explanation what's going on.
1) When start dumping we detect vvar zone by reading /proc/pid/smap
and looking up for "[vvar]" token. Note the vvar zone is mapped
by a kernel with PF/IO flags so we should not fail here.
Also it's assumed that at least for now kernel won't be changed
much and [vvar] zone always follows the [vdso] zone, otherwise
criu will print error.
2) In previous commits we disabled dumping vvar area contents so
the restorer code never try to read vvar data but still we need
to map vvar zone thus vma entry remains in image.
3) As with previous vdso format we might have 2 cases
a) Dump and restore is happening on same kernel
b) Dump and restore are done on different kernels
To detect which case we have we parse vdso data from image
and find symbols offsets then compare their values with runtime
symbols provided us by a kernel. If they match and (!!!) the
size of vvar zone is the same -- we simply remap both zones
from runtime kernel into the positions dumpee had at checkpoint
time. This is that named "inplace" remap (a).
If this happens the vdso_proxify() routine drops VMA_AREA_REGULAR
from vvar area provided by a caller code and restorer won't try
to handle this vma. It looks somehow strange and probably should
be reworked but for now I left it as is to minimize the patch.
In case of (b) we need to generate a proxy. We do that in same
way as we were before just include vvar zone into proxy and save
vvar proxy address inside vdso mark injected into vdso area. Thus
on subsequent checkpoint we can detect proxy vvar zone and rip
it off the list of vmas to handle.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Acked-by: Andrew Vagin <avagin@parallels.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
