kea/src/lib/process/libprocess.dox

// Copyright (C) 2016-2021 Internet Systems Consortium, Inc. ("ISC")
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.

/**
 @page libprocess libkea-process - Controllable Process Layer (CPL)

@section cpl Controllable Process Layer (CPL)
During the design and development of D2 (Kea's DHCP-DDNS process), an abstract
layer for process control, called the Controllable Process Layer or CPL, was
created.  Kea's DHCP servers were initially developed prior to D2 and thus
before CPL existed.

Out of short term convenience and the fact that only D2 was using it, the CPL
was initially developed as part of D2 in src/bin/d2.  In order to use CPL for
new Kea processes, it has since been moved into its own library, libkea-process.
The following sections describe the architecture of CPL and how it can be used to implement new daemons in Kea.

The CPL provides the essentials for a controllable, configurable,
asynchronous process.  They are the result of an effort to distill the
common facets of process control currently duplicated in Kea's
DHCP servers into a reusable construct.  The classes which form this abstract
base are shown in the following class diagram:

@image html abstract_app_classes.svg  "Controllable Process Layer Classes"

- isc::process::DControllerBase - provides all of the services necessary to manage
an application process class derived from isc::d2::DProcess. These services include:
    - Command line argument handling
    - Process instantiation and initialization
    - Support for stand-alone execution
    - Process event loop invocation and shutdown

    It creates and manages an instance of isc::process::DProcessBase.  The CPL is
    designed for asynchronous event processing applications.  It is constructed
    to use ASIO library for IO processing.  @c DControllerBase owns an
    isc::asiolink::IOService instance and it passes this into the @c
    DProcessBase constructor. It is this @c IOService that is used to drive the
    process's event loop.  The controller is designed to provide any interfaces
    between the process it controls and the outside world.

    @c DControllerBase provides configuration for its process via a JSON file
    specified as a mandatory command line argument. The file structure is
    expected be as follows:

    { "<module-name>": {<module-config>} }

    where:
        - module-name : is a label which uniquely identifies the
        configuration data for the (i.e. the controlled process.)
        It is the value returned by @ref
        isc::process::DControllerBase::getAppName()

        - module-config: a set of zero or more JSON elements which comprise
        application's configuration values.  Element syntax is governed
        by those elements supported in isc::cc.

    The file may contain an arbitrary number of other modules.

    @todo Eventually, some sort of secure socket interface which supports remote
    control operations such as configuration changes or status reporting will
    likely be implemented.

- isc::process::DProcessBase - defines an asynchronous-event processor (i.e.
application) which provides a uniform interface to:
    - Instantiate and initialize a process instance
    - "Run" the application by starting its event loop
    - Inject events to control the process
It owns an instance of @c DCfgMgrBase.

- isc::process::DCfgMgrBase - provides the mechanisms for managing an application's
configuration.  This includes services for parsing sets of configuration
values, storing the parsed information in its converted form, and retrieving
the information on demand.  It owns an instance of @c DCfgContextBase, which
provides a "global" context for information that is accessible before, during,
and after parsing.

- isc::process::DCfgContextBase - implements a container for configuration
information or "context".  It provides a single enclosure for the storage of
configuration parameters or any other information that needs to accessible
within a given context.

The following sequence diagram shows how a configuration from file moves
through the CPL layer:

@image html config_from_file_sequence.svg "CPL Configuration From File Sequence"

The CPL classes will likely move into a common library.

@section cplSignals CPL Signal Handling

CPL supports interaction with the outside world via OS signals. The default
implementation supports the following signal driven behavior:
- SIGHUP receipt of this signal will cause a reloading of the configuration
file.
- SIGINT/SIGTERM receipt of either of these signals will initiate an
orderly shutdown.

CPL applications wait for for process asynchronous IO events through
isc::asiolink::IOService::run() or its variants.  These calls are not
interrupted upon signal receipt as is the select() function and while
boost::asio provides a signal mechanism it requires linking in additional
libraries.  Therefore, CPL provides its own signal handling mechanism to
propagate an OS signal such as SIGHUP to an IOService as a ready event with a
callback.

isc::process::DControllerBase uses two mechanisms to carry out signal handling.  It
uses isc::util::SignalSet to catch OS signals, and isc::process::IOSignalQueue to
propagate them to its isc::asiolink::IOService as instances of
isc::process::IOSignal.

This CPL signaling class hierarchy is illustrated in the following diagram:

@image html cpl_signal_classes.svg "CPL Signal Classes"

The mechanics of isc::process::IOSignal are straight forward. Upon construction it
is given the target isc::asiolink::IOService, the value of the OS signal to
send (e.g. SIGINT, SIGHUP...), and an isc::process::IOSignalHandler.  This handler
should contain the logic the caller would normally execute in its OS signal
handler. Each isc::process::IOSignal instance has a unique identifier called its
sequence_id.

Internally, IOSignal creates a 1 ms, one-shot timer, on the given
IOService.  When the timer expires its event handler invokes the caller's
IOSignalHandler passing it the sequence_id of the IOSignal.

Sending IOSignals is done through an isc::process::IOSignalQueue.  This class is
used to create the signals, house them until they are delivered, and dequeue
them so they can be been handled.  To generate an IOSignal when an OS signal
arrives, the process's OS signal handler need only call
isc::process::IOSignalQueue::pushSignal() with the appropriate values.

To dequeue the IOSignal inside the caller's IOSignalHandler, one simply
invokes isc::process::IOSignalQueue::popSignal() passing it the sequence_id
parameter passed to the handler.  This method returns a pointer to
instigating IOSignal from which the value of OS signal (i.e. SIGINT,
SIGUSR1...) can be obtained.  Note that calling popSignal() removes the
IOSignalPtr from the queue, which should reduce its reference count to
zero upon exiting the handler (unless a deliberate copy of it is made).

A typical isc::process::IOSignalHandler might be structured as follows:
@code

    void processSignal(IOSignalId sequence_id) {
    // Pop the signal instance off the queue.
    IOSignalPtr signal = io_signal_queue_->popSignal(sequence_id);

    int os_signal_value = signal->getSignum();
    :
    // logic based on the signal value
    :
    }

@endcode

IOSignal's handler invocation code will catch, log ,and then swallow any
exceptions thrown by an IOSignalHandler.  This is done to protect the integrity
IOService context.

The following sequence diagram depicts the initialization of signal handling
during startup and the subsequent receipt of a SIGHUP:

@image html cpl_signal_sequence.svg "CPL Signal Handling Sequence"

@section redact Redact Passwords

There are two tools to remove sensitive data as passwords or secrets from logs:
 - redactedAccessString for database access strings
 - redactConfig for full configurations

The jsonPathsToRedact method must be defined in derived classes following this
procedure:
 - Get all possible JSON paths from the root of the configuration to leaves that
 fulfill the role of map keys and which contain "password" or "secret".
 - For each of these paths, remove the root node and the leaf node.
 - Include all the paths in the method. Duplicate subpaths are expected in the
 case of common subpaths to different leaves.

There are two special syntaxes:
 - "[]" suggests that the searched element is a list. This is required for all
 lists and is for performance gain.
 - "*" as a last element in a JSON path tells the redacter to look in all
 elements that follow for elements that contain "password" and "secret". This is
 when the particular configuration that is targeted by the "*" does not have a
 well defined structure, such as is the case for "parameters" in the
 "hooks-libraries" map in "Dhcp4" and "Dhcp6".

@section cplMTConsiderations Multi-Threading Consideration for Controllable Process Layer

By default this library is not thread safe and currently there is no known
exception.

*/