multiprocess: Add comments and documentation

Author: ryanofsky

doc/multiprocess.md

@@ -15,7 +15,7 @@ Specific next steps after [#10102](https://github.com/bitcoin/bitcoin/pull/10102
 
 ## Debugging
 
-After [#10102](https://github.com/bitcoin/bitcoin/pull/10102), the `-debug=ipc` command line option can be used to see requests and responses between processes.
+The `-debug=ipc` command line option can be used to see requests and responses between processes.
 
 ## Installation
 
@@ -33,3 +33,40 @@ BITCOIND=bitcoin-node test/functional/test_runner.py
 The configure script will pick up settings and library locations from the depends directory, so there is no need to pass `--enable-multiprocess` as a separate flag when using the depends system (it's controlled by the `MULTIPROCESS=1` option).
 
 Alternately, you can install [Cap'n Proto](https://capnproto.org/) and [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) packages on your system, and just run `./configure --enable-multiprocess` without using the depends system. The configure script will be able to locate the installed packages via [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/). See [Installation](https://github.com/chaincodelabs/libmultiprocess#installation) section of the libmultiprocess readme for install steps. See [build-unix.md](build-unix.md) and [build-osx.md](build-osx.md) for information about installing dependencies in general.
+
+## IPC implementation details
+
+Cross process Node, Wallet, and Chain interfaces are defined in
+[`src/interfaces/`](../src/interfaces/). These are C++ classes which follow
+[conventions](developer-notes.md#internal-interface-guidelines), like passing
+serializable arguments so they can be called from different processes, and
+making methods pure virtual so they can have proxy implementations that forward
+calls between processes.
+
+When Wallet, Node, and Chain code is running in the same process, calling any
+interface method invokes the implementation directly. When code is running in
+different processes, calling an interface method invokes a proxy interface
+implementation that communicates with a remote process and invokes the real
+implementation in the remote process. The
+[libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) code
+generation tool internally generates proxy client classes and proxy server
+classes for this purpose that are thin wrappers around Cap'n Proto
+[client](https://capnproto.org/cxxrpc.html#clients) and
+[server](https://capnproto.org/cxxrpc.html#servers) classes, which handle the
+actual serialization and socket communication.
+
+As much as possible, calls between processes are meant to work the same as
+calls within a single process without adding limitations or requiring extra
+implementation effort. Processes communicate with each other by calling regular
+[C++ interface methods](../src/interfaces/README.md). Method arguments and
+return values are automatically serialized and sent between processes. Object
+references and `std::function` arguments are automatically tracked and mapped
+to allow invoked code to call back into invoking code at any time, and there is
+a 1:1 threading model where any thread invoking a method in another process has
+a corresponding thread in the invoked process responsible for executing all
+method calls from the source thread, without blocking I/O or holding up another
+call, and using the same thread local variables, locks, and callbacks between
+calls. The forwarding, tracking, and threading is implemented inside the
+[libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library
+which has the design goal of making calls between processes look like calls in
+the same process to the extent possible.

src/interfaces/README.md

@@ -12,6 +12,8 @@ The following interfaces are defined here:
 
 * [`Handler`](handler.h) — returned by `handleEvent` methods on interfaces above and used to manage lifetimes of event handlers.
 
-* [`Init`](init.h) — used by multiprocess code to access interfaces above on startup. Added in [#10102](https://github.com/bitcoin/bitcoin/pull/10102).
+* [`Init`](init.h) — used by multiprocess code to access interfaces above on startup. Added in [#19160](https://github.com/bitcoin/bitcoin/pull/19160).
 
-The interfaces above define boundaries between major components of bitcoin code (node, wallet, and gui), making it possible for them to run in different processes, and be tested, developed, and understood independently. These interfaces are not currently designed to be stable or to be used externally.
+* [`Ipc`](ipc.h) — used by multiprocess code to access `Init` interface across processes. Added in [#19160](https://github.com/bitcoin/bitcoin/pull/19160).
+
+The interfaces above define boundaries between major components of bitcoin code (node, wallet, and gui), making it possible for them to run in [different processes](../../doc/multiprocess.md), and be tested, developed, and understood independently. These interfaces are not currently designed to be stable or to be used externally.

src/interfaces/ipc.h

@@ -13,7 +13,30 @@ namespace interfaces {
 class Init;
 
 //! Interface providing access to interprocess-communication (IPC)
-//! functionality.
+//! functionality. The IPC implementation is responsible for establishing
+//! connections between a controlling process and a process being controlled.
+//! When a connection is established, the process being controlled returns an
+//! interfaces::Init pointer to the controlling process, which the controlling
+//! process can use to get access to other interfaces and functionality.
+//!
+//! When spawning a new process, the steps are:
+//!
+//! 1. The controlling process calls interfaces::Ipc::spawnProcess(), which
+//!    calls ipc::Process::spawn(), which spawns a new process and returns a
+//!    socketpair file descriptor for communicating with it.
+//!    interfaces::Ipc::spawnProcess() then calls ipc::Protocol::connect()
+//!    passing the socketpair descriptor, which returns a local proxy
+//!    interfaces::Init implementation calling remote interfaces::Init methods.
+//! 2. The spawned process calls interfaces::Ipc::startSpawnProcess(), which
+//!    calls ipc::Process::checkSpawned() to read command line arguments and
+//!    determine whether it is a spawned process and what socketpair file
+//!    descriptor it should use. It then calls ipc::Protocol::serve() to handle
+//!    incoming requests from the socketpair and invoke interfaces::Init
+//!    interface methods, and exit when the socket is closed.
+//! 3. The controlling process calls local proxy interfaces::Init object methods
+//!    to make other proxy objects calling other remote interfaces. It can also
+//!    destroy the initial interfaces::Init object to close the connection and
+//!    shut down the spawned process.
 class Ipc
 {
 public: