Skip to content

Commit

Permalink
fixup! doc: add tuning parameters for MPI_THREAD_MULTIPLE
Browse files Browse the repository at this point in the history
  • Loading branch information
sagarth committed Apr 1, 2022
1 parent 61afa30 commit 38a129b
Showing 1 changed file with 16 additions and 0 deletions.
16 changes: 16 additions & 0 deletions doc/mpich/TUNING_PARAMETERS
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,26 @@ VCI lock and a `lockless` mode where MPI does not grab a lock and relies on the
network provider to provide fine grain synchronization (MPICH uses the
FI_THREAD_SAFE mode from OFI to support the `lockless` mode).

A threading synchronization strategy can be selected at the configuration time
as:
--enable-thread-cs=global to select coarse-grained locking, and
--enable-thread-cs=per-vci to select fine-grained locking.

Under fine-grained locking, a specific locking mode at configuration time as:
--enable-ch4-mt=<mode>, mode can be one of direct, lockless and runtime.

If configuration is done with --enable-ch4-mt=runtime, user can select a
threading mode at runtime using the environment MPIR_CVAR_CH4_MT_MODEL
(described below in Section 1.1).

**1.1 CVARs (Environment Variables)**

MPICH users can use the CVAR MPIR_CVAR_CH4_MT_MODEL to select at runtime among
the threading synchronization models, `global`, `direct`, or `lockless`. If the
CVAR is not set, the default model is `direct`.

e.g., export MPIR_CVAR_CH4_MT_MODEL=direct

Users can specify the number of VCIs to use by setting the environment variable
`MPIR_CVAR_CH4_NUM_VCIS=<number_vcis>`, where the value `<num_vcis>` should be
less or equal than the number of network contexts provided by the OFI provider.
Expand All @@ -58,6 +72,8 @@ receive contexts (max_ep_rx_ctx) it supports. MPICH uses the minimum of
max_ep_tx_ctx and max_ep_rx_ctx as the maximum number of VCIs it can support. If
the system has multiple NICS, this value is the `minimum across all the NICS`.

e.g., export MPIR_CVAR_CH4_NUM_VCIS=16

When multiple VCIs are used, MPICH tries to make per-vci progress to avoid lock
contention inside the progress engine. However, it also progresses all VCIs,
i.e., make global progress, once in a while to avoid potential deadlocks. Such
Expand Down

0 comments on commit 38a129b

Please sign in to comment.