Skip to content

Commit

Permalink
docs/dask.md: update links due to src/mintpy structure change (#1308
Browse files Browse the repository at this point in the history
)

* docs: update mintpy references

Signed-off-by: Emmanuel Ferdman <[email protected]>

* Update dask.md

---------

Signed-off-by: Emmanuel Ferdman <[email protected]>
  • Loading branch information
emmanuel-ferdman authored Dec 30, 2024
1 parent 5ecef06 commit 8f7d186
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/dask.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Configure dask for parallel processing #

Most computations in MintPy are operated in either a pixel-by-pixel or a epoch-by-epoch basis. This implementation strategy allows processing different blocks (in space or in time) in parallel. For this purpose, we use the [`Dask`](https://docs.dask.org/en/latest/) library for its dynamic task scheduling and data collection. Dask support is currently implemented in `ifgram_inversion.py` and `dem_error.py` only (expansions to other modules are welcomed) through a thin wrapper in [`mintpy.objects.cluster`](../mintpy/objects/cluster.py). We have tested two types of clusters:
Most computations in MintPy are operated in either a pixel-by-pixel or a epoch-by-epoch basis. This implementation strategy allows processing different blocks (in space or in time) in parallel. For this purpose, we use the [`Dask`](https://docs.dask.org/en/latest/) library for its dynamic task scheduling and data collection. Dask support is currently implemented in `ifgram_inversion.py` and `dem_error.py` only (expansions to other modules are welcomed) through a thin wrapper in [`mintpy.objects.cluster`](../src/mintpy/objects/cluster.py). We have tested two types of clusters:

+ **local cluster:** on a single machine (laptop or computing node) with multiple CPU cores, suitable for laptops, local cluster/stations and distributed High Performance Cluster (HPC). No job scheduler is required.
+ **non-local cluster:** on a distributed HPC with job scheduler installed, including PBS, LSF and SLURM.
Expand Down Expand Up @@ -69,7 +69,7 @@ PBScluster did not work either. But we tested only on a small server without sha
-------------------------------------------
The parallel proceesing on multiple machines is supported via [`Dask-jobqueue`](https://jobqueue.dask.org/en/latest/index.html). One can specify configuration either with keyword arguments when creating a `Cluster` object, or with a configuration file in YAML format. MintPy assumes the YAML configuration file only.

We provide an example [YAML configuration file](../mintpy/defaults/mintpy.yaml), besides the `dask.yaml`, `distributed.yaml` and `jobqueue.yaml` files in `~/.config/dask` installed by dask by default. One can copy it over to the `~/.config/dask` directory as below for dask to identify and use it.
We provide an example [YAML configuration file](../src/mintpy/defaults/mintpy.yaml), besides the `dask.yaml`, `distributed.yaml` and `jobqueue.yaml` files in `~/.config/dask` installed by dask by default. One can copy it over to the `~/.config/dask` directory as below for dask to identify and use it.

```bash
cp $MINTPY_HOME/src/mintpy/defaults/mintpy.yaml ~/.config/dask/mintpy.yaml
Expand Down

0 comments on commit 8f7d186

Please sign in to comment.