From 46acf7003a9897a8b54adfda0deb6798a28a6632 Mon Sep 17 00:00:00 2001 From: Gregor von Laszewski Date: Thu, 18 Aug 2022 07:45:11 -0400 Subject: [PATCH 1/3] Create README-summit.md Detailed description of how to compile cylon on summit --- README-summit.md | 94 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 README-summit.md diff --git a/README-summit.md b/README-summit.md new file mode 100644 index 000000000..a7bde0f9d --- /dev/null +++ b/README-summit.md @@ -0,0 +1,94 @@ +# SUMMIT Instalation guide + +This document describes the instalation on summit. +The instalition is first described in detail and then an abbreviated +instalation with a Makefile is provided. + +Notation: + +* `login$` denotes the login node +* `compute$` donates a compute node + +## Details + +### Activites on the login node + +```bash +login$ module purge +login$ module load gcc/9.3.0 +login$ module load spectrum-mpi/10.4.0.3-20210112 +login$ module load python/3.8-anaconda3 +``` + +Next check the python version + +```bash +login$ which python +> /sw/summit/python/3.8/anaconda3/2020.07-rhel8/bin/python +login$ python -V +> Python 3.8.3 +``` + +Now create a venv called `CYLON` in the home directory and activate it + +```bash +login$ python -m venv ~/CYLON +login$ source ~/CYLON/bin/activate +``` + +```bash +login$ which python +> ~/CYLON/bin/python +login$ python -V +> Python 3.8.3 +``` + +Update pip and install pytest: + +```bash +login$ pip install pip -U +login$ pip install pytest +``` + +Checkout the cylon code + +```bash +cd ~git clone https://github.com/cylondata/cylon.git +``` + +Now we obtain two interactive compute nodes + +```bash +login$ bsub -Is -W 0:30 -nnodes 2 -P gen150_bench $SHELL +``` + +After you get them you will be in a compute node + +### Activities on the compute node + +```bash +compute$ module purge +compute$ module load gcc/9.3.0 +compute$ module load spectrum-mpi/10.4.0.3-20210112 +compute$ module load python/3.8-anaconda3 +compute$ source ~/CYLON/bin/activate +compute$ export CC=`which gcc` +compute$ export CXX=`which g++` +compute$ export ARROW_DEFAULT_MEMORY_POOL=system +compute$ CC=gcc MPICC=mpicc pip install --no-binary mpi4py install mpi4py +compute$ pip install pytest-mpi +compute$ pip install cmake +compute$ pip install numpy +compute$ ./build.sh -pyenv ~/CYLON -bpath $(pwd)/build --cpp --test --cmake-flags "-DMPI_C_COMPILER=$(which mpicc) -DMPI_CXX_COMPILER=$(which mpicxx)" -j 4 +``` + +The compilation will take some time. After it is completed you can conduct a test with + +```bash +compute$ cd ~/cylon/build +compute$ jsrun -n 1 -c 4 -a 1 ./build/bin/join_example m n 1000 0.9 +``` + +### Running batch scripts + +Please note that the module load and the source of the CYLON venv must be at the beginning of each batsch script you want to use cylon in From 84eba4a446f3c195aaa58adda17bb912d92aabf7 Mon Sep 17 00:00:00 2001 From: Gregor von Laszewski Date: Thu, 18 Aug 2022 08:03:07 -0400 Subject: [PATCH 2/3] Update README-summit.md --- README-summit.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/README-summit.md b/README-summit.md index a7bde0f9d..0805abe7f 100644 --- a/README-summit.md +++ b/README-summit.md @@ -89,6 +89,15 @@ compute$ cd ~/cylon/build compute$ jsrun -n 1 -c 4 -a 1 ./build/bin/join_example m n 1000 0.9 ``` +If everything is ok, you will see at the end of the test output + +``` +... +================================================ +All tests passed (66 assertions in 4 test cases) +``` + + ### Running batch scripts -Please note that the module load and the source of the CYLON venv must be at the beginning of each batsch script you want to use cylon in +Please note that the module load and the source of the CYLON venv must be at the beginning of each batsch script you want to use cylon in. From 3e8be438c223d9aaab7e5bf3a54b9e76ecff8ead Mon Sep 17 00:00:00 2001 From: Gregor von Laszewski Date: Thu, 18 Aug 2022 08:10:04 -0400 Subject: [PATCH 3/3] Update README-summit.md --- README-summit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README-summit.md b/README-summit.md index 0805abe7f..1428d3e3e 100644 --- a/README-summit.md +++ b/README-summit.md @@ -1,4 +1,4 @@ -# SUMMIT Instalation guide +# SUMMIT Installation guide This document describes the instalation on summit. The instalition is first described in detail and then an abbreviated