What is the recommended way of using OpenMPI compiler wrappers for CUDA applications?

[gonzalobg]

For C++ applications I use the mpicxx compiler wrapper, and I can select the C++ compiler using the OMPI_CXX environment variable.

What's the recommended way to compile CUDA applications with the OpenMPI compiler wrappers?

[janjust]

For mpi applications just the wrapper will do.

For building .cu object files and link with mpi.
Build the .cu objects and .c objects, and link them with the mpi wrapper.

Take a look at this

[gonzalobg]

Thanks. This:

module load mpi cuda
mpicc -c main.c -o main.o
nvcc -c multiply.cu -o multiply.o
mpicc main.o multiply.o -lcudart

does not cover how to build a main.cu with the mpi wrapper.

That is, given a single GPU application in which all files, including main.cu, are .cu files, and a user wants to drop in some MPI calls here and there to turn it into a multi-GPU application, how does the user do that?

It would be nice to have this documented in the OpenMPI documentation.

Ideally it would be as easy as:

modue load mpi cuda
mpi___ -c main.cu -o main.o
mpi___ -c multiply.cu -o multiply.o
mpi___ main.o multiply.o

[janjust]

It would be nice to have this documented in the OpenMPI documentation.

Agreed, should probably be documented better @Akshay-Venkatesh
@gonzalobg Thanks for the proposal, would you mind joining a dev call and pitch the proposal?