Skip to content

Commit

Permalink
Disable CuMemMap check for ROCm (#411)
Browse files Browse the repository at this point in the history
Co-authored-by: Changho Hwang <[email protected]>
  • Loading branch information
Binyang2014 and chhwang committed Dec 20, 2024
1 parent ee75caf commit 0cc0256
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 8 deletions.
10 changes: 3 additions & 7 deletions docs/design/nccl-over-mscclpp.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,7 @@ The table below lists all NCCL APIs (v2.21). We may cover more APIs in the futur

The executor is a versatile tool designed to specify how mscclpp executes algorithms. Currently, only the allReduce operation allows for algorithm customization. The following environment variables can be managed:

- ALLREDUCEPKT_IP_JSON_FILE: Specifies the path to the JSON file that defines the algorithm for small-sized, in-place operations.
- ALLREDUCEPKT_OP_JSON_FILE: Specifies the path to the JSON file that defines the algorithm for small-sized, out-of-place operations.
- ALLREDUCE_IP_JSON_FILE: Specifies the path to the JSON file that defines the algorithm for larger-sized, in-place operations.
- ALLREDUCE_OP_JSON_FILE: Specifies the path to the JSON file that defines the algorithm for larger-sized, out-of-place operations.
- ALLREDUCE_SMALL_MSG_BOUNDARY: Defines the size threshold at which the algorithm will switch between fallback code and the customized algorithm for small messages.
- ALLREDUCE_LARGE_MSG_BOUNDARY: Defines the size threshold at which the algorithm will switch between the customized algorithm for small messages and that for larger messages.
- MSCCLPP_EXECUTION_PLAN_DIR: Specifies the directory where the executor will look for JSON files.

```{figure} ../figs/size_boundary_diagram.png
:name: MMSCCL++ Abstractions
Expand All @@ -68,4 +63,5 @@ Decision Flowchart for Message Size-Based Algorithm Execution

This is an example of executing the interface with the executor:
``` bash
mpirun -np 8 -x ALLREDUCEPKT_IP_JSON_FILE=/root/azure-mscclpp/nccl/test/execution-files/allreducepacket.json -x ALLREDUCE_IP_JSON_FILE=/root/azure-mscclpp/nccl/test/execution-files/allreducesm.json -x ALLREDUCE_SMALL_MSG_BOUNDARY=16K -x ALLREDUCE_LARGE_MSG_BOUNDARY=1M ./apps/nccl/test/nccl_api_test
mpirun -np 8 -x MSCCLPP_EXECUTION_PLAN_DIR=/root/azure-mscclpp/nccl/test/execution-files ./apps/nccl/test/nccl_api_test
```
6 changes: 5 additions & 1 deletion src/registered_memory.cc
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,10 @@ CUmemAllocationHandleType getNvlsCompatibleMemHandleType() {
}

// Check if ptr is allocaed by cuMemMap
bool isCuMemMapAllocated(void* ptr) {
bool isCuMemMapAllocated([[maybe_unused]] void* ptr) {
#if defined(__HIP_PLATFORM_AMD__)
return false;
#else
CUmemGenericAllocationHandle handle;
CUresult result = cuMemRetainAllocationHandle(&handle, ptr);
if (result != CUDA_SUCCESS) {
Expand All @@ -66,6 +69,7 @@ bool isCuMemMapAllocated(void* ptr) {
throw mscclpp::Error("cuMemMap is used in env without NVLS support", mscclpp::ErrorCode::InvalidUsage);
}
return true;
#endif
}

} // namespace
Expand Down

0 comments on commit 0cc0256

Please sign in to comment.