MAINT: Use RAPIDS 24.10 cmake and scikit-build-core setup #195

seberg · 2024-12-17T12:57:33Z

This changes the setup to align it with the typical setup in 24.10 RAPIDS (and future) using scikit-build-core.
legate is currently using 24.10 rapids-cmake, so it seems good to keep this in sync.

seberg · 2024-12-17T13:44:14Z

pyproject.toml

+[tool.rapids-build-backend]
+build-backend = "scikit_build_core.build"
+dependencies-file = "dependencies.yaml"
+commit_files = ["legateboost/GIT_COMMIT"]


rapids-build-backend default to adding a GIT_COMMIT file, which is fine. But because the project name legate-boost and the module name legateboost mismatch, we have to add it explicitly (I think).

EDIT: spelled commit-files here of course.

Yes you're exactly right about this. Here's where that behavior was introduced: rapidsai/rapids-build-backend#30

This changes the setup to align it with the typical setup in 24.10 RAPIDS (and future) using scikit-build-core. `legate` is currently using 24.10 rapids-cmake, so it seems good to keep this in sync. Signed-off-by: Sebastian Berg <[email protected]>

jameslamb

Awesome work!

I totally support this... getting caught up to a newer rapids-cmake and switching to scikit-build-core are really nice changes. It's great to see setup.py deleted 😁

Left a few small comments, but overall I support this. I think we should answer the rapids-cmake-sha question before merging this.

jameslamb · 2024-12-17T15:51:11Z

dependencies.yaml

+    pyproject_dir: .
+    extras:
+      table: tool.rapids-build-backend
+      key: requires
    includes:
      - build
      - build_tools


If you want this to look a bit more like other RAPIDS libraries, then by convention these 2 lists would be spelled like this:

py_build_legateboost: output: pyproject pyproject_dir: . extras: table: build-system includes: - rapids_build_skbuild py_rapids_build_legateboost: output: pyproject pyproject_dir: . extras: table: tool.rapids-build-backend key: requires includes: - build - build_tools

There are some docs on those conventions at https://github.com/rapidsai/build-planning/blob/d9e3c606d95c835ee384ac6480a4af0ac6cb024a/docs/docs/packaging.md#L85

Not critical, but following RAPIDS conventions does make it easier for more other RAPIDS folks to come in and help out on these repos.

jameslamb · 2024-12-17T15:54:50Z

setup.py

-cmake_flags = [
-    f"-Dlegate_ROOT:STRING={legate_dir}",
-    f"-DCMAKE_CUDA_ARCHITECTURES={cuda_arch}",
-]


With this removed, will the following still work?

legate-boost/contributing.md

Lines 47 to 52 in ba062bf

If installing with `pip`, set the `CUDAARCHS` environment variable, as described in the CMake docs ([link](https://cmake.org/cmake/help/latest/variable/CMAKE_CUDA_ARCHITECTURES.html)).

```shell

CUDAARCHS="70;80" \

pip install --no-build-isolation --no-deps .

```

It might need to be updated to something like

SKBUILD_CMAKE_ARGS="-DCMAKE_CUDA_ARCHITECTURES='70;80'"

Or something similar.

Good find! Pushing another commit to introduce rapids_cuda_init_architectures(legateboost), that seems like a reasonable thing to me. In legate-raft and legate-dataframe, I did use the "RAPIDS" architectures (but then, they use rapids libraries...)

The difference is that if you just do a plain pip install . RAPIDS seems to default to the "RAPIDS" architectures and not "native".
So there is a slightly unfortunate change, but at least this will keep working (maybe only with the new commit).

Ah perfect, rapids_cuda_init_architectures() seems like the right way to do this (I did not actually know about that 🙈 )

jameslamb · 2024-12-17T15:59:49Z

cmake/legateboost/get_rapids_cmake.cmake

@@ -17,7 +17,7 @@ macro(legateboost_get_rapids_cmake)

  if(NOT rapids-cmake-version)
    # default
-    set(rapids-cmake-version 24.08)
+    set(rapids-cmake-version 24.10)
    set(rapids-cmake-sha "3cc764f287a6f3caeee5dd1c96c24b1710d4cdf1")


I don't know from memory... does the version matter if we're still pinning to a specific commit of rapids-cmake?

I can't tell what version is getting downloaded from the build logs since they're pretty quiet: https://github.com/rapidsai/legate-boost/actions/runs/12374421493/job/34536930592?pr=195

I'd support updating their verbosity. Not sure the right place...maybe PIP INSTALL_ARGS should pick up -v in build.sh? Or maybe we need to set CMake verbosity in scikit-build-core: https://scikit-build-core.readthedocs.io/en/latest/configuration.html#verbosity. Or maybe both?

I added a lot of verbosity, but I think not in the one command (the first cmake run) that actually matters :).

As far as I can tell, this was not used at all and I removed it. My suspicion would be it was a temporary work-around at some point, but didn't double check.

Ha ok thanks! I think it's great that we're getting this CMake configure output now in CI logs... a lot of solutions to bugs are found in those logs in my experience:

If just removing the sha works then I think we're good to merge this 😁

Signed-off-by: Sebastian Berg <[email protected]>

jameslamb

looks great!

seberg · 2025-01-02T11:47:18Z

Putting it in, I suppose the build in CI may actually fail because gh-196 is required. I'll ask for a review there (it is trivial).

seberg force-pushed the rapids-24.10-build branch 5 times, most recently from c1cb9f4 to f2eb602 Compare December 17, 2024 13:43

seberg commented Dec 17, 2024

View reviewed changes

seberg force-pushed the rapids-24.10-build branch from f2eb602 to 25ede7f Compare December 17, 2024 13:51

seberg marked this pull request as ready for review December 17, 2024 14:20

seberg requested a review from jameslamb December 17, 2024 14:20

jameslamb reviewed Dec 17, 2024

View reviewed changes

seberg added 3 commits December 17, 2024 16:36

BLD: Use rapids_cuda_init_architectures and make build verbose

0138f90

Signed-off-by: Sebastian Berg <[email protected]>

MAINT: Remove rapids-cmake-sha (not sure this did anything)

8a713f8

Signed-off-by: Sebastian Berg <[email protected]>

MAINT: Rename build section to match rapids convention (include name)

2c460bd

Signed-off-by: Sebastian Berg <[email protected]>

seberg requested a review from jameslamb December 31, 2024 11:39

jameslamb approved these changes Dec 31, 2024

View reviewed changes

seberg merged commit 074ac15 into rapidsai:main Jan 2, 2025
10 checks passed

seberg deleted the rapids-24.10-build branch January 2, 2025 11:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MAINT: Use RAPIDS 24.10 cmake and scikit-build-core setup #195

MAINT: Use RAPIDS 24.10 cmake and scikit-build-core setup #195

seberg commented Dec 17, 2024

seberg Dec 17, 2024 •

edited

Loading

jameslamb Dec 17, 2024

jameslamb left a comment

jameslamb Dec 17, 2024

jameslamb Dec 17, 2024

seberg Dec 17, 2024

jameslamb Dec 17, 2024

jameslamb Dec 17, 2024

seberg Dec 17, 2024

jameslamb Dec 17, 2024

jameslamb left a comment

seberg commented Jan 2, 2025

	If installing with `pip`, set the `CUDAARCHS` environment variable, as described in the CMake docs ([link](https://cmake.org/cmake/help/latest/variable/CMAKE_CUDA_ARCHITECTURES.html)).

	```shell
	CUDAARCHS="70;80" \
	pip install --no-build-isolation --no-deps .
	```

MAINT: Use RAPIDS 24.10 cmake and scikit-build-core setup #195

MAINT: Use RAPIDS 24.10 cmake and scikit-build-core setup #195

Conversation

seberg commented Dec 17, 2024

seberg Dec 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jameslamb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jameslamb left a comment

Choose a reason for hiding this comment

seberg commented Jan 2, 2025

seberg Dec 17, 2024 •

edited

Loading