Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support building multi-arch containers for CI #90

Merged
merged 5 commits into from
Jan 15, 2025

Conversation

dagardner-nv
Copy link
Contributor

Description

  • Build CI containers for arm64 & amd64
  • Document using qemu for performing arm64 builds from an amd64 host (or potentially vice versa)

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@dagardner-nv dagardner-nv added non-breaking Non-breaking change 2 - In Progress PR/Issue is currently a work in progress improvement Improvement to existing functionality labels Dec 20, 2024
@dagardner-nv dagardner-nv self-assigned this Dec 20, 2024
@dagardner-nv dagardner-nv requested a review from a team as a code owner December 20, 2024 17:03
@dagardner-nv dagardner-nv marked this pull request as draft December 20, 2024 17:04
@dagardner-nv dagardner-nv added 3 - Ready for Review PR/Issue is complete and ready for review by team and removed 2 - In Progress PR/Issue is currently a work in progress labels Dec 21, 2024
@dagardner-nv dagardner-nv marked this pull request as ready for review December 21, 2024 00:29
@dagardner-nv
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit d97859b into nv-morpheus:branch-25.02 Jan 15, 2025
2 checks passed
@dagardner-nv dagardner-nv deleted the david-arm-ci branch January 15, 2025 00:52
rapids-bot bot pushed a commit to nv-morpheus/MRC that referenced this pull request Jan 15, 2025
* Add Arm64 build and test stages to CI
* Refactor `Dockerfile` as a multi-arch container
* Add Arm64 (`aarch64`) to `dependencies.yaml` matrix
* Applies fix to #525 provided by @willkill07 
* Adds Arm64 Conda package output

Related to nv-morpheus/utilities#90
Closes #525

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Will Killian (https://github.com/willkill07)
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #524
rapids-bot bot pushed a commit to nv-morpheus/Morpheus that referenced this pull request Jan 16, 2025
* Add Arm64 build and test stages to CI
* Refactor CI docker images as a multi-arch container
* Add Arm64 Conda package output
* Add Arm64 (`aarch64`) to `dependencies.yaml` matrix
* Skip DOCA builds for arm (#2092).
* Skip tests for ARM conda builds, this avoids the need for an ARM GPU runner.
* The following packages are x86_64 specific: include-what-you-use (later versions support ARM), vale, milvus and pymilvus.
* The following packages have arm builds on pypi but not on conda-forge: pypdfium2, newspaper3k (package is no-arch but some deps are x86_64 only). For these we obtain them via pip on arm, but continue to install via conda for x86_64.
* pytorch cuda builds exist for ARM but lack the same meta-data that the x86_64 builds do. For now the cpu version of torch is installed for ARM #2095 

Related to nv-morpheus/utilities#90
Requires nv-morpheus/MRC#524 to be merged first
Closes [#2094](#2094)

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #2093
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review PR/Issue is complete and ready for review by team improvement Improvement to existing functionality non-breaking Non-breaking change
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants