Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Support running on MIG on Spark on YARN #4023

Closed
tgravescs opened this issue Nov 3, 2021 · 3 comments · Fixed by #4106
Closed

[FEA] Support running on MIG on Spark on YARN #4023

tgravescs opened this issue Nov 3, 2021 · 3 comments · Fixed by #4106
Assignees
@tgravescs
Labels
feature request New feature or request

Comments

@tgravescs
Copy link
Collaborator

Allow the plugin to run using MIG on Spark on YARN.

This will likely depend on changes to YARN to support properly, but filing this for changes in plugin and documentation changes.
@tgravescs tgravescs added feature request New feature or request ? - Needs Triage Need team to review and classify and removed ? - Needs Triage Need team to review and classify labels Nov 3, 2021
@tgravescs tgravescs self-assigned this Nov 3, 2021
@tgravescs tgravescs mentioned this issue Nov 3, 2021
Merged
@tgravescs
Copy link
Collaborator Author

Note Spark support for remapping yarn.io/gpu to any resource type: https://issues.apache.org/jira/browse/SPARK-37208

@tgravescs
Copy link
Collaborator Author

YARN 3.3.0+ device plugin to support MIG gpus: NVIDIA/spark-rapids-examples#48

@tgravescs
Copy link
Collaborator Author

YARN 3.1.2 thru 3.3.0 pr: NVIDIA/spark-rapids-examples#51

@tgravescs tgravescs mentioned this issue Nov 15, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tgravescs tgravescs

Labels
feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add docs for MIG on YARN [skip ci] tgravescs/spark-rapids
1 participant
@tgravescs