Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add Linux CUDA builds #9804

Closed
wants to merge 3 commits into from
Closed

Conversation

jakirkham
Copy link
Member

@jakirkham jakirkham commented Oct 12, 2019

Fixes #9773
Merges #9802

This adds Linux CUDA builds for the supported versions of CUDA. These are part of a matrix job that builds each CUDA version sequentially to avoid using too many jobs when these are not needed. Uses the same build images and overall strategy that is used in the feedstocks.

Checklist

  • License file is packaged (see here for an example)
  • Source is from official source
  • Package does not vend other packages
  • Build number is 0
  • A tarball (url) rather than a repo (e.g. git_url) is used in your recipe (see here for more details)
  • GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there

@jakirkham jakirkham changed the title Add Linux CUDA builds WIP: Add Linux CUDA builds Oct 12, 2019
@jakirkham
Copy link
Member Author

We still need to update this to identify and use the right variant.

This was referenced Oct 23, 2019
@jaimergp
Copy link
Member

Subscribed! Let me know how can I help.

strategy:
maxParallel: 1
matrix:
linux_cuda-9_2:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jakirkham, I don't understand. This is adding 3 more azure jobs, right?

Copy link
Member Author

@jakirkham jakirkham Oct 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It adds 1 job that sequentially builds each supported CUDA version.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I guess jobs name in azure is slightly different from what is in travis.

Is there a way to skip these builds if there is no cuda needed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do a fan out as in conda-forge/_azure_pipelines_test#3

You can do the first job which is the existing one and set the variable NEED_CUDA to 0 or 1 depending on whether cuda compiler is used or not.

Then the 3 cuda jobs can run in parallel if NEED_CUDA is 1 and if not they are skipped.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I was wondering that same question, but didn't have a good answer at the time. Thanks for investigating.

Fanning out seems reasonable. Guessing this will require an initial parsing of the recipes collected here before building, do you know if there is a good way to hook into the parsing process here to get that information?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can, but if you skip cuda_compiler_version == None, then it wouldn't work. So do the easiest way and check for - {{ compiler('cuda') }} in the recipe meta.yaml. There'll be false positives (if that line is commented), but there won't be false negatives. This can be done in the build_all.py where you can just echo that magic line.

@jakirkham jakirkham mentioned this pull request Nov 1, 2019
6 tasks
@jakirkham jakirkham mentioned this pull request Dec 16, 2019
8 tasks
@stale
Copy link

stale bot commented Mar 21, 2020

Hi friend!

We really, really, really appreciate that you have taken the time to make a PR on conda-forge/staged-recipes! conda-forge only exists because people like you donate their time to build and maintain conda recipes for use by the community.

In an effort to maintain this repository and increase the signal-to-noise for open PRs, the maintainers of staged-recipes close excessively old PRs after six months. This PR will remain open for another month, and then will be closed.

If you'd like to keep it open, please comment/push and we will be happy to oblige! Note that very old PRs will likely need to be rebased on master so that they can be rebuilt with the most recent CI scripts. If you have any trouble, or we missed reviewing this PR in the first place (sorry!), feel free to ping the team using a special command in a comment on the PR to get the attention of the staged-recipes team.

Cheers and thank you for contributing to this community effort!

@stale stale bot added the stale will be closed in 30 days label Mar 21, 2020
@jakirkham jakirkham removed the stale will be closed in 30 days label Mar 21, 2020
@jakirkham jakirkham mentioned this pull request Apr 16, 2020
8 tasks
@stale
Copy link

stale bot commented Aug 18, 2020

Hi friend!

We really, really, really appreciate that you have taken the time to make a PR on conda-forge/staged-recipes! conda-forge only exists because people like you donate their time to build and maintain conda recipes for use by the community.

In an effort to maintain this repository and increase the signal-to-noise for open PRs, the maintainers of staged-recipes close excessively old PRs after six months. This PR will remain open for another month, and then will be closed.

If you'd like to keep it open, please comment/push and we will be happy to oblige! Note that very old PRs will likely need to be rebased on master so that they can be rebuilt with the most recent CI scripts. If you have any trouble, or we missed reviewing this PR in the first place (sorry!), feel free to ping the team using a special command in a comment on the PR to get the attention of the staged-recipes team.

Cheers and thank you for contributing to this community effort!

@stale stale bot added the stale will be closed in 30 days label Aug 18, 2020
@jakirkham jakirkham removed the stale will be closed in 30 days label Aug 18, 2020
@jakirkham jakirkham mentioned this pull request Dec 14, 2020
9 tasks
@isuruf
Copy link
Member

isuruf commented Dec 15, 2020

Done in #13469

@isuruf isuruf closed this Dec 15, 2020
@jakirkham jakirkham deleted the bld_gpu branch March 23, 2021 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Adding CUDA builds
3 participants