Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update test-linux-gpu.yml #7019

Closed
wants to merge 3 commits into from
Closed

Update test-linux-gpu.yml #7019

wants to merge 3 commits into from

Conversation

vfdev-5
Copy link
Collaborator

@vfdev-5 vfdev-5 commented Dec 8, 2022

Fixes CUDA drivers issue with test-linux-gpu job

@vfdev-5 vfdev-5 requested a review from osalpekar December 8, 2022 13:02
@vfdev-5
Copy link
Collaborator Author

vfdev-5 commented Dec 8, 2022 •
edited
Loading

@osalpekar do you know why the following error can happen : https://github.com/pytorch/vision/actions/runs/3648497295/attempts/1 ?
image

@pmeier
Copy link
Collaborator

pmeier commented Dec 8, 2022

There are some OOM errors for the model tests: https://github.com/pytorch/vision/actions/runs/3648584911/jobs/6162177024#step:9:35363

@pmeier pmeier mentioned this pull request Dec 8, 2022
Closed
@pmeier
Copy link
Collaborator

pmeier commented Dec 8, 2022

It seems on linux.4xlarge.nvidia.gpu we are getting OOM errors, while on linux.g5.4xlarge.nvidia.gpu the driver is not set up properly.

@osalpekar
Copy link
Member

@osalpekar do you know why the following error can happen : https://github.com/pytorch/vision/actions/runs/3648497295/attempts/1 ? image

cc @seemethere or @atalman Looks like cleaning the workspace in the generic job fails due to permissions issues with certain directories. @lequytra also saw this while migrating CI jobs in torchrec: https://github.com/pytorch/torchrec/actions/runs/3651587724/jobs/6168971846.

@osalpekar osalpekar mentioned this pull request Dec 9, 2022
Closed
@vfdev-5 vfdev-5 closed this Jan 20, 2023
@vfdev-5 vfdev-5 deleted the fix-test-linux-gpu branch January 20, 2023 11:38
@vfdev-5
Copy link
Collaborator Author

vfdev-5 commented Jan 20, 2023

Outdated PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pmeier pmeier pmeier left review comments

@osalpekar osalpekar Awaiting requested review from osalpekar

Assignees
No one assigned
Labels
cla signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@vfdev-5 @pmeier @osalpekar @facebook-github-bot