Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PR autotesting reports time outs #9818

Closed
jhux2 opened this issue Oct 15, 2021 · 4 comments
Closed

PR autotesting reports time outs #9818

jhux2 opened this issue Oct 15, 2021 · 4 comments
Labels
autotester Issues related to the autotester. PA: Framework Issues that fall under the Trilinos Framework Product Area type: bug The primary issue is a bug in Trilinos code or tests

Comments

@jhux2
Copy link
Member

jhux2 commented Oct 15, 2021

Bug Report

@trilinos/framework

Description

In at least some PR's, the tests are failing due to timeouts.

Screen Shot 2021-10-15 at 8 39 12 AM

For example
#9811
#9683
@jhux2 jhux2 added type: bug The primary issue is a bug in Trilinos code or tests PA: Framework Issues that fall under the Trilinos Framework Product Area autotester Issues related to the autotester. labels Oct 15, 2021
@e10harvey
Copy link
Contributor

This is due to the cuda jobs targeting a machine with a full job queue. A ticket has been opened.

@jhux2
Copy link
Member Author

jhux2 commented Oct 15, 2021

@e10harvey Thanks for the update!

@jwillenbring
Copy link
Member

Jobs were not timing out properly. A bunch of hanging jobs were filing the queue. It is possible there is more going on here, but for the time being I have emptied hanging jobs out of the queue.

@jhux2
Copy link
Member Author

jhux2 commented Oct 22, 2021

@jwillenbring Thanks for your help, PRs are going through now. There is still a recurring error message that reports a timeout. It seems that resubmitting enough times gets around the error.

See, for example, #9849 (comment).

https://github.com/trilinos/Trilinos/pulls?q=is%3Apr+is%3Aopen+%22Timed+out+waiting+for+job%22+in%3Adescription+.

@jhux2 jhux2 closed this as completed May 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
autotester Issues related to the autotester. PA: Framework Issues that fall under the Trilinos Framework Product Area type: bug The primary issue is a bug in Trilinos code or tests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jwillenbring @jhux2 @e10harvey