-
Notifications
You must be signed in to change notification settings - Fork 578
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adelus_vector_random_MPI_X tests failing in CUDA builds since they were added #8477
Comments
Test results for issue #8477 as of 2020-12-27
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=15
|
Test results for issue #8477 as of 2021-01-10
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=22
|
Test results for issue #8477 as of 2021-01-17
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=22
|
Test results for issue #8477 as of 2021-01-24
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=7
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat. |
@srajama1: Who is the assignee for this issue? |
Test results for issue #8477 as of 2021-01-31
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=22
|
Test results for issue #8477 as of 2021-02-07
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=7
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat. |
Test results for issue #8477 as of 2021-02-14
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=22
|
Test results for issue #8477 as of 2021-02-21
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=10
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat. |
Test results for issue #8477 as of 2021-02-28
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=18
|
Test results for issue #8477 as of 2021-03-07
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=22
|
Test results for issue #8477 as of 2021-03-14
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Missing: twim=3
Tests with issue trackers Failed: twif=19
|
@trilinos/framework, as per #8858 (comment), someone should just disable these failing tests on all platforms where they are failing. Apparently, these tests are not designed to run with the general test harness on these systems (which makes you wonder why they are even being enabled in these cases). |
CC: @jjellio But as shown in this query they do actually pass for the 'ride' builds somehow. (That would explain why these pass in Trilinos PR testing which runs CUDA builds on 'ride'.) Would take just a couple of minutes to add disables for these tests on the various platforms. (In fact, they should be disabled for all cuda builds by default given that they even fail for x86 processors with v100s as shown for the 'sems-rhel7' and 'cee-rhel7' cuda builds.) Related to my epic SEPW-213 |
Test results for issue #8477 as of 2021-03-21
Tests with issue trackers Passed: twip=18 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=18
Tests with issue trackers Failed: twif=19
|
Test results for issue #8477 as of 2021-03-28
Tests with issue trackers Passed: twip=18 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=18
Tests with issue trackers Failed: twif=19
|
- Skip Adelus tests when TPETRA_ASSUME_CUDA_AWARE_MPI is 0. Related to trilinos#8477.
Test results for issue #8477 as of 2021-04-04
Tests with issue trackers Passed: twip=18 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=18
Tests with issue trackers Failed: twif=20
|
Test results for issue #8477 as of 2021-04-11
Tests with issue trackers Passed: twip=18 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=18
Tests with issue trackers Failed: twif=17
|
Test results for issue #8477 as of 2021-04-18
Tests with issue trackers Passed: twip=17 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=17
Tests with issue trackers Failed: twif=10
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat. |
Test results for issue #8477 as of 2021-04-25
Tests with issue trackers Passed: twip=18 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=18
Tests with issue trackers Failed: twif=20
|
Test results for issue #8477 as of 2021-05-02
Tests with issue trackers Passed: twip=18 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=18
Tests with issue trackers Failed: twif=16
|
Test results for issue #8477 as of 2021-05-09
Tests with issue trackers Passed: twip=18 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=18
Tests with issue trackers Failed: twif=20
|
Test results for issue #8477 as of 2021-05-18
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=7
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat. |
Test results for issue #8477 as of 2021-05-23
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=7
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat. |
Test results for issue #8477 as of 2021-05-30
Tests with issue trackers Passed: twip=18 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=18
Tests with issue trackers Missing: twim=3
Tests with issue trackers Failed: twif=17
|
Test results for issue #8477 as of 2021-06-06
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Missing: twim=15
|
Test results for issue #8477 as of 2021-06-13
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Failed: twif=7
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat. |
Test results for issue #8477 as of 2021-06-20
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Missing: twim=15
|
@ccober6 Sorry for my late reply. I will look at this issue this week. |
Test results for issue #8477 as of 2021-06-27
Tests with issue trackers Missing: twim=10 Detailed test results: (click to expand)Tests with issue trackers Missing: twim=10
Tests with issue trackers Failed: twif=7
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat. |
Test results for issue #8477 as of 2021-07-04
Tests with issue trackers Missing: twim=10 Detailed test results: (click to expand)Tests with issue trackers Missing: twim=10
Tests with issue trackers Failed: twif=7
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat. |
Test results for issue #8477 as of 2021-07-11
Tests with issue trackers Missing: twim=10 Detailed test results: (click to expand)Tests with issue trackers Missing: twim=10
Tests with issue trackers Failed: twif=7
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat. |
Test results for issue #8477 as of 2021-07-18
Tests with issue trackers Passed: twip=16 Detailed test results: (click to expand)Tests with issue trackers Passed: twip=16
Tests with issue trackers Missing: twim=15
|
@vqd8a It looks like many of these are passing, but the rest are "missing". I assume that means it has not run. Should we close this and let Grover start a new issue with the missing tests? |
@ccober6 I am not sure what "missing" means. Please close this one and do what you suggest. Thanks. |
Closing this issue as it appears many test are passing and the remaining are now "missing" (not sure what that means ). Will pick up new issue on the remaining tests. |
CC: @srajama1 (Trilinos Linear Solvers Product Lead), @e10harvey
Next Action Status
Description
As shown in this query the tests:
Adelus_vector_random_MPI_1
Adelus_vector_random_MPI_2
Adelus_vector_random_MPI_3
Adelus_vector_random_MPI_4
in the builds:
Trilinos-atdm-ats2-cuda-10.1.243-gnu-7.3.1-spmpi-rolling_complex_static_opt
Trilinos-atdm-ats2-cuda-10.1.243-gnu-7.3.1-spmpi-rolling_complex_static_opt_cuda-aware-mpi
Trilinos-atdm-ats2-cuda-10.1.243-gnu-7.3.1-spmpi-rolling_static_dbg
Trilinos-atdm-ats2-cuda-10.1.243-gnu-7.3.1-spmpi-rolling_static_dbg_cuda-aware-mpi
Trilinos-atdm-ats2-cuda-10.1.243-gnu-7.3.1-spmpi-rolling_static_opt
Trilinos-atdm-ats2-cuda-10.1.243-gnu-7.3.1-spmpi-rolling_static_opt_cuda-aware-mpi
Trilinos-atdm-cee-rhel6_cuda-10.1.243_gcc-7.2.0_openmpi-4.0.3_shared_dbg
Trilinos-atdm-cee-rhel6_cuda-10.1.243_gcc-7.2.0_openmpi-4.0.3_shared_opt
Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-shared-release-debug
Trilinos-atdm-white-ride-cuda-9.2-gnu-7.2.0-debug
Trilinos-atdm-white-ride-cuda-9.2-gnu-7.2.0-rdc-release-debug
Trilinos-atdm-white-ride-cuda-9.2-gnu-7.2.0-release
Trilinos-atdm-white-ride-cuda-9.2-gnu-7.2.0-release-debug
started failing on testing day 2020-08-18.
These look to all be failing with segfaults like:
and
These tests were failing when the Adelus package was first added to ATDM Trilinos testing.
Current Status on CDash
Run the above query adjusting the "Begin" and "End" dates to match today any other data range.
Steps to Reproduce
One should be able to reproduce this failure as described in:
and the system-specific instructions at:
Just log into any of the associated machines and copy and paste the full CDash build name
<build-name>
listed above and run commands like:where
<package-name>
is any package that you want to enabled to reproduce build and/or test results.Again, for exact system-specific details on what commands to run to build and run tests, see:
And if you can't figure out what commands to run to produce the issue given the above-referenced documentation, please post a comment here and we will give you the exact minimal commands to reproduce the failures.
The text was updated successfully, but these errors were encountered: