Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TeuchosNumerics_LAPACK_test_MPI_1 failing in build Trilinos-atdm-waterman-cuda-9.2-debug starting 2019-10-21 #6166

Closed
bartlettroscoe opened this issue Oct 25, 2019 · 3 comments
Closed

TeuchosNumerics_LAPACK_test_MPI_1 failing in build Trilinos-atdm-waterman-cuda-9.2-debug starting 2019-10-21 #6166

bartlettroscoe opened this issue Oct 25, 2019 · 3 comments
Labels
ATDM DevOps Issues that will be worked by the Coordinated ATDM DevOps teams ATDM Sev: Nonblocker Problems with Trilinos that should not block ATDM APPs from getting updates client: ATDM Any issue primarily impacting the ATDM project PA: Framework Issues that fall under the Trilinos Framework Product Area pkg: Teuchos Issues primarily dealing with the Teuchos Package stage: in review Primary work is completed and now is just waiting for human review and/or test feedback type: bug The primary issue is a bug in Trilinos code or tests
Milestone
Keep promoted "AT...

Comments

@bartlettroscoe
Copy link
Member

bartlettroscoe commented Oct 25, 2019 •
edited
Loading

As shown in this query the test:

  • TeuchosNumerics_LAPACK_test_MPI_1

started failing in the build:

  • Trilinos-atdm-waterman-cuda-9.2-debug

staring 2019-10-21.

Looking at the commits pulled taht day here it seems likely this was a result of removing the disable for the unit test for STEQR that was disabled as part of #2410 in PR #4064. Looks like the PR #6126 accidentally removed the setting of TeuchosNumerics_DISABLE_STEQR_TEST=ON.
@bartlettroscoe bartlettroscoe added type: bug The primary issue is a bug in Trilinos code or tests pkg: Teuchos Issues primarily dealing with the Teuchos Package client: ATDM Any issue primarily impacting the ATDM project ATDM DevOps Issues that will be worked by the Coordinated ATDM DevOps teams ATDM Sev: Nonblocker Problems with Trilinos that should not block ATDM APPs from getting updates PA: Framework Issues that fall under the Trilinos Framework Product Area labels Oct 25, 2019
@bartlettroscoe
Copy link
Member Author

We can see in the Trilinos-atdm-waterman-cuda-9.2-debug build configure output on 2019-10-20::

-- Setting default TeuchosNumerics_DISABLE_STEQR_TEST=ON

but on 2019-10-21 the output did not show this.

So I just need to add this back.

bartlettroscoe added a commit that referenced this issue Oct 25, 2019
@bartlettroscoe
Add back TeuchosNumerics_DISABLE_STEQR_TEST=ON (#2410, #6166)
e6843b4 
This got removed by accident as part of an ealier refactoring to a single
Tweaks.cmake file.  This should allow this test to pass now (without this
single unit test).
@bartlettroscoe bartlettroscoe mentioned this issue Oct 25, 2019
Merged
bartlettroscoe added a commit that referenced this issue Oct 25, 2019
@bartlettroscoe
Merge branch '6166-add-back-lapack-steqr-test-disable' into atdm-nigh…
f3488b1 
…tly-manual-updates (#6166, #6167)
trilinos-autotester added a commit that referenced this issue Oct 25, 2019
@trilinos-autotester
Merge Pull Request #6167 from trilinos/Trilinos/6166-add-back-lapack-…
69fedd6 
…steqr-test-disable

Automatically Merged using Trilinos Pull Request AutoTester
PR Title: Add back TeuchosNumerics_DISABLE_STEQR_TEST=ON (#2410, #6166)
PR Author: bartlettroscoe
@bartlettroscoe bartlettroscoe added the stage: in review Primary work is completed and now is just waiting for human review and/or test feedback label Oct 25, 2019
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Oct 26, 2019
Merge 'trilinos/Trilinos:develop' (19e4fb2) into 'tcad-charon/Trilino…
44653bf 
…s:develop' (19158f2).

* trilinos-develop:
  Add back TeuchosNumerics_DISABLE_STEQR_TEST=ON (trilinos#2410, trilinos#6166)
  MueLu: fixed build error
  kokkos-kernels: update gcc check for c++14 workaround macro
  Ifpack2 ScaledDampedResidual: Cache vectors
  Tpetra/MueLu: switched performance tests to StackedTimer
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Oct 27, 2019
Merge 'trilinos/Trilinos:develop' (19e4fb2) into 'tcad-charon/Trilino…
e3f515c 
…s:develop' (19158f2).

* trilinos-develop:
  Add back TeuchosNumerics_DISABLE_STEQR_TEST=ON (trilinos#2410, trilinos#6166)
  MueLu: fixed build error
  kokkos-kernels: update gcc check for c++14 workaround macro
  Ifpack2 ScaledDampedResidual: Cache vectors
  Tpetra/MueLu: switched performance tests to StackedTimer
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Oct 27, 2019
Merge 'trilinos/Trilinos:develop' (19e4fb2) into 'tcad-charon/Trilino…
1281148 
…s:develop' (19158f2).

* trilinos-develop:
  Add back TeuchosNumerics_DISABLE_STEQR_TEST=ON (trilinos#2410, trilinos#6166)
  MueLu: fixed build error
  kokkos-kernels: update gcc check for c++14 workaround macro
  Ifpack2 ScaledDampedResidual: Cache vectors
  Tpetra/MueLu: switched performance tests to StackedTimer
@bartlettroscoe
Copy link
Member Author

After the merge of PR #6167 on 2019-10-25, the test is now passing starting 2019-10-26 as shown in this query.

I will close this issue as complete. The original issue #2410 for the bad STEQR LAPACK function will be left open.

@bartlettroscoe
Copy link
Member Author

NOTE: As shown in this query, this same unit test is crashing on the builds

Site Build Name Test Name Status Time Proc Time Details Build Time Processors
vortex Trilinos-atdm-ats2-gnu-7.3.1-spmpi-2019.06.24_serial_static_dbg TeuchosNumerics_LAPACK_test_MPI_1 Failed 2s 380ms 2s 380ms Completed (Failed) 2020-02-03T02:10:05 MST 1
vortex Trilinos-atdm-ats2-cuda-10.1.243-gnu-7.3.1-spmpi-2019.06.24_static_dbg TeuchosNumerics_LAPACK_test_MPI_1 Failed 3s 410ms 3s 410ms Completed (Failed) 2020-02-03T02:07:05 MST 1

I will disable them there too.

bartlettroscoe added a commit to bartlettroscoe/Trilinos that referenced this issue Feb 3, 2020
@bartlettroscoe
Disable STEQR() LAPACK test on ats2 deug builds (trilinos#2410, trili…
95c41cd 
…nos#6166)
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Feb 4, 2020
Merge 'trilinos/Trilinos:develop' (44f13cb) into 'tcad-charon/Trilino…
9e2a818 
…s:develop' (2bfd2c7).

* trilinos-develop: (177 commits)
  Add a fix for a stk cmake file
  Promote atdm ats2 gnu+dbg and cuda+gnu+dbg to 'Specialized' (CDOFA-72)
  Intrepid2: remove unnecessary finalize calls in unit tests
  Disable STEQR() LAPACK test on ats2 deug builds (trilinos#2410, trilinos#6166)
  Disable some timing out ROL tests (trilinos#6124)
  Disable timing out Tempus tests on ats2 (trilinos#6009)
  fixed some broken teuchos unit tests and removed missed deprecated methods
  Promoting ats2+gnu+opt build which is 100% clean (CDOFA-27)
  removed deprecated overload of << in SerialDenseMatrix, SerialBandDenseMatrix, SerialSymDenseMatrix, and SerialDenseVector
  removed deprecated Teuchos::Comm helpers reduceAll and scan that take pointers to return arguments
  removed deprecated MPITraits class
  removed deprecated ArrayArg class
  removed deprecated LAPACK::GEBAL method that takes ilo and ihi by value
  removed deprecated LAPACK::POSVX and LAPACK::GESVX methods that take EQUED by value
  removed deprecated LAPACK::TREXC method that takes ifst and ilst by value
  removed deprecated count method in ArrayRCP, RCP, and RCPNode
  removed deprecated PerformanceMonitorBase::clearTimer methods
  Intrepid2: Temporarily disabling tests failing on some machines (Issue trilinos#6246)
  Remove misspelled RTop_HIDE_DEPRECATED_CODE (trilinos#6217)
  Disable/hide deprecated code (trilinos#6217)
  ...
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Feb 5, 2020
Merge 'trilinos/Trilinos:develop' (aa11e22) into 'tcad-charon/Trilino…
0d85a8c 
…s:develop' (2bfd2c7).

* trilinos-develop: (186 commits)
  zoltan2:  upgrading testing for issues fixed in trilinos#6375
  tpetra: disable kokkos warnings in initialize tests
  Tacho - disable matrix market reader/writer test to improve PR test stability.
  kokkos: cmake fixes for clang +/- cuda
  kokkos/cmake/kokkos_arch.cmake: Fix for clang + NO cuda
  Fix some scopes in nlnml_nonlinearlevel.cpp
  Zoltan2: fix reversal of Cuthill McKee ordering
  Add a fix for a stk cmake file
  Promote atdm ats2 gnu+dbg and cuda+gnu+dbg to 'Specialized' (CDOFA-72)
  Intrepid2: remove unnecessary finalize calls in unit tests
  Disable STEQR() LAPACK test on ats2 deug builds (trilinos#2410, trilinos#6166)
  Disable some timing out ROL tests (trilinos#6124)
  Disable timing out Tempus tests on ats2 (trilinos#6009)
  Intrepid2: reenabling JacobiLegendrePolynomial_Tests and Hierarchical_Basis_Tests.
  fixed some broken teuchos unit tests and removed missed deprecated methods
  Promoting ats2+gnu+opt build which is 100% clean (CDOFA-27)
  removed deprecated overload of << in SerialDenseMatrix, SerialBandDenseMatrix, SerialSymDenseMatrix, and SerialDenseVector
  removed deprecated Teuchos::Comm helpers reduceAll and scan that take pointers to return arguments
  removed deprecated MPITraits class
  removed deprecated ArrayArg class
  ...
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Feb 5, 2020
Merge 'trilinos/Trilinos:develop' (aa11e22) into 'tcad-charon/Trilino…
c0afcc6 
…s:develop' (2bfd2c7).

* trilinos-develop: (186 commits)
  zoltan2:  upgrading testing for issues fixed in trilinos#6375
  tpetra: disable kokkos warnings in initialize tests
  Tacho - disable matrix market reader/writer test to improve PR test stability.
  kokkos: cmake fixes for clang +/- cuda
  kokkos/cmake/kokkos_arch.cmake: Fix for clang + NO cuda
  Fix some scopes in nlnml_nonlinearlevel.cpp
  Zoltan2: fix reversal of Cuthill McKee ordering
  Add a fix for a stk cmake file
  Promote atdm ats2 gnu+dbg and cuda+gnu+dbg to 'Specialized' (CDOFA-72)
  Intrepid2: remove unnecessary finalize calls in unit tests
  Disable STEQR() LAPACK test on ats2 deug builds (trilinos#2410, trilinos#6166)
  Disable some timing out ROL tests (trilinos#6124)
  Disable timing out Tempus tests on ats2 (trilinos#6009)
  Intrepid2: reenabling JacobiLegendrePolynomial_Tests and Hierarchical_Basis_Tests.
  fixed some broken teuchos unit tests and removed missed deprecated methods
  Promoting ats2+gnu+opt build which is 100% clean (CDOFA-27)
  removed deprecated overload of << in SerialDenseMatrix, SerialBandDenseMatrix, SerialSymDenseMatrix, and SerialDenseVector
  removed deprecated Teuchos::Comm helpers reduceAll and scan that take pointers to return arguments
  removed deprecated MPITraits class
  removed deprecated ArrayArg class
  ...
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Feb 5, 2020
Merge 'trilinos/Trilinos:develop' (aa11e22) into 'tcad-charon/Trilino…
024431d 
…s:develop' (2bfd2c7).

* trilinos-develop: (186 commits)
  zoltan2:  upgrading testing for issues fixed in trilinos#6375
  tpetra: disable kokkos warnings in initialize tests
  Tacho - disable matrix market reader/writer test to improve PR test stability.
  kokkos: cmake fixes for clang +/- cuda
  kokkos/cmake/kokkos_arch.cmake: Fix for clang + NO cuda
  Fix some scopes in nlnml_nonlinearlevel.cpp
  Zoltan2: fix reversal of Cuthill McKee ordering
  Add a fix for a stk cmake file
  Promote atdm ats2 gnu+dbg and cuda+gnu+dbg to 'Specialized' (CDOFA-72)
  Intrepid2: remove unnecessary finalize calls in unit tests
  Disable STEQR() LAPACK test on ats2 deug builds (trilinos#2410, trilinos#6166)
  Disable some timing out ROL tests (trilinos#6124)
  Disable timing out Tempus tests on ats2 (trilinos#6009)
  Intrepid2: reenabling JacobiLegendrePolynomial_Tests and Hierarchical_Basis_Tests.
  fixed some broken teuchos unit tests and removed missed deprecated methods
  Promoting ats2+gnu+opt build which is 100% clean (CDOFA-27)
  removed deprecated overload of << in SerialDenseMatrix, SerialBandDenseMatrix, SerialSymDenseMatrix, and SerialDenseVector
  removed deprecated Teuchos::Comm helpers reduceAll and scan that take pointers to return arguments
  removed deprecated MPITraits class
  removed deprecated ArrayArg class
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ATDM DevOps Issues that will be worked by the Coordinated ATDM DevOps teams ATDM Sev: Nonblocker Problems with Trilinos that should not block ATDM APPs from getting updates client: ATDM Any issue primarily impacting the ATDM project PA: Framework Issues that fall under the Trilinos Framework Product Area pkg: Teuchos Issues primarily dealing with the Teuchos Package stage: in review Primary work is completed and now is just waiting for human review and/or test feedback type: bug The primary issue is a bug in Trilinos code or tests
Projects
None yet
Milestone
Keep promoted "ATDM" builds of Trilin...
Development

No branches or pull requests
1 participant
@bartlettroscoe