How to enable all downstream packages in nightly dashboard testing

[jhux2]

The Tpetra team has a specialized build that is tests Trilinos with deprecated code in Tpetra disabled. This build enables just Tpetra and its tests. I would like to create a similar build that enables all downstream packages+tests/examples that depend on Tpetra. Does the Tribits/cmake testing infrastructure have some way of doing this in an automatic fashion, e.g., "-DTrilinos_ENABLE_ALL_DOWNSTREAM_PACKAGES"?

@trilinos/framework @bartlettroscoe

[rppawlo]

https://tribits.org/doc/TribitsBuildReference.html#enable-to-test-all-effects-of-changing-a-given-package-s

[jhux2]

@rppawlo Thanks, Roger!

[jhux2]

The options given in the tribits link result in a test that has no tests run. Here are the results. Here is the relevant part of the Tpetra driver script. @trilinos/framework Any other ideas?

INCLUDE("${CTEST_SCRIPT_DIRECTORY}/TrilinosCTestDriverCore.rocketman.gcc.cmake")

#
# Set the options specific to this build case
#

# The variable BUILD_DIR_NAME is based COMM_TYPE, BUILD_TYPE, and BUILD_NAME_DETAILS.
# Tribits creates the variable listed under "Build Name" by prepending the OS type and compiler
# details to BUILD_DIR_NAME.
SET(COMM_TYPE MPI)
SET(BUILD_TYPE RELEASE)
SET(BUILD_NAME_DETAILS TPETRA_DEPRECATED_CODE_OFF_ENABLE_DOWNSTREAM)

SET(CTEST_PARALLEL_LEVEL 8)
SET(CTEST_TEST_TYPE Experimental)
SET(Trilinos_TRACK  Experimental)  # Set the CDash track to Nightly
SET(CTEST_TEST_TIMEOUT 14400) # twice the default value, for valgrind
SET(CTEST_DO_MEMORY_TESTING FALSE)

SET(Trilinos_PACKAGES Tpetra)

SET(EXTRA_CONFIGURE_OPTIONS
  "-DTpetra_ENABLE_DEPRECATED_CODE=OFF"
  "-DTrilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES=ON"
  "-DTrilinos_ENABLE_TESTS=ON"
  "-DKOKKOS_ENABLE_DEPRECATED_CODE=OFF"
  "-DTrilinos_ENABLE_DEPENDENCY_UNIT_TESTS=OFF"
  "-DTrilinos_ENABLE_Epetra:BOOL=OFF"
)

#
# Set the rest of the system-specific options and run the dashboard build/test
#

TRILINOS_SYSTEM_SPECIFIC_CTEST_DRIVER()

[jhux2]

I tried with the option -DTrilinos_ENABLE_DEPENDENCY_UNIT_TESTS=ON, and got the same results.

[jhux2]

I got things working. It turns out I was only looking at the Tpetra subpackage results. When I looked at the results across the entire Trilinos dashboard, I noticed that the Kokkos build was failing. In summary, the options to enable forward packages are

  "-DTrilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES=ON"
  "-DTrilinos_ENABLE_TESTS=ON"

[jhux2]

I'm explicitly disabling Epetra, yet it is getting reenabled, I suspect by the forward package enable option. @bartlettroscoe @jwillenbring @william76 any ideas why?

[rppawlo]

@jhux2 panzer requires epetra - it currently requires both the type I and type II stacks. So if you enable all downstream, it may be forcing this requirement.

[tjfulle]

@jhux2 is this still a problem? In PR #5756 Epetra was made an explicit dependency of Panzer so Panzer should be turned off if Epetra is off, even with downstream dependencies enabled. Have you tried setting the options directly, ie

SET(Trilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES ON CACHE BOOL "")
SET(Trilinos_ENABLE_Epetra OFF CACHE BOOL "")

instead of through SET(EXTRA_CONFIGURE_OPTIONS ...)?

[bartlettroscoe]

This always confuses people but you need to set these in the outer ctest -S program. See:

• https://tribits.org/doc/TribitsDevelopersGuide.html#determining-what-packages-get-tested-tribits-ctest-driver

You can set those as CTest vars or as env vars.

[jhux2]

@bartlettroscoe The documentation you pointed to is helpful, thanks. I just tested the driver below, which gives these results: https://testing.sandia.gov/cdash/index.php?project=Trilinos&parentid=5524772.

There are some errors that the Tpetra team will need to work through.

SET(COMM_TYPE MPI)
SET(BUILD_TYPE RELEASE)
SET(BUILD_NAME_DETAILS TPETRA_DEPRECATED_CODE_OFF_ENABLE_DOWNSTREAM)
SET(CTEST_BUILD_FLAGS     "-j20 -i" )

SET(CTEST_PARALLEL_LEVEL 8)
SET(CTEST_TEST_TYPE Experimental)
SET(Trilinos_TRACK  Experimental)  # Set the CDash track to Nightly
SET(CTEST_TEST_TIMEOUT 14400) # twice the default value, for valgrind
SET(CTEST_DO_MEMORY_TESTING FALSE)

SET(Trilinos_PACKAGES Tpetra)
SET(Trilinos_EXCLUDE_PACKAGES Epetra)

SET(EXTRA_CONFIGURE_OPTIONS
  "-DTpetra_ENABLE_DEPRECATED_CODE=OFF"
  "-DTrilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES=ON"
  "-DTrilinos_ENABLE_TESTS=ON"
  "-DKOKKOS_ENABLE_DEPRECATED_CODE=OFF"
  "-DTrilinos_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=ON"
  "-DTPL_ENABLE_Matio=OFF"
  "-DTPL_ENABLE_X11=OFF"
  "-DTrilinos_ENABLE_Epetra:BOOL=OFF"
  "-DTrilinos_ENABLE_Moertel:BOOL=OFF"
  "-DTrilinos_ENABLE_PyTrilinos:BOOL=OFF"
  "-DTrilinos_ENABLE_Domi:BOOL=OFF"
  "-DTrilinos_ENABLE_GlobiPack:BOOL=OFF"
  "-DTrilinos_ENABLE_OptiPack:BOOL=OFF"
  "-DKokkos_ENABLE_TESTS:BOOL=OFF"
  "-DTeuchos_ENABLE_TESTS:BOOL=OFF"
  "-DKokkosKernels_ENABLE_TESTS:BOOL=OFF"
)

#
# Set the rest of the system-specific options and run the dashboard build/test
#

TRILINOS_SYSTEM_SPECIFIC_CTEST_DRIVER()

[bartlettroscoe]

@jhux2, You should move Trilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES and Trilinos_DISABLE_ENABLED_FORWARD_DEP_PACKAGES outside of EXTRA_CONFIGURE_OPTIONS as so it impacts the outer ctest -S script as shown below:

SET(COMM_TYPE MPI)
SET(BUILD_TYPE RELEASE)
SET(BUILD_NAME_DETAILS TPETRA_DEPRECATED_CODE_OFF_ENABLE_DOWNSTREAM)
SET(CTEST_BUILD_FLAGS     "-j20 -i" )

SET(CTEST_PARALLEL_LEVEL 8)
SET(CTEST_TEST_TYPE Experimental)
SET(Trilinos_TRACK  Experimental)  # Set the CDash track to Nightly
SET(CTEST_TEST_TIMEOUT 14400) # twice the default value, for valgrind
SET(CTEST_DO_MEMORY_TESTING FALSE)

SET(Trilinos_PACKAGES Tpetra)
SET(Trilinos_EXCLUDE_PACKAGES Epetra)
SET(Trilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES ON) SET(Trilinos_DISABLE_ENABLED_FORWARD_DEP_PACKAGES ON)

SET(EXTRA_CONFIGURE_OPTIONS
  "-DTpetra_ENABLE_DEPRECATED_CODE=OFF"
  "-DTrilinos_ENABLE_TESTS=ON"
  "-DKOKKOS_ENABLE_DEPRECATED_CODE=OFF"
  "-DTPL_ENABLE_Matio=OFF"
  "-DTPL_ENABLE_X11=OFF"
  "-DTrilinos_ENABLE_Epetra:BOOL=OFF"
  "-DTrilinos_ENABLE_Moertel:BOOL=OFF"
  "-DTrilinos_ENABLE_PyTrilinos:BOOL=OFF"
  "-DTrilinos_ENABLE_Domi:BOOL=OFF"
  "-DTrilinos_ENABLE_GlobiPack:BOOL=OFF"
  "-DTrilinos_ENABLE_OptiPack:BOOL=OFF"
  "-DKokkos_ENABLE_TESTS:BOOL=OFF"
  "-DTeuchos_ENABLE_TESTS:BOOL=OFF"
  "-DKokkosKernels_ENABLE_TESTS:BOOL=OFF"
)

#
# Set the rest of the system-specific options and run the dashboard build/test
#

TRILINOS_SYSTEM_SPECIFIC_CTEST_DRIVER()

See:

• https://tribits.org/doc/TribitsDevelopersGuide.html#setting-variables-tribits-ctest-driver
• https://tribits.org/doc/TribitsDevelopersGuide.html#determining-what-packages-get-tested-tribits-ctest-driver
• https://tribits.org/doc/TribitsDevelopersGuide.html#setting-variables-in-the-inner-cmake-configure-tribits-ctest-driver

[jhux2]

@bartlettroscoe Ok, I've made those changes. Looking at the build errors more carefully in the link above, I see that the errors are all in ROL, ShyLU, and TrilinosCouplings examples and/or tests. I was expecting to see tests for other packages that depend on Tpetra, e.g., Belos, Ifpack2 and MueLu. Is there a reason that I don't see any evidence that tests from those packages even built?

[jhux2]

@bartlettroscoe Could you have a look at #5796?

[bartlettroscoe]

@jhux2 said:

I was expecting to see tests for other packages that depend on Tpetra, e.g., Belos, Ifpack2 and MueLu. Is there a reason that I don't see any evidence that tests from those packages even built?

That is not good. Note the post-post CI server running shown here is set up to enable all downstream packages.

We are set up to meet tomorrow but if you could attach the STDOUT from the ctest -S driver script, I should be able to see the logic for what is happening.

[jhux2]

@bartlettroscoe Thanks for your help! He walked me through the relevant parts of the nightly ctest scripts. The downstream testing is now running properly as can be seen here, so I will close this issue.