Framework: Trilinos Installation Testing

[william76]

@trilinos/framework

This is a top-level tracking issue for Trilinos installation testing. This topic has been posted a few times before but it keeps getting down prioritized... but we also haven't really ever broken the overall task into sub-tasks that may be easier to work on and track.

Tasks

This is an initial broad-brush list of tasks that I think we probably need for this. It will surely evolve over time.

• Add installation testing support to CMake
• Modify PR driver scripts
  • Enable "Install Trilinos" sequence.
  • Enable "Test Installed Trilinos" sequence.
• Develop Jenkins jobs to Perform Installation Tests
• Verify Proper CDash Reporting
• Add Packages to Installation Testing

Details

Add Installation Testing Support to CMake

Most of the infrastructure should already be in place. See https://github.com/TriBITSPub/TriBITS/tree/5027-enable-installation-testing for more information on this. With luck, we just need to verify that installation testing works w/in TriBiTS.

We may need to re-learn what we need to do to use this and there may be some modifications necessary in the CMakeLists.txt files in Trilinos packages to make everything work properly.

Modify PR Driver Scripts

There are two sequences that we will generally need to support:

Install Trilinos

1. Configure without tests
2. Build Trilinos
3. Install Trilinos

Test Installed Trilinos

1. Configure with tests against installed Trilinos
2. Build Tests
3. Run Tests

Develop Jenkins jobs to Perform Installation Tests

We'll need to create the necessary Jenkins infrastructure to run these test
sequences. Without getting too far down into the weeds, I see two main ways we might implement this on our Jenkins server...

Single Jenkins Job w/ 2 Stages

This may be the simplest one to do from a Jenkins perspective since everything
is in a single self-contained Jenkins job.

Two Jenkins Jobs, triggered in sequence

This will be more complicated because:

1. We can't guarantee that two jobs would run on the same node in the build farm using a trilinos-any tag.
2. Even if the jobs are on the same node we can't necessarily guarantee the test job would share the same workspace.

This approach could offer some additional optimizations that we don't currently have in our testing though since notionally we should need fewer cores to run the tests than we do to build Trilinos. It may be easier to schedule the test phase than it is for the build phase.

Passing the 'installed' Trilinos from one job to another may be tricky. I think we can do it via 'artifacts' in Jenkins, but I haven't tested it personally yet.

Verify Proper CDash Reporting

CDash may track these properly but we'll need to verify once we have Jenkins jobs working that results get reported to CDash in some reasonable manner that's traceable.

Add Packages to Installation Testing

This is probably the big task once all the infrastructure is in place. We will initially start with just one package to build out the infrastructutre. I would recommend using Teuchos for this.

Once the infrastructure is in place, we'll need to pull in packages one by one and make sure they are doing the right thing in each phase:

1. Package must install all the required files, headers, etc.
2. Package should not install its tests during the install phase
   • Unless tests are enabled too?
3. Package should use the installed library and headers, but the tests should come from the test directory(s) in the source when running the test phase against an installed Trilinos.

Related Issues

The following issues look related to this topic. We may be able to close out some of them if they're superseded by this. Linking to them from here so they can be easily found (and closed when we get installation testing up and running).

• Please consider moving all installed headers into include/Trilinos #2102 - Please consider moving all installed headers into include/Trilinos - Semi-related but we should consider it if we're digging into Trilinos installation stuff. This issue is ancient and I closed it recently but we can always reopen if we need to. It might be outdated or irrelevant depending on how the installation prefix is set up.
• Add install hooks for ATDM Trilinos configuration and ctest -S driver #2689 - Add install hooks for ATDM Trilinos configuration and ctest -S driver - An old ticket but we should look at what's been done here in the TriBiTS to support Trilinos Installation. ATDM will need to know if we change anything that might affect them.
• Trilinos Framework: Enable testing against installation builds #2812 - Trilinos Framework: Enable testing against installation builds - Issue requesting this same thing w/out planning details.
• Framework: Installation Testing #5027 - Framework: Installation Testing - Good information here on installation testing. Does not specify any tasks. Framework: Installation Testing #5027 references the following issues:
  • Xpetra broke Albany and SPARC nightly builds  #5011
  • Xpetra fatal error due to missing headers, similar to #5011 #5098
  • TriBITS snapshot on 6/7/2019 to bring in Trilinos_ENABLE_INSTALLATION_TESTING documentation (#5097) #5352
  • TrilinosBuildReference.html no longer getting rebuilt automatically? #5357
  • Do 'make install' as part of Trilinos PR testing #6528

FYI @jwillenbring

[bartlettroscoe]

With luck, we just need to verify that installation testing works w/in TriBiTS.

@william76, it is verified. There is an automated TriBITS test TriBITS_TribitsExampleProject_ALL_ST_NoFortran_enable_installation_testing showing that it works. See here.

[bartlettroscoe]

Related to:

• SEPW-213
• SEPW-214

[wadeburgess]

Migrated to Jira, so closing this issue in GitHub.