Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to new devpack/20180521/openmpi/2.1.2/gcc/7.2.0/cuda/9.2.88 env on 'white' and 'ride' (#3290) #3549

Conversation

bartlettroscoe
Copy link
Member

CC: @trilinos/seacas, @fryeguy52

Description

This changes to use a new consistent GCC 7.2.0 + OpenMPI 2.1.2 + CUDA 9.2 + TPLs env on 'white' and 'ride'. Before this, on 'white' we were using OpenMPI 2.1.2 to compile and link libraries and execuables in Trilinos but were linking against TPLs built with OpenMPI 3.1.0 (see #3290). This new set of modules uses OpenMPI 2.1.2 for the TPLs as well to create a 100% consistent env.

@nmhamster noted that he had to reduce the compiler optimization level of HDF5 from -O3 to -O2 or you get a could SEACAS tests that fail, as described in #3288. (So the problem in #3288 was not the NetCDF configuration but was the HDF5 compiler options).

Motivation and Context

It was not good to be using a mixture of OpenMPI 2.1.2 and OpenMPI 3.1.0 on 'white'. And we need to use OpenMPI 2.1.2 to avoid problems with MPI handling of CUDA memory impacting Ifpack2 (see #3290). Also, we need the TPLs built so that the SEACAS tests pass as described above and in #3288.

How Has This Been Tested?

I tested this on 'white' by running:

$ cd ~/Trilinos.base/BUILD/WHITE/CHECKIN/

$ bsub -x -I -q rhel7F -n 16 \
  ./checkin-test-atdm.sh gnu-opt-openmp-Power8 cuda-9.2-opt-Power8-Kepler37 \
  --enable-all-packages=on --local-do-all

That gave the test results ...

gnu-opt-openmp-Power8:

  99% tests passed, 4 tests failed out of 2259
  
  Subproject Time Summary:
  Amesos                    =  34.52 sec*proc (13 tests)
  Amesos2                   =  16.71 sec*proc (8 tests)
  Anasazi                   = 215.21 sec*proc (74 tests)
  AztecOO                   =  32.36 sec*proc (17 tests)
  Belos                     = 247.43 sec*proc (83 tests)
  Domi                      = 199.80 sec*proc (125 tests)
  Epetra                    = 104.40 sec*proc (63 tests)
  EpetraExt                 =  31.64 sec*proc (10 tests)
  Galeri                    =   8.90 sec*proc (9 tests)
  Ifpack                    = 117.91 sec*proc (48 tests)
  Ifpack2                   =  76.19 sec*proc (36 tests)
  Intrepid2                 = 261.37 sec*proc (248 tests)
  Kokkos                    = 251.78 sec*proc (27 tests)
  KokkosKernels             = 433.57 sec*proc (8 tests)
  Komplex                   =   2.34 sec*proc (1 test)
  ML                        =  90.20 sec*proc (34 tests)
  MueLu                     = 422.49 sec*proc (52 tests)
  NOX                       = 218.62 sec*proc (105 tests)
  Panzer                    = 1254.57 sec*proc (159 tests)
  Phalanx                   =  27.17 sec*proc (27 tests)
  Pike                      =   3.71 sec*proc (7 tests)
  Piro                      =  26.25 sec*proc (11 tests)
  Pliris                    =   4.81 sec*proc (2 tests)
  RTOp                      =  27.72 sec*proc (24 tests)
  Rythmos                   =  76.80 sec*proc (83 tests)
  SEACAS                    =  21.90 sec*proc (20 tests)
  STK                       =   1.20 sec*proc (1 test)
  Sacado                    = 183.44 sec*proc (297 tests)
  Shards                    =   1.46 sec*proc (4 tests)
  Stratimikos               =  44.43 sec*proc (39 tests)
  Teko                      =  66.47 sec*proc (19 tests)
  Tempus                    = 514.51 sec*proc (54 tests)
  Teuchos                   = 190.46 sec*proc (135 tests)
  Thyra                     = 112.25 sec*proc (81 tests)
  Tpetra                    = 325.77 sec*proc (174 tests)
  TrilinosCouplings         =   4.45 sec*proc (1 test)
  TrilinosFrameworkTests    =   6.10 sec*proc (4 tests)
  Triutils                  =   4.34 sec*proc (2 tests)
  Xpetra                    =  62.20 sec*proc (18 tests)
  Zoltan                    = 3368.46 sec*proc (35 tests)
  Zoltan2                   = 255.36 sec*proc (101 tests)
  
  Total Test time (real) = 668.82 sec
  
  The following tests FAILED:
  	566 - Zoltan_ch_ewgt_zoltan_parallel (Failed)
  	567 - Zoltan_ch_grid20x19_zoltan_parallel (Failed)
  	571 - Zoltan_ch_nograph_zoltan_parallel (Failed)
  	574 - Zoltan_ch_simple_zoltan_parallel (Failed)
  Errors while running CTest

cuda-9.2-opt-Power8-Kepler37:

  99% tests passed, 8 tests failed out of 2264
  
  Subproject Time Summary:
  Amesos                    =  28.89 sec*proc (13 tests)
  Amesos2                   =  47.96 sec*proc (8 tests)
  Anasazi                   = 356.47 sec*proc (74 tests)
  AztecOO                   =  30.41 sec*proc (17 tests)
  Belos                     = 349.99 sec*proc (81 tests)
  Domi                      = 417.10 sec*proc (125 tests)
  Epetra                    = 120.54 sec*proc (63 tests)
  EpetraExt                 =  30.12 sec*proc (10 tests)
  Galeri                    =  14.84 sec*proc (9 tests)
  Ifpack                    = 109.28 sec*proc (48 tests)
  Ifpack2                   = 480.91 sec*proc (36 tests)
  Intrepid2                 = 912.53 sec*proc (255 tests)
  Kokkos                    = 776.72 sec*proc (27 tests)
  KokkosKernels             = 287.47 sec*proc (8 tests)
  Komplex                   =   2.41 sec*proc (1 test)
  ML                        =  84.72 sec*proc (34 tests)
  MueLu                     = 3228.43 sec*proc (50 tests)
  NOX                       = 439.74 sec*proc (105 tests)
  Panzer                    = 6489.05 sec*proc (156 tests)
  Phalanx                   =  52.69 sec*proc (27 tests)
  Pike                      =  11.69 sec*proc (7 tests)
  Piro                      =  23.42 sec*proc (11 tests)
  Pliris                    =  14.03 sec*proc (2 tests)
  RTOp                      =  27.86 sec*proc (24 tests)
  Rythmos                   = 131.34 sec*proc (83 tests)
  SEACAS                    =  29.45 sec*proc (20 tests)
  STK                       =   1.64 sec*proc (1 test)
  Sacado                    = 230.45 sec*proc (300 tests)
  Shards                    =   1.61 sec*proc (4 tests)
  Stratimikos               =  68.10 sec*proc (39 tests)
  Teko                      = 659.03 sec*proc (19 tests)
  Tempus                    = 883.42 sec*proc (54 tests)
  Teuchos                   = 236.24 sec*proc (135 tests)
  Thyra                     = 185.69 sec*proc (81 tests)
  Tpetra                    = 2127.52 sec*proc (176 tests)
  TrilinosCouplings         =   4.47 sec*proc (1 test)
  TrilinosFrameworkTests    =  10.87 sec*proc (4 tests)
  Triutils                  =   6.44 sec*proc (2 tests)
  Xpetra                    = 235.55 sec*proc (18 tests)
  Zoltan                    = 4126.41 sec*proc (35 tests)
  Zoltan2                   = 634.72 sec*proc (101 tests)
  
  Total Test time (real) = 3047.83 sec
  
  The following tests FAILED:
  	  6 - KokkosCore_UnitTest_Cuda_MPI_1 (Timeout)
  	569 - Zoltan_ch_ewgt_zoltan_parallel (Failed)
  	570 - Zoltan_ch_grid20x19_zoltan_parallel (Failed)
  	574 - Zoltan_ch_nograph_zoltan_parallel (Failed)
  	577 - Zoltan_ch_simple_zoltan_parallel (Failed)
  	1013 - Pliris_vector_random_MPI_3 (Failed)
  	1014 - Pliris_vector_random_MPI_4 (Failed)
  	2237 - PanzerAdaptersSTK_PoissonInterfaceExample_2d_diffsideids_MPI_1 (Timeout)
  Errors while running CTest

Note that we don't run run Zoltan or Pliris tests as part of the current ATDM Trilinos builds. I just use --enable-all-packages=on as a shortcut. (I will set up the ATDM Trilinos build to automatically disable tests in the non-tested packages so that we can use this shortcut in the future).

As for the timing out KokkosCore and Panzer tests, I ran them by by themselves and they both passed just fine as shown with:

$ cd cuda-9.2-opt-Power8-Kepler37

$ . load-env.sh 
Hostname 'white11' matches known ATDM host 'white' and system 'ride'
ATDM_CONFIG_TRILNOS_DIR = /home/rabartl/Trilinos.base/Trilinos
Setting default compiler and build options for ATDM_CONFIG_JOB_NAME='cuda-9.2-opt-Power8-Kepler37'
Using white/ride compiler stack CUDA-9.2 to build RELEASE code with Kokkos node type CUDA and KOKKOS_ARCH=Power8,Kepler37

$ cd packages/panzer/

$ bsub -x -I -q rhel7F -n 16 ctest -R PanzerAdaptersSTK_PoissonInterfaceExample_2d_diffsideids_MPI_1
...
    Start 137: PanzerAdaptersSTK_PoissonInterfaceExample_2d_diffsideids_MPI_1
1/1 Test #137: PanzerAdaptersSTK_PoissonInterfaceExample_2d_diffsideids_MPI_1 ...   Passed  129.70 sec

100% tests passed, 0 tests failed out of 1

Label Time Summary:
Panzer    = 129.70 sec*proc (1 test)

Total Test time (real) = 129.86 sec

$ cd ../kokkos

$ bsub -x -I -q rhel7F -n 16 ctest -R KokkosCore_UnitTest_Cuda_MPI_1
...
    Start 2: KokkosCore_UnitTest_Cuda_MPI_1
1/1 Test #2: KokkosCore_UnitTest_Cuda_MPI_1 ...   Passed  117.60 sec

100% tests passed, 0 tests failed out of 1

Label Time Summary:
Kokkos    = 117.60 sec*proc (1 test)

Total Test time (real) = 117.64 sec

We know that these tests run on top of each other and all run on the same GPU so increased runtimes are expected. Hopefully the nightly drivers will not show any timeouts. If they do, we will deal with them.

Checklist

  • My commit messages mention the appropriate GitHub issue numbers.
  • All new and existing tests passed.

trilinos#3290)

This new env also has the correct netcdf build for SEACAS (see trilinos#3288).
@bartlettroscoe bartlettroscoe added pkg: Ifpack2 pkg: seacas client: ATDM Any issue primarily impacting the ATDM project labels Oct 2, 2018
@bartlettroscoe bartlettroscoe added the stage: in progress Work on the issue has started label Oct 2, 2018
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

  • Build Num: 1009
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 1768
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 1463
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Using Repos:

Repo: TRILINOS (bartlettroscoe/Trilinos)
  • Branch: 3290-white-ride-new-openmpi-2.1.2-env
  • SHA: de9fb24
  • Mode: TEST_REPO

Pull Request Author: bartlettroscoe

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

  • Build Num: 1009
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 1768
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 1463
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697
Console Output (last 100 lines) : Trilinos_pullrequest_intel_17.0.1 # 1009 (click to expand)

CMake suite maintained and supported by Kitware (kitware.com/cmake).
Currently Loaded Modulefiles:

  1. sems-env
  2. sems-git/2.10.1
  3. sems-gcc/4.9.3
  4. sems-intel/17.0.1
  5. sems-mpich/3.2
  6. sems-python/2.7.9
  7. sems-boost/1.63.0/base
  8. sems-zlib/1.2.8/base
  9. sems-hdf5/1.8.12/parallel
  10. sems-netcdf/4.4.1/exo_parallel
  11. sems-parmetis/4.0.3/parallel
  12. sems-scotch/6.0.3/nopthread_64bit_parallel
  13. sems-superlu/4.3/base
  14. sems-cmake/3.10.3
  15. atdm-env
  16. atdm-cmake/3.11.1
  17. atdm-ninja_fortran/1.7.2
    MPI type = sems-mpich/3.2
    CDash Track = Pull Request

*** Generating set of Trilinos enables given modified packages from
*** git commit origin/develop to HEAD


TRILINOS_DIR=/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/Trilinos
TRILINOS_SCRIPTS_DIR=/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/Trilinos/cmake/std/../..
TRIBITS_DIR=/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/Trilinos/cmake/std/../../cmake/tribits

A) Generate the Trilinos Packages definition and depencencies XML file

Wrote the file 'TrilinosPackageDependencies.xml'

B) Get the set of changed files

Current directory: /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/Trilinos

git diff --name-only origin/develop..HEAD > /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/changed-files.txt

Wrote file 'changed-files.txt'

Current directory: /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2

C) Get the unfiltered list of changed Trilinos packages (including 'ALL_PACKAGES')

CHANGED_PACKAGES_FULL_LIST=''

D) Filter list of changed packages to get only the PT packages

CHANGED_PACKAGES_PT_LIST=''

E) Generate the *.cmake enables file

Wrote file 'packageEnables.cmake'
Enabled packages:
-- NOTE: No packages are being enabled!
Build name = PR-3549-test-Trilinos_pullrequest_intel_17.0.1-1009
Cur dir = /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/TFW_testing_single_configure_prototype
Source dir = /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/Trilinos
Binary dir = /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/pull_request_test
Parallel level = 18
skip_by_parts_submit = OFF
skip_single_submit = ON
skip_update_step = ON
skip_upload_config_files = OFF
skip_clean_build_dir = OFF
Subproject count = 53
Dashboard model = Experimental
Dashboard track = Pull Request
Running configuration:
/projects/sems/install/rhel6-x86_64/atdm/binary-install/cmake-3.11.1-Linux-x86_64/bin/cmake
-C "/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/Trilinos/cmake/std/PullRequestLinuxIntelTestingSettings.cmake"
-C "/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/packageEnables.cmake"
-DTrilinos_ENABLE_TESTS:BOOL=ON
-G "Ninja"
/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_intel_17.0.1@2/Trilinos
CTEST_DROP_LOCATION = /cdash/submit.php?project=Trilinos
CDash URL1 = https://testing-vm.sandia.gov/cdash/index.php?project=Trilinos&display=project&filtercount=3&showfilters=1&filtercombine=and&field1=site&compare1=61&value1=ascic158&field2=buildname&compare2=61&value2=PR-3549-test-Trilinos_pullrequest_intel_17.0.1-1009&field3=buildstamp&compare3=61&value3=20181002-1915-Pull Request
CDash URL2 = https://testing-vm.sandia.gov/cdash/index.php?project=Trilinos&display=project&filtercount=2&showfilters=0&filtercombine=and&field1=buildname&compare1=61&value1=PR-3549-test-Trilinos_pullrequest_intel_17.0.1-1009&field2=buildstamp&compare2=61&value2=20181002-1915-Pull Request
CDash URL3 = https://testing-vm.sandia.gov/cdash/index.php?project=Trilinos&filtercount=2&showfilters=0&filtercombine=and&field1=buildname&compare1=61&value1=PR-3549-test-Trilinos_pullrequest_intel_17.0.1-1009&field2=buildstamp&compare2=61&value2=20181002-1915-Pull Request
Starting configure step.
Each . represents 1024 bytes of output
............................ Size of output: 27K
configure submit error = 0
Configure suceeded.
Starting build step.
Each symbol represents 1024 bytes of output.
. Size of output: 0K
Build succeeded.
build submit error = 0
Starting testing step.
No tests were found!!!
Tests succeeded.
test submit error = 0
File upload submit error = 0
Archiving artifacts
Finished: SUCCESS

Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.9.3 # 1768 (click to expand)

The most recent SHA for repo: https://github.com/bartlettroscoe/Trilinos on branch: 3290-white-ride-new-openmpi-2.1.2-env is: de9fb24a49eea078a6e9ef43e86ba0ba578e528b
cmake version 3.11.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).
Currently Loaded Modulefiles:

  1. sems-env
  2. sems-git/2.10.1
  3. sems-gcc/4.9.3
  4. sems-openmpi/1.10.1
  5. sems-python/2.7.9
  6. sems-boost/1.63.0/base
  7. sems-zlib/1.2.8/base
  8. sems-hdf5/1.8.12/parallel
  9. sems-netcdf/4.4.1/exo_parallel
  10. sems-parmetis/4.0.3/parallel
  11. sems-scotch/6.0.3/nopthread_64bit_parallel
  12. sems-superlu/4.3/base
  13. sems-cmake/3.10.3
  14. atdm-env
  15. atdm-cmake/3.11.1
  16. atdm-ninja_fortran/1.7.2
    MPI type = sems-openmpi/1.10.1
    CDash Track = Pull Request

*** Generating set of Trilinos enables given modified packages from
*** git commit origin/develop to HEAD


TRILINOS_DIR=/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/Trilinos
TRILINOS_SCRIPTS_DIR=/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/Trilinos/cmake/std/../..
TRIBITS_DIR=/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/Trilinos/cmake/std/../../cmake/tribits

A) Generate the Trilinos Packages definition and depencencies XML file

Wrote the file 'TrilinosPackageDependencies.xml'

B) Get the set of changed files

Current directory: /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/Trilinos

git diff --name-only origin/develop..HEAD > /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/changed-files.txt

Wrote file 'changed-files.txt'

Current directory: /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3

C) Get the unfiltered list of changed Trilinos packages (including 'ALL_PACKAGES')

CHANGED_PACKAGES_FULL_LIST=''

D) Filter list of changed packages to get only the PT packages

CHANGED_PACKAGES_PT_LIST=''

E) Generate the *.cmake enables file

Wrote file 'packageEnables.cmake'
Enabled packages:
-- NOTE: No packages are being enabled!
Build name = PR-3549-test-Trilinos_pullrequest_gcc_4.9.3-1768
Cur dir = /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/TFW_testing_single_configure_prototype
Source dir = /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/Trilinos
Binary dir = /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/pull_request_test
Parallel level = 18
skip_by_parts_submit = OFF
skip_single_submit = ON
skip_update_step = ON
skip_upload_config_files = OFF
skip_clean_build_dir = OFF
Subproject count = 53
Dashboard model = Experimental
Dashboard track = Pull Request
Running configuration:
/projects/sems/install/rhel6-x86_64/atdm/binary-install/cmake-3.11.1-Linux-x86_64/bin/cmake
-C "/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/Trilinos/cmake/std/PullRequestLinuxGCC4.9.3TestingSettings.cmake"
-C "/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/packageEnables.cmake"
-DTrilinos_ENABLE_TESTS:BOOL=ON
-G "Ninja"
/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.9.3/Trilinos
CTEST_DROP_LOCATION = /cdash/submit.php?project=Trilinos
CDash URL1 = https://testing-vm.sandia.gov/cdash/index.php?project=Trilinos&display=project&filtercount=3&showfilters=1&filtercombine=and&field1=site&compare1=61&value1=ascic143&field2=buildname&compare2=61&value2=PR-3549-test-Trilinos_pullrequest_gcc_4.9.3-1768&field3=buildstamp&compare3=61&value3=20181002-1915-Pull Request
CDash URL2 = https://testing-vm.sandia.gov/cdash/index.php?project=Trilinos&display=project&filtercount=2&showfilters=0&filtercombine=and&field1=buildname&compare1=61&value1=PR-3549-test-Trilinos_pullrequest_gcc_4.9.3-1768&field2=buildstamp&compare2=61&value2=20181002-1915-Pull Request
CDash URL3 = https://testing-vm.sandia.gov/cdash/index.php?project=Trilinos&filtercount=2&showfilters=0&filtercombine=and&field1=buildname&compare1=61&value1=PR-3549-test-Trilinos_pullrequest_gcc_4.9.3-1768&field2=buildstamp&compare2=61&value2=20181002-1915-Pull Request
Starting configure step.
Each . represents 1024 bytes of output
............................ Size of output: 27K
configure submit error = 0
Configure suceeded.
Starting build step.
Each symbol represents 1024 bytes of output.
. Size of output: 0K
Build succeeded.
build submit error = 0
Starting testing step.
No tests were found!!!
Tests succeeded.
test submit error = 0
File upload submit error = 0
Finished: SUCCESS

Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.8.4 # 1463 (click to expand)

 1 file changed, 2 insertions(+), 6 deletions(-)
The most recent SHA for repo: https://github.com/bartlettroscoe/Trilinos on branch: 3290-white-ride-new-openmpi-2.1.2-env is: de9fb24a49eea078a6e9ef43e86ba0ba578e528b
ModuleCmd_Load.c(213):ERROR:105: Unable to locate a modulefile for 'atdm-cmake/3.11.1'
cmake version 3.10.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).
Currently Loaded Modulefiles:

  1. sems-env
  2. sems-git/2.10.1
  3. sems-gcc/4.8.4
  4. sems-openmpi/1.10.1
  5. sems-python/2.7.9
  6. sems-boost/1.63.0/base
  7. sems-zlib/1.2.8/base
  8. sems-hdf5/1.8.12/parallel
  9. sems-netcdf/4.4.1/exo_parallel
  10. sems-parmetis/4.0.3/parallel
  11. sems-scotch/6.0.3/nopthread_64bit_parallel
  12. sems-superlu/4.3/base
  13. sems-cmake/3.10.3
  14. atdm-env
  15. atdm-ninja_fortran/1.7.2
    MPI type = sems-openmpi/1.10.1
    CDash Track = Pull Request

*** Generating set of Trilinos enables given modified packages from
*** git commit origin/develop to HEAD


TRILINOS_DIR=/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/Trilinos
TRILINOS_SCRIPTS_DIR=/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/Trilinos/cmake/std/../..
TRIBITS_DIR=/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/Trilinos/cmake/std/../../cmake/tribits

A) Generate the Trilinos Packages definition and depencencies XML file

Wrote the file 'TrilinosPackageDependencies.xml'

B) Get the set of changed files

Current directory: /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/Trilinos

git diff --name-only origin/develop..HEAD > /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/changed-files.txt

Wrote file 'changed-files.txt'

Current directory: /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4

C) Get the unfiltered list of changed Trilinos packages (including 'ALL_PACKAGES')

CHANGED_PACKAGES_FULL_LIST=''

D) Filter list of changed packages to get only the PT packages

CHANGED_PACKAGES_PT_LIST=''

E) Generate the *.cmake enables file

Wrote file 'packageEnables.cmake'
Enabled packages:
-- NOTE: No packages are being enabled!
Build name = PR-3549-test-Trilinos_pullrequest_gcc_4.8.4-1463
Cur dir = /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/TFW_testing_single_configure_prototype
Source dir = /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/Trilinos
Binary dir = /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/pull_request_test
Parallel level = 18
skip_by_parts_submit = OFF
skip_single_submit = ON
skip_update_step = ON
skip_upload_config_files = OFF
skip_clean_build_dir = OFF
Subproject count = 53
Dashboard model = Experimental
Dashboard track = Pull Request
Running configuration:
/projects/sems/install/rhel7-x86_64/sems/utility/cmake/3.10.3/bin/cmake
-C "/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/Trilinos/cmake/std/PullRequestLinuxGCC4.8.4TestingSettings.cmake"
-C "/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/packageEnables.cmake"
-DTrilinos_ENABLE_TESTS:BOOL=ON
-G "Ninja"
/scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/Trilinos
CTEST_DROP_LOCATION = /cdash/submit.php?project=Trilinos
CDash URL1 = https://testing-vm.sandia.gov/cdash/index.php?project=Trilinos&display=project&filtercount=3&showfilters=1&filtercombine=and&field1=site&compare1=61&value1=ascic166&field2=buildname&compare2=61&value2=PR-3549-test-Trilinos_pullrequest_gcc_4.8.4-1463&field3=buildstamp&compare3=61&value3=20181002-1915-Pull Request
CDash URL2 = https://testing-vm.sandia.gov/cdash/index.php?project=Trilinos&display=project&filtercount=2&showfilters=0&filtercombine=and&field1=buildname&compare1=61&value1=PR-3549-test-Trilinos_pullrequest_gcc_4.8.4-1463&field2=buildstamp&compare2=61&value2=20181002-1915-Pull Request
CDash URL3 = https://testing-vm.sandia.gov/cdash/index.php?project=Trilinos&filtercount=2&showfilters=0&filtercombine=and&field1=buildname&compare1=61&value1=PR-3549-test-Trilinos_pullrequest_gcc_4.8.4-1463&field2=buildstamp&compare2=61&value2=20181002-1915-Pull Request
Starting configure step.
Each . represents 1024 bytes of output
................. Size of output: 16K
Error(s) when configuring the project
Error when uploading file: /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/pull_request_test/Testing/20181002-1915/Configure.xml
Error message was: Failed to connect to testing-vm.sandia.gov port 80: Connection timed out
Problems when submitting via HTTP
configure submit error = -1
CMake Error at /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/TFW_testing_single_configure_prototype/simple_testing.cmake:172 (message):
Configure failed with error -1

Single configure/build/test failed. The error code was: 255
Build step 'Execute shell' marked build as failure
Finished: FAILURE


CDash Test Results for PR# 3549.


Wiki: How to Reproduce PR Testing Builds and Errors.

@bartlettroscoe
Copy link
Member Author

@trilinos/framework,

Looks like the auto tester failed above due to not being able to submit results to CDash for the GCC 4.8.4 build showing the error:

Error when uploading file: /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/pull_request_test/Testing/20181002-1915/Configure.xml
Error message was: Failed to connect to testing-vm.sandia.gov port 80: Connection timed out
Problems when submitting via HTTP
configure submit error = -1
CMake Error at /scratch/trilinos/workspace/trilinos-folder/Trilinos_pullrequest_gcc_4.8.4/TFW_testing_single_configure_prototype/simple_testing.cmake:172 (message):
Configure failed with error -1

Can the CTest -S driver turn on the CTest feature for doing multiple retries? The TriBITS CTest -S driver has been doing that for nearly 10 years (and Kitware implemented that feature because of us and our contract with Kitware years ago).

@bartlettroscoe bartlettroscoe added the AT: RETEST Causes the PR autotester to run a new round of PR tests on the next iteration label Oct 2, 2018
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - User Requested Retest - Label AT: RETEST will be reset after testing.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

  • Build Num: 1010
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 1769
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 1464
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Using Repos:

Repo: TRILINOS (bartlettroscoe/Trilinos)
  • Branch: 3290-white-ride-new-openmpi-2.1.2-env
  • SHA: de9fb24
  • Mode: TEST_REPO

Pull Request Author: bartlettroscoe

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - User Requested Retest - Label AT: RETEST will be reset after testing.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

  • Build Num: 1014
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 1773
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 1468
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Using Repos:

Repo: TRILINOS (bartlettroscoe/Trilinos)
  • Branch: 3290-white-ride-new-openmpi-2.1.2-env
  • SHA: de9fb24
  • Mode: TEST_REPO

Pull Request Author: bartlettroscoe

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

  • Build Num: 1014
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 1773
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 1468
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3549
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH 3290-white-ride-new-openmpi-2.1.2-env
TRILINOS_SOURCE_REPO https://github.com/bartlettroscoe/Trilinos
TRILINOS_SOURCE_SHA de9fb24
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bb78697


CDash Test Results for PR# 3549.

@trilinos-autotester trilinos-autotester removed the AT: RETEST Causes the PR autotester to run a new round of PR tests on the next iteration label Oct 2, 2018
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO REVIEWS HAVE BEEN PERFORMED ON THIS PULL REQUEST!

@trilinos-autotester
Copy link
Contributor

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@bartlettroscoe
Copy link
Member Author

@fryeguy52 or @mhoemmen, if you are online and can see this, can I get a quick approval for this PR?

@bartlettroscoe bartlettroscoe added the AT: AUTOMERGE Causes the PR autotester to automatically merge the PR branch once approvals are completed label Oct 2, 2018
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ fryeguy52 ]!

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Pull Request will be Automerged

@trilinos-autotester trilinos-autotester merged commit 8cd56b3 into trilinos:develop Oct 2, 2018
@trilinos-autotester
Copy link
Contributor

Merge on Pull Request# 3549: IS A SUCCESS - Pull Request successfully merged

@trilinos-autotester trilinos-autotester removed the AT: AUTOMERGE Causes the PR autotester to automatically merge the PR branch once approvals are completed label Oct 2, 2018
@bartlettroscoe bartlettroscoe removed the stage: in progress Work on the issue has started label Oct 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client: ATDM Any issue primarily impacting the ATDM project pkg: Ifpack2 pkg: seacas
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants