Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Framework: ATDM configuration should update the way Kokkos arch(es) are set #7247

Closed
brian-kelley opened this issue Apr 23, 2020 · 4 comments · Fixed by #7448
Closed

Framework: ATDM configuration should update the way Kokkos arch(es) are set #7247

brian-kelley opened this issue Apr 23, 2020 · 4 comments · Fixed by #7448
Labels
ATDM Config Issues that are specific to the ATDM configuration settings ATDM DevOps Issues that will be worked by the Coordinated ATDM DevOps teams client: ATDM Any issue primarily impacting the ATDM project type: bug The primary issue is a bug in Trilinos code or tests

Comments

@brian-kelley
Copy link
Contributor

Bug Report

@trilinos/framework

Description

What went wrong? What should have happened? Do you have an idea what might
fix things? How has this bug affected you? What are you trying to accomplish?
What specifically must we demonstrate to call this work complete?

The ATDM environment scripts use the deprecated syntax -DKOKKOS_ARCH=XYZ. After Kokkos 3.0, this should be -DKokkos_ARCH_XYZ=ON. If there are multiple architectures enabled (like CPU and GPU) there should be multiple arch enable flags.

Steps to Reproduce

Example on Stria:

source $TRILINOS_SRC/cmake/std/atdm/load-env.sh stria-arm-release
cmake DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=$TRILINOS_SRC/cmake/std/atdm/ATDMDevEnv.cmake ...... $TRILINOS_SRC

# This prints out warning:

CMake Warning at packages/kokkos/cmake/kokkos_functions.cmake:33 (MESSAGE):
  Given deprecated option list KOKKOS_ARCH.  This must now be given as
  separate -D options, which assuming you spelled options correctly would be:

    -DKokkos_ARCH_ARMV8-TX2=ON

  Remove CMakeCache.txt and re-run.  For a list of valid options, refer to
  BUILD.md or even look at CMakeCache.txt (before deleting it).
Call Stack (most recent call first):
  packages/kokkos/cmake/kokkos_arch.cmake:25 (KOKKOS_DEPRECATED_LIST)
  packages/kokkos/cmake/kokkos_tribits.cmake:181 (INCLUDE)
  packages/kokkos/CMakeLists.txt:156 (KOKKOS_SETUP_BUILD_ENVIRONMENT)
@brian-kelley brian-kelley added the type: bug The primary issue is a bug in Trilinos code or tests label Apr 23, 2020
@prwolfe
Copy link
Contributor

prwolfe commented Apr 27, 2020

Actually these are maintained by @bartlettroscoe

@bartlettroscoe bartlettroscoe added ATDM Config Issues that are specific to the ATDM configuration settings ATDM DevOps Issues that will be worked by the Coordinated ATDM DevOps teams labels Apr 27, 2020
@bartlettroscoe
Copy link
Member

@brian-kelley, yea, I noticed that. That should not be too hard to fix. I have put this on the "Selected" list under https://github.com/trilinos/Trilinos/projects/6.

@bartlettroscoe bartlettroscoe added the client: ATDM Any issue primarily impacting the ATDM project label Apr 27, 2020
@jjellio jjellio mentioned this issue May 28, 2020
Merged
bartlettroscoe added a commit to jjellio/Trilinos that referenced this issue May 28, 2020
@bartlettroscoe
ATDM: Add some comments for clues on how to use (trilinos#7247)
87da239
bartlettroscoe added a commit to jjellio/Trilinos that referenced this issue May 28, 2020
@bartlettroscoe
ATDM: Make KOKKOS_ARCH looping work with commas (trilinos#7247)
8ea5106
bartlettroscoe added a commit to jjellio/Trilinos that referenced this issue May 28, 2020
@bartlettroscoe
ATDM: Kokkos now likes upper-case arch names (trilinos#7247)
b806ffb
@bartlettroscoe
Copy link
Member

With PR #7448, this should be done. (Thanks @jjellio!)

Putting in review.

jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue May 29, 2020
Merge 'trilinos/Trilinos:develop' (42ec188) into 'tcad-charon/Trilino…
7968534 
…s:develop' (f96bb4d).

* trilinos-develop: (32 commits)
  ATDM: Kokkos now likes upper-case arch names (trilinos#7247)
  ATDM: Make KOKKOS_ARCH looping work with commas (trilinos#7247)
  ATDM: Add some comments for clues on how to use (trilinos#7247)
  Update KOKKOS_ARCH settings to Kokkos_ARCH_arch=ON
  Summary:
  MiniTensor: Sierra/SM tolerance changes.
  MueLu: skip test for SC=complex
  ATDM: ats1: Promote all to 'Specialized' (ATDV-355)
  MueLu: remove out-dated comment
  ATDM: ats1: Remove ATDM_CONFIG_Kokkos_ENABLE_SERIAL=OFF (trilinos#7435)
  ATDM: Add unset ATDM_CONFIG_Kokkos_ENABLE_SERIAL and cleanup some unused (trilinos#7435)
  MueLu: fix issue trilinos#7411
  Add ortho option to Epetra Gmres ex for testing.
  Print return value for jsrun and some reformatting (trilinos#7211)
  ATDM: ats2: Purge then load StdEnv (ATDV-351)
  ATDM: ats2: Update to use patchec cmake 3.17.2 and CTest GPU allocation (ATDV-333)
  ATDM: ats2: Only load on sparc-dev module, other cleanup (ATDV-351)
  ATDM: ats2: Make 'rolling' uppercase in ATDM_CONFIG_COMPILER (ATDM-351)
  ATDM: ats2: Switch to atdm_match_any_buildname_keyword (ATDV-351)
  ATDM: ats2: Add unit tests for custom_builds.sh (ATDV-351)
  ...
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue May 30, 2020
Merge 'trilinos/Trilinos:develop' (b0ec091) into 'tcad-charon/Trilino…
1df2350 
…s:develop' (f96bb4d).

* trilinos-develop: (36 commits)
  Fixes calls to normalize zero vectors in ICGS/IMGS (trilinos#7454)
  Tacho - remove valgrind error
  ATDM: Kokkos now likes upper-case arch names (trilinos#7247)
  ATDM: Make KOKKOS_ARCH looping work with commas (trilinos#7247)
  ATDM: Add some comments for clues on how to use (trilinos#7247)
  Update KOKKOS_ARCH settings to Kokkos_ARCH_arch=ON
  Summary:
  MiniTensor: Sierra/SM tolerance changes.
  MueLu: skip test for SC=complex
  ATDM: ats1: Promote all to 'Specialized' (ATDV-355)
  MueLu: remove out-dated comment
  ATDM: ats1: Remove ATDM_CONFIG_Kokkos_ENABLE_SERIAL=OFF (trilinos#7435)
  ATDM: Add unset ATDM_CONFIG_Kokkos_ENABLE_SERIAL and cleanup some unused (trilinos#7435)
  MueLu: fix issue trilinos#7411
  Add ortho option to Epetra Gmres ex for testing.
  Print return value for jsrun and some reformatting (trilinos#7211)
  ATDM: ats2: Purge then load StdEnv (ATDV-351)
  ATDM: ats2: Update to use patchec cmake 3.17.2 and CTest GPU allocation (ATDV-333)
  ATDM: ats2: Only load on sparc-dev module, other cleanup (ATDV-351)
  ATDM: ats2: Make 'rolling' uppercase in ATDM_CONFIG_COMPILER (ATDM-351)
  ...
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue May 31, 2020
Merge 'trilinos/Trilinos:develop' (7ed2016) into 'tcad-charon/Trilino…
3c1fa33 
…s:develop' (f96bb4d).

* trilinos-develop: (57 commits)
  Fixes calls to normalize zero vectors in ICGS/IMGS (trilinos#7454)
  ATDM: ats2: Elevate complex build to 'Specialized' for now (ATDV-300)
  Amesos2: Fix Epetra crash for SuperLU
  ATDM: ats2: Add cuda-complex build to support GEMMA (ATDV-300)
  Tacho - remove valgrind error
  ATDM: Kokkos now likes upper-case arch names (trilinos#7247)
  ATDM: Make KOKKOS_ARCH looping work with commas (trilinos#7247)
  ATDM: Add some comments for clues on how to use (trilinos#7247)
  Update KOKKOS_ARCH settings to Kokkos_ARCH_arch=ON
  Summary:
  MiniTensor: Sierra/SM tolerance changes.
  MueLu: skip test for SC=complex
  ATDM: ats1: Promote all to 'Specialized' (ATDV-355)
  MueLu: remove out-dated comment
  Amesos2: Fix sign compare warning for SuperLU
  ATDM: ats1: Remove ATDM_CONFIG_Kokkos_ENABLE_SERIAL=OFF (trilinos#7435)
  ATDM: Add unset ATDM_CONFIG_Kokkos_ENABLE_SERIAL and cleanup some unused (trilinos#7435)
  MueLu: fix issue trilinos#7411
  Add ortho option to Epetra Gmres ex for testing.
  Print return value for jsrun and some reformatting (trilinos#7211)
  ...
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Jun 1, 2020
Merge 'trilinos/Trilinos:develop' (7ed2016) into 'tcad-charon/Trilino…
6f4d146 
…s:develop' (f96bb4d).

* trilinos-develop: (57 commits)
  Fixes calls to normalize zero vectors in ICGS/IMGS (trilinos#7454)
  ATDM: ats2: Elevate complex build to 'Specialized' for now (ATDV-300)
  Amesos2: Fix Epetra crash for SuperLU
  ATDM: ats2: Add cuda-complex build to support GEMMA (ATDV-300)
  Tacho - remove valgrind error
  ATDM: Kokkos now likes upper-case arch names (trilinos#7247)
  ATDM: Make KOKKOS_ARCH looping work with commas (trilinos#7247)
  ATDM: Add some comments for clues on how to use (trilinos#7247)
  Update KOKKOS_ARCH settings to Kokkos_ARCH_arch=ON
  Summary:
  MiniTensor: Sierra/SM tolerance changes.
  MueLu: skip test for SC=complex
  ATDM: ats1: Promote all to 'Specialized' (ATDV-355)
  MueLu: remove out-dated comment
  Amesos2: Fix sign compare warning for SuperLU
  ATDM: ats1: Remove ATDM_CONFIG_Kokkos_ENABLE_SERIAL=OFF (trilinos#7435)
  ATDM: Add unset ATDM_CONFIG_Kokkos_ENABLE_SERIAL and cleanup some unused (trilinos#7435)
  MueLu: fix issue trilinos#7411
  Add ortho option to Epetra Gmres ex for testing.
  Print return value for jsrun and some reformatting (trilinos#7211)
  ...
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Jun 2, 2020
Merge 'trilinos/Trilinos:develop' (7ed2016) into 'tcad-charon/Trilino…
340f5f0 
…s:develop' (f96bb4d).

* trilinos-develop: (57 commits)
  Fixes calls to normalize zero vectors in ICGS/IMGS (trilinos#7454)
  ATDM: ats2: Elevate complex build to 'Specialized' for now (ATDV-300)
  Amesos2: Fix Epetra crash for SuperLU
  ATDM: ats2: Add cuda-complex build to support GEMMA (ATDV-300)
  Tacho - remove valgrind error
  ATDM: Kokkos now likes upper-case arch names (trilinos#7247)
  ATDM: Make KOKKOS_ARCH looping work with commas (trilinos#7247)
  ATDM: Add some comments for clues on how to use (trilinos#7247)
  Update KOKKOS_ARCH settings to Kokkos_ARCH_arch=ON
  Summary:
  MiniTensor: Sierra/SM tolerance changes.
  MueLu: skip test for SC=complex
  ATDM: ats1: Promote all to 'Specialized' (ATDV-355)
  MueLu: remove out-dated comment
  Amesos2: Fix sign compare warning for SuperLU
  ATDM: ats1: Remove ATDM_CONFIG_Kokkos_ENABLE_SERIAL=OFF (trilinos#7435)
  ATDM: Add unset ATDM_CONFIG_Kokkos_ENABLE_SERIAL and cleanup some unused (trilinos#7435)
  MueLu: fix issue trilinos#7411
  Add ortho option to Epetra Gmres ex for testing.
  Print return value for jsrun and some reformatting (trilinos#7211)
  ...
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Jun 3, 2020
Merge 'trilinos/Trilinos:develop' (201bed2) into 'tcad-charon/Trilino…
c735320 
…s:develop' (f96bb4d).

* trilinos-develop: (58 commits)
  Piro: bug fix required for transient forward sensitivities to work in Albany and extension of observer to observe dx/dp (trilinos#7456)
  Fixes calls to normalize zero vectors in ICGS/IMGS (trilinos#7454)
  ATDM: ats2: Elevate complex build to 'Specialized' for now (ATDV-300)
  Amesos2: Fix Epetra crash for SuperLU
  ATDM: ats2: Add cuda-complex build to support GEMMA (ATDV-300)
  Tacho - remove valgrind error
  ATDM: Kokkos now likes upper-case arch names (trilinos#7247)
  ATDM: Make KOKKOS_ARCH looping work with commas (trilinos#7247)
  ATDM: Add some comments for clues on how to use (trilinos#7247)
  Update KOKKOS_ARCH settings to Kokkos_ARCH_arch=ON
  Summary:
  MiniTensor: Sierra/SM tolerance changes.
  MueLu: skip test for SC=complex
  ATDM: ats1: Promote all to 'Specialized' (ATDV-355)
  MueLu: remove out-dated comment
  Amesos2: Fix sign compare warning for SuperLU
  ATDM: ats1: Remove ATDM_CONFIG_Kokkos_ENABLE_SERIAL=OFF (trilinos#7435)
  ATDM: Add unset ATDM_CONFIG_Kokkos_ENABLE_SERIAL and cleanup some unused (trilinos#7435)
  MueLu: fix issue trilinos#7411
  Add ortho option to Epetra Gmres ex for testing.
  ...
jmgate pushed a commit to tcad-charon/Trilinos that referenced this issue Jun 4, 2020
Merge 'trilinos/Trilinos:develop' (184c490) into 'tcad-charon/Trilino…
0326409 
…s:develop' (f96bb4d).

* trilinos-develop: (61 commits)
  Belos: Fix BiCGStab restart
  Piro: bug fix required for transient forward sensitivities to work in Albany and extension of observer to observe dx/dp (trilinos#7456)
  ML: Adding debug printout
  Fixes calls to normalize zero vectors in ICGS/IMGS (trilinos#7454)
  ATDM: ats2: Elevate complex build to 'Specialized' for now (ATDV-300)
  Amesos2: Fix Epetra crash for SuperLU
  ATDM: ats2: Add cuda-complex build to support GEMMA (ATDV-300)
  Tacho - remove valgrind error
  ATDM: Kokkos now likes upper-case arch names (trilinos#7247)
  ATDM: Make KOKKOS_ARCH looping work with commas (trilinos#7247)
  ATDM: Add some comments for clues on how to use (trilinos#7247)
  Update KOKKOS_ARCH settings to Kokkos_ARCH_arch=ON
  Summary:
  MiniTensor: Sierra/SM tolerance changes.
  MueLu: skip test for SC=complex
  ATDM: ats1: Promote all to 'Specialized' (ATDV-355)
  MueLu: remove out-dated comment
  Amesos2: Fix sign compare warning for SuperLU
  ATDM: ats1: Remove ATDM_CONFIG_Kokkos_ENABLE_SERIAL=OFF (trilinos#7435)
  ATDM: Add unset ATDM_CONFIG_Kokkos_ENABLE_SERIAL and cleanup some unused (trilinos#7435)
  ...
@bartlettroscoe
Copy link
Member

This was addressed in PR #7448 and has not seemed to be causing any problems.

Closing as complete.

@trilinos-autotester trilinos-autotester mentioned this issue Apr 11, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ATDM Config Issues that are specific to the ATDM configuration settings ATDM DevOps Issues that will be worked by the Coordinated ATDM DevOps teams client: ATDM Any issue primarily impacting the ATDM project type: bug The primary issue is a bug in Trilinos code or tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Trilinos: Fix ATDM Kokkos arch jjellio/Trilinos
3 participants
@bartlettroscoe @brian-kelley @prwolfe