Kokkos: Configure error with -DKokkos_ARCH_INTEL_XEHP=ON

[nchaimov]

Bug Report

@crtrott

Description

Starting with commit 05540e2, a SYCL build of Trilinos with -DKokkos_ARCH_INTEL_XEHP=ON results in an error at configure time. A configuration which worked prior to that commit now fails with

-- Built-in Execution Spaces:
--     Device Parallel: Kokkos::Experimental::SYCL
--     Host Parallel: Kokkos::OpenMP
--       Host Serial: SERIAL
-- 
-- Architectures:
--  INTEL_XEHP
CMake Error at packages/kokkos/cmake/kokkos_tpls.cmake:29 (MESSAGE):
  Enabled TPL ONEDPL inside TriBITS build, but this can only be enabled in a
  standalone build
Call Stack (most recent call first):
  packages/kokkos/cmake/kokkos_tpls.cmake:48 (KOKKOS_TPL_OPTION)
  packages/kokkos/cmake/kokkos_tribits.cmake:207 (INCLUDE)
  packages/kokkos/CMakeLists.txt:226 (KOKKOS_SETUP_BUILD_ENVIRONMENT)
-- Configuring incomplete, errors occurred!

Steps to Reproduce

1. SHA1: 05540e2 or later
2. Configure script: config-script-sycl.txt
3. Configure log: config-log.txt

[jhux2]

@trilinos/kokkos @bartlettroscoe

[crtrott]

@masterleinad can you also take a look at this?

[masterleinad]

@masterleinad can you also take a look at this?

We started using oneDPL in the last release but that is not a Trilinos/TriBITS TPL and so we fail at configure-time if that dependency is enabled. We either need to support oneDPL as TPL in Trilinos or just disable oneDPL support for Trilinos (which only affects Kokkos::sort at this point).

[bartlettroscoe]

What is the problem with making oneDPL a TriBITS/Trilinos TPL? How is oneDPL found and how is it utilized by Kokkos in its native CMake build system?

[masterleinad]

What is the problem with making oneDPL a TriBITS/Trilinos TPL? How is oneDPL found and how is it utilized by Kokkos in its native CMake build system?

It's probably just that no one has looked into it.
https://github.com/kokkos/kokkos/blob/1a3ea28f6e97b4c9dd2c8ceed53ad58ed5f94dfe/cmake/Modules/FindTPLONEDPL.cmake#L1-L46 checks first for native compiler support and then for a separate installation (using find_package). On success, we create an imported target to which we link kokkoscore to.

[bartlettroscoe]

@masterleinad,

That's pretty close to what the TriBITS FindTPLOneDPL.cmake file would look like, except for needing to create the the OneDPL::all_libs target and to write a consistent OneDPLConfig.cmake file in the external_packages/ directory. It is a little more tricky in that the implementation may just be compiler defines (which can be used to create an interface target) or a find_package() call with an imported target.

@crtrott,

Is there an SNL system where these Intel OneAPI configurations can be reproduced for all these cases? Looking at the FindTPLONEDPL.cmake file, it seems there are four different cases that need to be tested:

• The header files under oneapi/dpl are already found in the include path:
  • No TBB conflict: No compiler defines needed
  • Has TBB conflict: Needs compiler defines PSTL_USE_PARALLEL_POLICIES=0 _GLIBCXX_USE_TBB_PAR_BACKEND=0
• else:
  • No TBB conflict: Link against imported library target oneDPL
  • Has TBB conflict: Needs compiler defines PSTL_USE_PARALLEL_POLICIES=0 _GLIBCXX_USE_TBB_PAR_BACKEND=0

Are all of these paths tested?

[masterleinad]

#12112 suggests simply disabling oneDPL support by default.

[jwillenbring]

Would this potential fix be delivered only with the next snapshot of Kokkos into Trilinos?

[masterleinad]

Would this potential fix be delivered only with the next snapshot of Kokkos into Trilinos?

No, it's already merged in Trilinos, see #12112. kokkos/kokkos#6342 is the corresponding Kokkos pull request which has also been merged.

[nchaimov]

I've tested this fix and can confirm that I am no longer experiencing the configure error. I can build with the listed configuration on the develop branch (tested with commit e9cbc3c).

[eugeneswalker]

Should we mark this as resolved?