Tpetra: Allow enabling SYCL backend in Tpetra #9086

masterleinad · 2021-05-03T22:35:52Z

@trilinos/tpetra

Motivation

This pull request allows enabling Kokkos' SYCL backend in Tpetra to allow Trilinos running on Intel GPUs.

Currently, it needs kokkos/kokkos#4012, kokkos/kokkos#4007, kokkos/kokkos#3983, kokkos/kokkos#3998 and kokkos/kokkos-kernels#959 (all relevant changes also contained in this pull request) to build Trilinos with Tpetra enabled and running the unit tests. At the moment,

ImbalancedRowMatrix,
assembleElement, and
MatMat

are failing and have been disabled using preprocessor macros and marking the relevant places with FIXME_SYCL.

Testing

Tested on a CPU and on a Intel GPU passing all tests (with some disabled as described above).

trilinos-autotester · 2021-05-03T22:38:19Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
WARNING: NO REVIEWERS HAVE BEEN REQUESTED FOR THIS PULL REQUEST!

kddevin

Thanks, @masterleinad , for these changes.
I looked at the Tpetra changes only, and request a few changes or explanations.
It would be good for @brian-kelley to review as well.

kddevin · 2021-05-03T23:00:42Z

packages/tpetra/core/src/Tpetra_Details_unpackCrsMatrixAndCombine_def.hpp

@@ -284,7 +284,7 @@ struct UnpackCrsMatrixAndCombineFunctor {

    if (expected_num_bytes > num_bytes)
    {
-      printf(
+      KOKKOS_IMPL_DO_NOT_USE_PRINTF(


I prefer not to use KOKKOS_IMPL details in Tpetra. Is there a problem with printf and SYCL? What does KOKKOS_IMPL_DO_NOT_USE_PRINTF do? How would one know to use this macro rather than printf?

Oh, I should have pointed this out directly. SYCL doesn't support printf on the device. In Kokkos we are defining a workaround to make this work regardless with the intention of only using it internally (such that it could be removed at any point). The alternative is to just guard these places with #ifndef KOKKOS_ENABLE_SYCL or similar.

yes there is a problem with printf in SYCL. Namely that it isn't supported currently and you have to call sycl::printf or something like that ...
We are talking to various compiler folks to get that changed, and thus didn't want to add an official portability functionality like KOKKOS_PRINTF yet. But without changing it this guy won't compile/work with SYCL right now.

Probably doing #ifndef KOKKOS_ENABLE_SYCL around the printfs is better than using a Kokkos implementation detail. Plus, it documents why we are doing something different from printf -- that is, that SYCL alone doesn't like it. When Kokkos adds an official portable KOKKOS_PRINTF, we will adopt it.

Fixed in 45fefc7 (#9086).

packages/tpetra/core/src/Tpetra_Details_StaticView.cpp

kddevin · 2021-05-03T23:03:55Z

packages/tpetra/core/example/BlockCrs/Tpetra_TestBlockCrs.cpp

@@ -471,7 +471,8 @@ int main (int argc, char *argv[])
      {
        TimeMonitor timerMultiVectorFill(*TimeMonitor::getNewTimer("4) MultiVectorFill"));

-        auto value = X->getLocalView<typename exec_space::memory_space>(Tpetra::Access::OverwriteAll);
+        using device_type = typename tpetra_multivector_type::node_type::device_type;
+        auto value = X->getLocalView<device_type>(Tpetra::Access::OverwriteAll);


OK, but prefer use of getLocalViewDevice(Tpetra::Access::OverwriteAll).

See 3221242 (#9086).

packages/tpetra/core/example/BlockCrs/Tpetra_TestBlockCrs.cpp

trilinos-autotester · 2021-05-03T23:18:14Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
THE LAST COMMIT TO THIS PULL REQUEST HAS BEEN REVIEWED, BUT NOT ACCEPTED OR REQUIRES CHANGES

crtrott · 2021-05-03T23:57:52Z

Btw. we are looking into making all the cahnges necessary for Kokkos/KokkosKernels part of the 3.4.1 patch release. So we wouldn't necessarily want you to go ahead and merge this before that is out and in Trilinos, and which point the changes are more or less confined to Tpetra.

brian-kelley

The only issue I see is the CommBufferMemorySpace thing is templated on execution space so the specialization should use SYCL, not the SYCLSharedUSMSpace. HIP is wrong above, I found that yesterday and will fix in another PR.

brian-kelley · 2021-05-04T16:19:54Z

packages/tpetra/core/src/Tpetra_Details_DefaultTypes.hpp

@@ -132,6 +135,14 @@ namespace DefaultTypes {
  };
 #endif

+#ifdef KOKKOS_ENABLE_SYCL
+  template<>
+  struct CommBufferMemorySpace<Kokkos::Experimental::SYCLSharedUSMSpace>


This should actually be specialized on SYCL the execution space. The HIPHostPinnedSpace above is wrong, it should be HIP.

Fixed in 68f57e3 (#9086).

kddevin

Thanks for your explanations. I resolved most of my conversations.
I'd still like to see KOKKOS_IMPL_DO_NOT_USE_PRINTF replaced with #ifndef KOKKOS_ENABLE_SYCL.

kddevin · 2021-05-04T16:58:09Z

packages/tpetra/core/src/Tpetra_Details_unpackCrsMatrixAndCombine_def.hpp

@@ -284,7 +284,7 @@ struct UnpackCrsMatrixAndCombineFunctor {

    if (expected_num_bytes > num_bytes)
    {
-      printf(
+      KOKKOS_IMPL_DO_NOT_USE_PRINTF(


Probably doing #ifndef KOKKOS_ENABLE_SYCL around the printfs is better than using a Kokkos implementation detail. Plus, it documents why we are doing something different from printf -- that is, that SYCL alone doesn't like it. When Kokkos adds an official portable KOKKOS_PRINTF, we will adopt it.

brian-kelley

Thanks for making that change.

trilinos-autotester · 2021-05-04T17:18:51Z

Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ brian-kelley ]!

trilinos-autotester · 2021-05-04T17:21:10Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_8.3.0

Build Num: 4349
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_serial

Build Num: 1885
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_debug

Build Num: 2368
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

Build Num: 9704
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105

Build Num: 1122
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_clang_10.0.0

Build Num: 2484
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105_uvm_off

Build Num: 119
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
PULLREQUEST_CDASH_TRACK	Experimental
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_python_3

Build Num: 5090
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Using Repos:

Repo: TRILINOS (masterleinad/Trilinos)

Pull Request Author: masterleinad

trilinos-autotester · 2021-05-04T22:07:16Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_8.3.0

Build Num: 4349
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_serial

Build Num: 1885
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_debug

Build Num: 2368
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

Build Num: 9704
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105

Build Num: 1122
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_clang_10.0.0

Build Num: 2484
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105_uvm_off

Build Num: 119
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
PULLREQUEST_CDASH_TRACK	Experimental
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

Build Information

Test Name: Trilinos_pullrequest_python_3

Build Num: 5090
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`45fefc7`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`c00ff3b`

CDash Test Results for PR# 9086.

trilinos-autotester · 2021-05-04T22:07:34Z

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ brian-kelley kddevin ]!

trilinos-autotester · 2021-05-04T22:07:41Z

Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR...

lucbv · 2021-05-04T23:21:19Z

packages/tpetra/CMakeLists.txt

@@ -529,6 +529,29 @@ ELSE () # NOT Tpetra_INST_HIP
    MESSAGE(STATUS "NOTE: Kokkos::HIP is ON (the CMake option Kokkos_ENABLE_HIP is ON), but the corresponding Tpetra Node type is disabled.  If you want to enable instantiation and use of Kokkos::HIP in Tpetra, please also set the CMake option Tpetra_INST_HIP:BOOL=ON.  If you use the Kokkos::HIP version of Tpetra without doing this, you will get link errors!")
  ENDIF ()
 ENDIF () # Tpetra_INST_HIP
+
+# Kokkos::SYCL (Kokkos::Compat::KokkosSYCLWrapperNode)
+GLOBAL_SET(HAVE_TPETRA_INST_SYCL_DEFAULT OFF)


It would make sense to move this statement further up around line 365 and you might want to add an ASSERT_DEFINED(Kokkos_ENABLE_SYCL)

See a2f73a1 (#9086).

lucbv

I just had a small comment regarding the top CMakeList.txt in tpetra, it's nothing big but we should try to be uniform if possible.

…ards

This reverts commit b918d95.

masterleinad · 2021-06-04T16:56:08Z

I rebased to resolve the merge conflicts and fixed the resulting problems (making sure that all the tests pass again on Intel GPUs).

trilinos-autotester · 2021-06-04T18:08:39Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
THE LAST COMMIT TO THIS PULL REQUEST HAS NOT BEEN REVIEWED YET!

trilinos-autotester · 2021-06-04T19:41:21Z

Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ brian-kelley ]!

trilinos-autotester · 2021-06-04T19:43:25Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_8.3.0

Build Num: 4589
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_serial

Build Num: 2120
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_debug

Build Num: 2601
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

Build Num: 9922
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105

Build Num: 1336
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105_uvm_off

Build Num: 333
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_clang_10.0.0

Build Num: 2701
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_python_3

Build Num: 5275
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Using Repos:

Repo: TRILINOS (masterleinad/Trilinos)

Pull Request Author: masterleinad

trilinos-autotester · 2021-06-04T23:00:11Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_8.3.0

Build Num: 4589
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_serial

Build Num: 2120
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_debug

Build Num: 2601
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

Build Num: 9922
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105

Build Num: 1336
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105_uvm_off

Build Num: 333
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_clang_10.0.0

Build Num: 2701
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

Build Information

Test Name: Trilinos_pullrequest_python_3

Build Num: 5275
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	pkg: Kokkos;pkg: KokkosKernels;pkg: Tpetra
PULLREQUESTNUM	9086
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	add_sycl
TRILINOS_SOURCE_REPO	https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA	`38a7a40`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`f75748f`

CDash Test Results for PR# 9086.

trilinos-autotester · 2021-06-04T23:00:29Z

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ brian-kelley ]!

trilinos-autotester · 2021-06-04T23:00:37Z

Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR...

trilinos-autotester · 2021-06-05T23:16:39Z

Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR...

trilinos-autotester · 2021-06-06T23:16:40Z

Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR...

kddevin

Thanks, @masterleinad

trilinos-autotester · 2021-06-08T00:49:55Z

Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR...

lucbv · 2021-06-08T15:56:47Z

Thanks @masterleinad for taking care of this, @crtrott and I are also OK with the state of the PR so I am merging it now.

masterleinad requested a review from a team as a code owner May 3, 2021 22:35

masterleinad mentioned this pull request May 3, 2021

Enable SYCL tests kokkos/kokkos-kernels#927

Merged

masterleinad marked this pull request as draft May 3, 2021 23:01

kddevin requested changes May 3, 2021

View reviewed changes

kddevin added pkg: Tpetra pkg: Kokkos pkg: KokkosKernels labels May 3, 2021

kddevin requested a review from brian-kelley May 3, 2021 23:08

masterleinad mentioned this pull request May 4, 2021

[3.4.1] Take over #3991: staticcsrgraph: use device type instead of execution space to construct views kokkos/kokkos#3998

Merged

brian-kelley requested changes May 4, 2021

View reviewed changes

kddevin requested changes May 4, 2021

View reviewed changes

masterleinad requested review from brian-kelley and kddevin and removed request for a team May 4, 2021 17:12

brian-kelley previously approved these changes May 4, 2021

View reviewed changes

kddevin previously approved these changes May 4, 2021

View reviewed changes

kddevin mentioned this pull request May 4, 2021

Framework: automated regression testing for HIP and SYCL is needed #9093

Closed

lucbv reviewed May 4, 2021

View reviewed changes

masterleinad mentioned this pull request May 5, 2021

Tpetra: enable and fix tests for HIP #9090

Merged

masterleinad dismissed stale reviews from kddevin and brian-kelley via a2f73a1 May 5, 2021 18:28

masterleinad added 11 commits June 4, 2021 09:29

Disable failing tests

bbe854a

Add missing implementation file

1bfebc9

Add missing test file

fe24474

Disable another test

9122fda

Prefer getLocalViewDevice

6b578a9

Use SYCLDeviceUSMSpace for CommBufferMemorySpace

6e539cb

Replace KOKKOS_IMPL_DO_NOT_USE_PRINTF by ifndef KOKKOS_ENABLE_SYCL gu…

8073b9c

…ards

Switch CommBufferMemorySpace to use host-accessible space

7c88e02

Tpetra: make cmake node logic consistent

1e6f7b9

Revert "Switch CommBufferMemorySpace to use host-accessible space"

663e2a4

This reverts commit b918d95.

Fix ups after rebasing

38a7a40

masterleinad dismissed stale reviews from lucbv and brian-kelley via 38a7a40 June 4, 2021 16:54

masterleinad force-pushed the add_sycl branch from 8b59e2f to 38a7a40 Compare June 4, 2021 16:54

trilinos-autotester removed the AT: STALE Added by the PR autotester if too much time has elapsed since the last successful PR test iteration label Jun 4, 2021

brian-kelley approved these changes Jun 4, 2021

View reviewed changes

kddevin approved these changes Jun 7, 2021

View reviewed changes

lucbv merged commit 332e870 into trilinos:develop Jun 8, 2021

masterleinad mentioned this pull request Jun 10, 2021

Tpetra: Enable more SYCL tests #9266

Merged

Tpetra: Allow enabling SYCL backend in Tpetra #9086

Tpetra: Allow enabling SYCL backend in Tpetra #9086

Conversation

masterleinad commented May 3, 2021 • edited Loading

Motivation

Testing

trilinos-autotester commented May 3, 2021

kddevin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trilinos-autotester commented May 3, 2021

crtrott commented May 3, 2021

brian-kelley left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kddevin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brian-kelley left a comment

Choose a reason for hiding this comment

trilinos-autotester commented May 4, 2021

trilinos-autotester commented May 4, 2021

Build Information

Test Name: Trilinos_pullrequest_gcc_8.3.0

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_serial

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_debug

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_clang_10.0.0

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105_uvm_off

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_python_3

Jenkins Parameters

Using Repos:

trilinos-autotester commented May 4, 2021

Build Information

Test Name: Trilinos_pullrequest_gcc_8.3.0

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_serial

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_gcc_7.2.0_debug

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_clang_10.0.0

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_cuda_10.1.105_uvm_off

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_python_3

Jenkins Parameters

trilinos-autotester commented May 4, 2021

masterleinad commented May 3, 2021 •

edited

Loading