KokkosKernels,Tpetra: Plug in TPLs (MKL, cuSPARSE, ...) for sparse matrix-matrix multiply

[mhoemmen]

@trilinos/tpetra @trilinos/muelu @mndevec @csiefer2

See #147 for discussion.
Depends on #148 (otherwise we can't plug in a local kernel).
Depends on #415 (MKL sparse matrix-matrix multiply wants to own deallocation of the result matrix's arrays, so Kokkos can't own them; deallocation has to happen in the handle)

[kddevin]

Discussion with @csiefer2 today:
@csiefer2 will be the assignee; targeting FY17 Q1-Q2 for the work.

[mhoemmen]

@csiefer2 discovered possible bugs in MKL's sparse matrix-matrix multiply, for the case of empty rows and/or columns in the input matrices. This suggests that we should avoid relying on MKL for this critical kernel.

[nmhamster]

@mhoemmen - can you provide a small reproducer so we can file a bug report?

[mhoemmen]

@csiefer2, would you happen to know what matrices didn't work with MKL's implementation?

[srajama1]

Note that KokkosKernel SpGEMM is anyway faster than MKL (except when using small number of threads and there is no reuse of structure). So we might not rely on it anyway.

[maherou]

In the long run, we want MKL to work well. If it’s not too much work, it would be good to provide them a test case, or at least mention this result to them. We want their kernels to be fast, so there is less work for us. From: Siva Rajamanickam <[email protected]> Reply-To: trilinos/Trilinos <[email protected]> Date: Monday, December 5, 2016 at 8:45 PM To: trilinos/Trilinos <[email protected]> Subject: [EXTERNAL] Re: [trilinos/Trilinos] KokkosKernels,Tpetra: Plug in TPLs (MKL, cuSPARSE, ...) for sparse matrix-matrix multiply (#430) Note that KokkosKernel SpGEMM is anyway faster than MKL (except when using small number of threads and there is no reuse of structure). So we might not rely on it anyway. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#430 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AA-USyEssxDYMTxWWUwdC4Xu9yKedqXjks5rFNp2gaJpZM4IyYlR>.

[mhoemmen]

I just heard from @csiefer2 today about this. I know his time is limited this fiscal year, so if he finds something, he should feel welcome just to point out the example (e.g., test number or matrix file) and put the rest of us to work in helping him turn that into a test case for MKL.

[csiefer2]

Mehmet already has a lists of problems where MKL doesn't work.

[mhoemmen]

@mndevec , would you happen to have some matrices that are known to fail with MKL? Thanks!

[mndevec]

@mhoemmen
In my case, MKL does not complete (in 20 min) with 256 threads on KNL (with intel 17) for laplace and brick matrices. I can provide the matrix market formats for those.

[nmhamster]

@mndevec please can you supply these so we can file an issue with Intel?

[mhoemmen]

@mndevec wrote:

... MKL does not complete (in 20 min) with 256 threads on KNL (with intel 17) for laplace and brick matrices.

Wow ;-P

[mndevec]

@nmhamster
The matrices are a bit large, laplace A and P matrices are located under /home/mndevec/matrix_market
in bowman.

[mndevec]

By the way, I am using inspector executor with the function: mkl_sparse_spmm.
There is another function in MKL for matrix-matrix multiply. I will plug that too in coming weeks.

[github-actions]

This issue has had no activity for 365 days and is marked for closure. It will be closed after an additional 30 days of inactivity.
If you would like to keep this issue open please add a comment and remove the MARKED_FOR_CLOSURE label.
If this issue should be kept open even with no activity beyond the time limits you can add the label DO_NOT_AUTOCLOSE.

[github-actions]

This issue was closed due to inactivity for 395 days.