[SYCL] Fix SYCL im2col and convert Overflow with Large Dims

[zhentaoyu]

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

changes

• downsample global_range for im2col when OH and OW are bigger (happens in stable-diffusion) and sync with related cuda code
• make convert int k to int64_t k
• downsample global_range for convert_unary when k is large (happens in stable-diffusion).

[airMeng]

@joeatodd @OuadiElfarouki if no issues from your side, I will merge soon

[NeoZhangJianyu]

Is it possible to add a unit test case for this case?

[zhentaoyu]

Is it possible to add a unit test case for this case?

Yes. I will add some im2col test_cases in test-backend-ops.cpp. And try to add ggml_get_to_fp32_sycl test case in mul_mat. Is that ok for you @NeoZhangJianyu?

[Alcpz]

@airMeng @zhentaoyu

This PR seems to create failures in both A100 and an Arc A770:

A100 test-backend-ops log:

IM2COL(type_input=f32,type_kernel=f16,dst_type=f16,ne_input=[1024,1024,256,1],ne_kernel=[3,3,256,1],s0=1,s1=1,p0=1,p1=1,d0=1,d1=1,is_2D=1): ggml_backend_alloc_ctx_tensors_from_buft: tensor  is too large to fit in a SYCL0 buffer (tensor size: 4831838208, max buffer size: 1073741824) failed to allocate tensors [SYCL0] 
IM2COL(type_input=f32,type_kernel=f16,dst_type=f32,ne_input=[1024,1024,256,1],ne_kernel=[3,3,256,1],s0=1,s1=1,p0=1,p1=1,d0=1,d1=1,is_2D=1): ggml_backend_alloc_ctx_tensors_from_buft: tensor  is too large to fit in a SYCL0 buffer (tensor size: 9663676416, max buffer size: 1073741824) failed to allocate tensors [SYCL0]
MUL_MAT(type_a=f16,type_b=f16,m=512,n=262144,k=9216,bs=[1,1],nr=[1,1]): ggml_backend_alloc_ctx_tensors_from_buft: tensor  is too large to fit in a SYCL0 buffer (tensor size: 4831838208, max buffer size: 1073741824)
failed to allocate tensors [SYCL0]

Arc 770 Log:

IM2COL(type_input=f32,type_kernel=f16,dst_type=f16,ne_input=[1024,1024,256,1],ne_kernel=[3,3,256,1],s0=1,s1=1,p0=1,p1=1,d0=1,d1=1,is_2D=1): failed to allocate tensors [SYCL0] 
IM2COL(type_input=f32,type_kernel=f16,dst_type=f32,ne_input=[1024,1024,256,1],ne_kernel=[3,3,256,1],s0=1,s1=1,p0=1,p1=1,d0=1,d1=1,is_2D=1): failed to allocate tensors [SYCL0]   
CONV_TRANSPOSE_1D(ne_input=[197,32,1,1],ne_kernel=[16,32,32,1],s0=1,p0=0,d0=1): ggml_backend_alloc_ctx_tensors_from_buft: tensor  is too large to fit in a SYCL0 buffer (tensor size: 4831838208, max buffer size: 4294959104)
MUL_MAT(type_a=f16,type_b=f16,m=512,n=262144,k=9216,bs=[1,1],nr=[1,1]): failed to allocate tensors [SYCL0]   MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): ggml_backend_alloc_ctx_tensors_from_buft: tensor  is too large to fit in a SYCL0 buffer (tensor size: 4831838208, max buffer size: 4294959104

[slaren]

Do not add backend-specific code or tests to test-backend-ops.

[NeoZhangJianyu]

I meet an error on Arc770 (16GB)

 IM2COL(type_input=f32,type_kernel=f16,dst_type=f16,ne_input=[1024,1024,256,1],ne_kernel=[3,3,256,1],s0=1,s1=1,p0=1,p1=1,d0=1,d1=1,is_2D=1): ggml_backend_alloc_ctx_tensors_from_buft: tensor  is too large to fit in a SYCL0 buffer (tensor size: 4831838208, max buffer size: 4294959104)
failed to allocate tensors [SYCL0]   IM2COL(type_input=f32,type_kernel=f16,dst_type=f32,ne_input=[1024,1024,256,1],ne_kernel=[3,3,256,1],s0=1,s1=1,p0=1,p1=1,d0=1,d1=1,is_2D=1): ggml_backend_alloc_ctx_tensors_from_buft: tensor  is too large to fit in a SYCL0 buffer (tensor size: 9663676416, max buffer size: 4294959104)

[zhentaoyu]

ok, I'll try to fix it.

[zhentaoyu]

Hi, @NeoZhangJianyu, @Alcpz. After discussing with @airMeng, I decide to comment these new test cases since the backend_buffer thing cannot be handled well in ARC770 for now. And it may break other backends tests as well for the same reason. I have tested them successfully in Intel(R) Data Center GPU Max 1100 (sycl backend, this branch) and NV A30 (cuda backend, master and this branch). As for ARC-770, I will find some time to verify stable-diffusion models with large output img and try to fix. Thanks.

[zhentaoyu]

Hi, @slaren, Could you please take a look at the newest commit 9262dcb if you have free time?

[NeoZhangJianyu]

@zhentaoyu
I'd like to see your improvement in this feature.

It's OK, not add new case for it if you make the existed cases are passed.

Thank you!

[NeoZhangJianyu]

Is it possible to add a unit test case for this case?

Yes. I will add some im2col test_cases in test-backend-ops.cpp. And try to add ggml_get_to_fp32_sycl test case in mul_mat. Is that ok for you @NeoZhangJianyu?

Yes, it's OK!