[RISCV64] [GSoC] Integrate SHL eltwise ops into OV #25674

BHbean · 2024-07-22T15:59:19Z

Details:

integrate eltwise ops from shl lib: Add, Sub, Mul, Div, Maximum, Minimum, Exp, ReLu, LeakyReLu and PReLu
PR to SHL repository: [Bug Fix] [GSoC] add broadcast mechanism before calculating PReLu shl#5

Tickets:

N/A

src/plugins/intel_cpu/src/nodes/executors/shl/shl.hpp

src/plugins/intel_cpu/src/nodes/eltwise.h

src/plugins/intel_cpu/src/nodes/eltwise.cpp

src/plugins/intel_cpu/src/nodes/executors/shl/shl_eltwise.hpp

src/plugins/intel_cpu/src/nodes/eltwise.cpp

src/plugins/intel_cpu/src/nodes/executors/shl/shl_eltwise.cpp

src/plugins/intel_cpu/src/nodes/eltwise.cpp

src/plugins/intel_cpu/src/nodes/executors/shl/shl_eltwise.cpp

src/plugins/intel_cpu/tests/functional/custom/single_layer_tests/classes/conversion.cpp

...gins/intel_cpu/tests/functional/custom/subgraph_tests/src/common/concat_conv_sum_inplace.cpp

src/plugins/intel_cpu/tests/functional/custom/subgraph_tests/src/common/eltwise_caching.cpp

src/plugins/intel_cpu/tests/functional/custom/single_layer_tests/classes/eltwise.cpp

src/plugins/intel_cpu/src/nodes/executors/shl/shl_eltwise.cpp

src/plugins/intel_cpu/tests/functional/CMakeLists.txt

src/plugins/intel_cpu/tests/functional/custom/single_layer_tests/classes/conversion.cpp

src/plugins/intel_cpu/tests/functional/custom/single_layer_tests/classes/eltwise.cpp

src/plugins/intel_cpu/tests/functional/utils/riscv64/filter_cpu_info.cpp

src/plugins/intel_cpu/src/nodes/executors/shl/shl_eltwise.hpp

src/plugins/intel_cpu/src/nodes/executors/shl/shl_eltwise.cpp

src/plugins/intel_cpu/src/nodes/executors/shl/shl_eltwise.hpp

src/plugins/intel_cpu/src/nodes/executors/shl/shl.hpp

a-sidorova · 2024-08-06T05:00:47Z

build_jenkins

a-sidorova · 2024-08-07T04:18:57Z

build_jenkins

a-sidorova · 2024-08-07T04:32:58Z

@BHbean ARM and x64 builds are failed with the following problem:

[2024-08-07T04:28:28.277Z] /Users/sys_k8sworker/ci-openvino/workspace/private-ci/ie/build-linux-macos/b/repos/openvino/src/plugins/intel_cpu/src/nodes/eltwise.cpp:3183: Redundant blank line at the end of a code block should be deleted. [whitespace/blank_line] [3]
[2024-08-07T04:28:28.277Z] /Users/sys_k8sworker/ci-openvino/workspace/private-ci/ie/build-linux-macos/b/repos/openvino/src/plugins/intel_cpu/src/nodes/eltwise.cpp:3185: Namespace should be terminated with "// namespace node" [readability/namespace] [5]
[2024-08-07T04:28:28.277Z] /Users/sys_k8sworker/ci-openvino/workspace/private-ci/ie/build-linux-macos/b/repos/openvino/src/plugins/intel_cpu/src/nodes/eltwise.cpp:3186: Namespace should be terminated with "// namespace intel_cpu" [readability/namespace] [5]
[2024-08-07T04:28:28.277Z] Done processing /Users/sys_k8sworker/ci-openvino/workspace/private-ci/ie/build-linux-macos/b/repos/openvino/src/plugins/intel_cpu/src/nodes/eltwise.cpp
[2024-08-07T04:28:28.277Z] Category 'readability/namespace' errors found: 2
[2024-08-07T04:28:28.277Z] Category 'whitespace/blank_line' errors found: 1
[2024-08-07T04:28:28.277Z] Total errors found: 3
[2024-08-07T04:28:28.277Z]
[2024-08-07T04:28:28.277Z] CMake Error at /Users/sys_k8sworker/ci-openvino/workspace/private-ci/ie/build-linux-macos/b/repos/openvino/cmake/developer_package/cpplint/cpplint_run.cmake:53 (message):
[2024-08-07T04:28:28.277Z] [cpplint] Code style check failed for :
[2024-08-07T04:28:28.277Z] /Users/sys_k8sworker/ci-openvino/workspace/private-ci/ie/build-linux-macos/b/repos/openvino/src/plugins/intel_cpu/src/nodes/eltwise.cpp
[2024-08-07T04:28:28.277Z]
[2024-08-07T04:28:28.277Z]
[2024-08-07T04:28:28.277Z] make[2]: *** [src/plugins/intel_cpu/cpplint/src/nodes/eltwise.cpp.cpplint] Error 1
[2024-08-07T04:28:28.277Z] make[2]: *** Deleting file `src/plugins/intel_cpu/cpplint/src/nodes/eltwise.cpp.cpplint'
[2024-08-07T04:28:28.277Z] make[1]: *** [src/plugins/intel_cpu/CMakeFiles/openvino_intel_cpu_plugin_cpplint.dir/all] Error 2

Could you please fix code style?

a-sidorova · 2024-08-09T05:17:18Z

build_jenkins

a-sidorova · 2024-08-09T08:57:00Z

build_jenkins

a-sidorova · 2024-08-09T11:43:52Z

build_jenkins

a-sidorova

Thank you for the contribution to OpenVINO! 🎉

src/plugins/intel_cpu/src/nodes/eltwise.cpp

[RISCV64] add shlExecutor option for eltwise node [RISCV64] fix some errors with eltwise executor [RISCV64] add constructor def and vector initialization for ShlTensor [RISCV64] remove redundant debug print [RISCV64] skip failed tests [RISCV64] change way of invoking kernels [RISCV64] set shapes for ShlTensor before SHL add [RISCV64] simplify eltwise kernel invocation [RISCV64] integrate other eltwise ops [RISCV64] fix tests for some eltwise and activation ops [RISCV64] integrate PRelu and LeakyRelu op and fix related tests [RISCV64] integrate Maximum and Minimum op for the need of some tests [RISCV64] fix some tests [RISCV64] fix inaccurate problem with LeakyReLu op [RISCV64] fix some tests [RISCV64] enable debugging for riscv64 [RISCV64] update due to API changes

a-sidorova · 2024-08-14T04:59:51Z

build_jenkins

dmitry-gorokhov

No blocking issue from my perspective. Let follow-up remaining comments in separate PR (if needed)

dmitry-gorokhov · 2024-08-14T07:32:33Z

src/plugins/intel_cpu/src/nodes/executors/shl/shl_eltwise.cpp

+namespace ov {
+namespace intel_cpu {
+
+inline void log_unsupported_prec(const std::vector<MemoryDescPtr>& srcDescs,


This function seems to be unused. Could you please delete?

dmitry-gorokhov · 2024-08-14T07:39:33Z

src/plugins/intel_cpu/src/nodes/executors/shl/shl_eltwise.cpp

+    for (const auto& srcDesc : srcDescs) {
+        csinn_layout_enum supportedLayout = getShlDataLayoutByMemoryDesc(srcDesc);
+        switch (eltwiseAttrs.algorithm) {
+            case Algorithm::EltwisePrelu:
+                // SHL PRelu op only supports these two kinds of layout
+                if (!(supportedLayout == csinn_layout_enum::CSINN_LAYOUT_NC1HWC0 || supportedLayout == csinn_layout_enum::CSINN_LAYOUT_NCHW)) {
+                    DEBUG_LOG("src descriptor layout is unsupported by SHL Prelu op: ", srcDesc->serializeFormat());
+                    return false;
+                }
+                break;
+            default:
+                if (supportedLayout == csinn_layout_enum::CSINN_LAYOUT_NULL) {
+                    DEBUG_LOG("src descriptor layout is unsupported by SHL: ", srcDesc->serializeFormat());
+                    return false;
+                }
+                continue;
+        }
+    }
+    for (const auto& dstDesc : dstDescs) {
+        if (getShlDataLayoutByMemoryDesc(dstDesc) == csinn_layout_enum::CSINN_LAYOUT_NULL) {
+            DEBUG_LOG("dst descriptor layout is unsupported by SHL: ", dstDesc->serializeFormat());
+            return false;
+        }
+    }


I see that Eltwise node creates only planar layoit desc (nchw) while this code checks doesn't prohibit nhwc layout. So what is really supported by SHL?

I think SHL Eltwise node supports only planar layout by creating descriptors with only nchw layout. This part of code does not block any other layouts as the eltwise ops in SHL does not constrain the layouts of their input and output (expect PRelu). So in the future more layouts can be supported by creating descriptors with different layouts under different situations.

BHbean requested review from a team as code owners July 22, 2024 15:59

github-actions bot added the category: CPU OpenVINO CPU plugin label Jul 22, 2024

sys-openvino-ci added the ExternalPR External contributor label Jul 22, 2024

a-sidorova self-assigned this Jul 23, 2024

dmitry-gorokhov added this to the 2024.4 milestone Jul 23, 2024

a-sidorova reviewed Jul 25, 2024

View reviewed changes

a-sidorova mentioned this pull request Jul 29, 2024

[RISCV64] [GSoC] Integrate SHL eltwise ops into OV a-sidorova/openvino#214

Closed

a-sidorova reviewed Jul 31, 2024

View reviewed changes

BHbean requested a review from a team as a code owner August 2, 2024 08:27

github-actions bot added the category: build OpenVINO cmake script / infra label Aug 2, 2024

a-sidorova reviewed Aug 2, 2024

View reviewed changes

a-sidorova reviewed Aug 4, 2024

View reviewed changes

BHbean force-pushed the shl-executor branch 2 times, most recently from b5e3cd7 to 68cc344 Compare August 5, 2024 14:03

BHbean force-pushed the shl-executor branch from 0e81e49 to c57d9db Compare August 6, 2024 15:01

BHbean force-pushed the shl-executor branch from 7cb5574 to 9e93d51 Compare August 9, 2024 02:16

a-sidorova approved these changes Aug 12, 2024

View reviewed changes

src/plugins/intel_cpu/src/nodes/eltwise.cpp Show resolved Hide resolved

a-sidorova assigned dmitry-gorokhov Aug 12, 2024

BHbean added 4 commits August 13, 2024 13:54

[RISCV64] fix and disable some tests

31c8e3a

[RISCV64] remove some comments and disable a test

c51a7d7

[RISCV64] update according to the review comments

5145ba9

BHbean added 12 commits August 13, 2024 13:54

[RISCV64] initialize execution func inside Executor::init process

c059a15

[RISCV64] align initialization behaviors with execution behaviors

e9091ef

[RISCV64] integrate eltwise clamp op

1f7dac4

[RISCV64] implement filter_cpu_info for rv64

afa1769

[RISCV64] init ShlTensor with static shapes

95023d3

[RISCV64] reformat based on comments

d3ddc37

[RISCV64] reformat based on comments

9a379ce

[RISCV64] better IShlParams class definition

13c7b83

[RISCV64] fix bugs in shl PRelu

02fa229

[RISCV64] fix code style

0b2722b

[RISCV64] fix code style on x86 and arm

b434b9f

[RISCV64] update shl submodule due to rebase

fb517e5

BHbean force-pushed the shl-executor branch from fab0b74 to fb517e5 Compare August 13, 2024 13:56

[RISCV64] update shl submodule due to rebase

508df88

dmitry-gorokhov approved these changes Aug 14, 2024

View reviewed changes

dmitry-gorokhov added this pull request to the merge queue Aug 14, 2024

Merged via the queue into openvinotoolkit:master with commit 25eac04 Aug 14, 2024
134 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RISCV64] [GSoC] Integrate SHL eltwise ops into OV #25674

[RISCV64] [GSoC] Integrate SHL eltwise ops into OV #25674

BHbean commented Jul 22, 2024 •

edited by a-sidorova

Loading

a-sidorova commented Aug 6, 2024

a-sidorova commented Aug 7, 2024

a-sidorova commented Aug 7, 2024 •

edited

Loading

a-sidorova commented Aug 9, 2024

a-sidorova commented Aug 9, 2024

a-sidorova commented Aug 9, 2024

a-sidorova left a comment

a-sidorova commented Aug 14, 2024

dmitry-gorokhov left a comment

dmitry-gorokhov Aug 14, 2024

BHbean Aug 14, 2024

dmitry-gorokhov Aug 14, 2024

BHbean Aug 14, 2024

[RISCV64] [GSoC] Integrate SHL eltwise ops into OV #25674

[RISCV64] [GSoC] Integrate SHL eltwise ops into OV #25674

Conversation

BHbean commented Jul 22, 2024 • edited by a-sidorova Loading

Details:

Tickets:

a-sidorova commented Aug 6, 2024

a-sidorova commented Aug 7, 2024

a-sidorova commented Aug 7, 2024 • edited Loading

a-sidorova commented Aug 9, 2024

a-sidorova commented Aug 9, 2024

a-sidorova commented Aug 9, 2024

a-sidorova left a comment

Choose a reason for hiding this comment

a-sidorova commented Aug 14, 2024

dmitry-gorokhov left a comment

Choose a reason for hiding this comment

dmitry-gorokhov Aug 14, 2024

Choose a reason for hiding this comment

BHbean Aug 14, 2024

Choose a reason for hiding this comment

dmitry-gorokhov Aug 14, 2024

Choose a reason for hiding this comment

BHbean Aug 14, 2024

Choose a reason for hiding this comment

BHbean commented Jul 22, 2024 •

edited by a-sidorova

Loading

a-sidorova commented Aug 7, 2024 •

edited

Loading