Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark mandatory conformance tests for HW plugins #26015

Closed
wants to merge 36 commits into from

Conversation

panhaiqi
Copy link

@panhaiqi panhaiqi commented Aug 12, 2024

Details:

  • Mark OVHoldersTestOnImportedNetwork and OVClassQueryModelTest mandatory for HW plugin and optional for meta-plugin
  • Remove the meta-plugin skipping in test cases

Tickets:

@panhaiqi panhaiqi requested review from a team as code owners August 12, 2024 02:11
@panhaiqi panhaiqi added the WIP work in progress label Aug 12, 2024
@github-actions github-actions bot added category: IE Tests OpenVINO Test: plugins and common and removed WIP work in progress labels Aug 12, 2024
@panhaiqi panhaiqi marked this pull request as draft August 12, 2024 02:17
@panhaiqi panhaiqi marked this pull request as ready for review August 23, 2024 03:20
@panhaiqi panhaiqi added the WIP work in progress label Aug 23, 2024
@panhaiqi panhaiqi requested a review from a team as a code owner September 11, 2024 09:18
@github-actions github-actions bot added the category: HETERO OpenVINO HETERO plugin label Sep 11, 2024
@panhaiqi panhaiqi removed the WIP work in progress label Sep 23, 2024
@peterchen-intel peterchen-intel changed the title Move query model test to optional conformance for meta-plugin Mark mandatory conformance tests for HW plugins Sep 23, 2024
OVClassQueryModelTest,
::testing::Values(ov::test::utils::target_device));
::testing::Values(ov::test::utils::target_device),
MARK_MANDATORY_API_FOR_HW_DEVICE(NULL_getTestCaseName));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please explain the technical details of the changes. I why you create NULLTestCaseName for sw plugin... Is there any other approaches? I asked you because it is not so clear...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is to support 2 types test cases with unify MACRO:
Some API test cases have implemented getTestCaseName()
For example OVHoldersTestOnImportedNetwork::getTestCaseName
Some are not. For example OVGetMetricPropsTest

Another option, define 2 MACROs:
MARK_MANDATORY_API_FOR_HW_DEVICE_0 without parameter.
MARK_MANDATORY_API_FOR_HW_DEVICE_1 with parameter

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to rename as WITH_PARAM and WITHOUT_PARAM

In general, LGTM

@peterchen-intel
Copy link
Contributor

Replaced by #27197

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: HETERO OpenVINO HETERO plugin category: IE Tests OpenVINO Test: plugins and common conformance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants