Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable the failing opset 18 model tests that are breaking the packaging pipeline #14259

Merged
merged 1 commit into from
Jan 12, 2023

Conversation

skottmckay
Copy link
Contributor

@skottmckay skottmckay commented Jan 12, 2023

Description

Skip tests for opset18 models that we haven't implemented kernels for yet.

Slice was checked in today so those failures should go away.

Resize: #13890 (all resize failures are fixed by this PR as confirmed in output here)
Col2Im: #12311
ScatterND and ScatterElement: #14224
Pad (should also fix CenterCropPad failures): #14219 Bitwise ops: #14197
Optional: Unknown if we're intending to support this in 1.14

Not sure about SoftPlus as that is failing due to Could not find an implementation for Exp(1). ORT supports Exp from opset 6 and on, and it seems incorrect for the test model created for opset 18 to be using a version of Exp that is so old. Would have expected it to use the latest - Exp(13). @liqunfu is this something that requires a fix to the ONNX model?

Motivation and Context

Fix pipeline

…ing pipeline.

Slice(was checked in today.
Resize: #13890
Col2Im: #12311
ScatterND and ScatterElement: #14224
Pad (should also fix CenterCropPad failures): #14219
Bitwise ops: #14197
Optional: Unknown if we're intending to support this in 1.14

Not sure about SoftPlus as that failing due to `Could not find an implementation for Exp(1)`. ORT supports opset 6 and on, so it seems incorrect for the test model created in opset 18 to be using a version of Exp that is so old.
@liqunfu
Copy link
Contributor

liqunfu commented Jan 12, 2023

I see softplus since version is 1. It is a function op so the test case has a model with opset import = 1. This in turn requires primary ops used to define the function body to be from version 1. This is a cause of the failure. From ONNX side, it shall consider generate test cases for different opset import versions so that Ort can choose test cases with support model. We just started function body versioning in 1.13.
BTW it is a difficult issue: there are many function bodies needed. See this code for details:

I suggest disable this test for now.
onnx/onnx#4771

@skottmckay
Copy link
Contributor Author

I see softplus since version is 1. It is a function op so the test case has a model with opset import = 1. This in turn requires primary ops used to define the function body to be from version 1. This is a cause of the failure. From ONNX side, it shall consider generate test cases for different opset import versions so that Ort can choose test cases with support model. We just started function body versioning in 1.13. BTW it is a difficult issue: there are many function bodies needed. See this code for details:

I suggest disable this test for now. onnx/onnx#4771

I'm am guessing based on the test names having 'expanded' in them that the test model doesn't actually have a Softplus node - it instead has nodes from the expansion of the Softplus function definition. If it had a Softplus node our kernel would be used and there would be no function expansion required.

Why do we have a test models with the expansion of function based nodes? Not sure what the purpose is. Will need to permanently disable these tests if we keep them.

@skottmckay skottmckay merged commit ea12b67 into main Jan 12, 2023
@skottmckay skottmckay deleted the skottmckay/DisableOpset18OnnxModelTestsTemporarily branch January 12, 2023 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants