Skip to content

Commit

Permalink
Disable the failing opset 18 model tests that are breaking the packag…
Browse files Browse the repository at this point in the history 
…ing pipeline (#14259)

### Description
<!-- Describe your changes. -->
Skip tests for opset18 models that we haven't implemented kernels for
yet.

Slice was checked in today so those failures should go away.

Resize: #13890 (all resize failures are fixed by this PR as confirmed in
output
[here](https://dev.azure.com/aiinfra/530acbc4-21bc-487d-8cd8-348ff451d2ff/_apis/build/builds/264725/logs/729))
Col2Im: #12311
ScatterND and ScatterElement: #14224
Pad (should also fix CenterCropPad failures): #14219 Bitwise ops: #14197
Optional: Unknown if we're intending to support this in 1.14

Not sure about SoftPlus as that is failing due to `Could not find an
implementation for Exp(1)`. ORT supports Exp from opset 6 and on, and it
seems incorrect for the test model created for opset 18 to be using a
version of Exp that is so old. Would have expected it to use the latest
- Exp(13). @liqunfu is this something that requires a fix to the ONNX
model?


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Fix pipeline
  • Loading branch information
@skottmckay
skottmckay authored Jan 12, 2023
1 parent c9a53c9 commit ea12b67
Showing 1 changed file with 67 additions and 1 deletion.
68 changes: 67 additions & 1 deletion csharp/test/Microsoft.ML.OnnxRuntime.Tests.NetCoreApp/InferenceTest.netcore.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -380,7 +380,73 @@ private static Dictionary<string, string> GetSkippedModels(DirectoryInfo modelsD
{ "test_sequence_map_add_2_sequences", "sequence type is not supported in test infra." },
{ "test_sequence_map_identity_1_sequence", "sequence type is not supported in test infra." },
{ "BERT-Squad-int8", "training domain"},
{ "YOLOv3-12-int8", "training_domain"}
{ "YOLOv3-12-int8", "training_domain"},
// opset 18 models. these should be supported by ORT 1.14 when released
{ "test_bitwise_and_i16_3d", "pending opset 18 support"},
{ "test_bitwise_and_i32_2d", "pending opset 18 support"},
{ "test_bitwise_and_ui64_bcast_3v1d", "pending opset 18 support"},
{ "test_bitwise_and_ui8_bcast_4v3d", "pending opset 18 support"},
{ "test_bitwise_not_2d", "pending opset 18 support"},
{ "test_bitwise_not_3d", "pending opset 18 support"},
{ "test_bitwise_not_4d", "pending opset 18 support"},
{ "test_bitwise_or_i16_4d", "pending opset 18 support"},
{ "test_bitwise_or_i32_2d", "pending opset 18 support"},
{ "test_bitwise_or_ui64_bcast_3v1d", "pending opset 18 support"},
{ "test_bitwise_or_ui8_bcast_4v3d", "pending opset 18 support"},
{ "test_bitwise_xor_i16_3d", "pending opset 18 support"},
{ "test_bitwise_xor_i32_2d", "pending opset 18 support"},
{ "test_bitwise_xor_ui8_bcast_4v3d", "pending opset 18 support"},
{ "test_bitwise_xor_ui64_bcast_3v1d", "pending opset 18 support"},
{ "test_center_crop_pad_crop", "pending opset 18 support"},
{ "test_center_crop_pad_crop_and_pad", "pending opset 18 support"},
{ "test_center_crop_pad_crop_and_pad_expanded", "pending opset 18 support"},
{ "test_center_crop_pad_crop_axes_chw", "pending opset 18 support"},
{ "test_center_crop_pad_crop_axes_chw_expanded", "pending opset 18 support"},
{ "test_center_crop_pad_crop_axes_hwc", "pending opset 18 support"},
{ "test_center_crop_pad_crop_axes_hwc_expanded", "pending opset 18 support"},
{ "test_center_crop_pad_crop_expanded", "pending opset 18 support"},
{ "test_center_crop_pad_pad", "pending opset 18 support"},
{ "test_center_crop_pad_pad_expanded", "pending opset 18 support"},
{ "test_col2im", "pending opset 18 support"},
{ "test_col2im_5d", "pending opset 18 support"},
{ "test_col2im_dilations", "pending opset 18 support"},
{ "test_col2im_pads", "pending opset 18 support"},
{ "test_col2im_strides", "pending opset 18 support"},
{ "test_constant_pad", "pending opset 18 support"},
{ "test_constant_pad_axes", "pending opset 18 support"},
{ "test_edge_pad", "pending opset 18 support"},
{ "test_reflect_pad", "pending opset 18 support"},
{ "test_resize_downsample_scales_cubic_antialias", "pending opset 18 support"},
{ "test_resize_downsample_scales_linear_antialias", "pending opset 18 support"},
{ "test_resize_downsample_sizes_cubic_antialias", "pending opset 18 support"},
{ "test_resize_downsample_sizes_linear_antialias", "pending opset 18 support"},
{ "test_resize_downsample_sizes_nearest_not_larger", "pending opset 18 support"},
{ "test_resize_downsample_sizes_nearest_not_smaller", "pending opset 18 support"},
{ "test_resize_tf_crop_and_resize_axes_2_3", "pending opset 18 support"},
{ "test_resize_tf_crop_and_resize_axes_3_2", "pending opset 18 support"},
{ "test_resize_upsample_scales_nearest_axes_2_3", "pending opset 18 support"},
{ "test_resize_upsample_scales_nearest_axes_3_2", "pending opset 18 support"},
{ "test_resize_upsample_sizes_nearest_axes_2_3", "pending opset 18 support"},
{ "test_resize_upsample_sizes_nearest_axes_3_2", "pending opset 18 support"},
{ "test_resize_upsample_sizes_nearest_not_larger", "pending opset 18 support"},
{ "test_scatter_elements_with_axis", "pending opset 18 support"},
{ "test_scatter_elements_without_axis", "pending opset 18 support"},
{ "test_scatter_elements_with_duplicate_indices", "pending opset 18 support"},
{ "test_scatter_elements_with_negative_indices", "pending opset 18 support"},
{ "test_scatter_elements_with_reduction_max", "pending opset 18 support"},
{ "test_scatter_elements_with_reduction_min", "pending opset 18 support"},
{ "test_scatternd", "pending opset 18 support"},
{ "test_scatternd_add", "pending opset 18 support"},
{ "test_scatternd_max", "pending opset 18 support"},
{ "test_scatternd_min", "pending opset 18 support"},
{ "test_scatternd_multiply", "pending opset 18 support"},
{ "test_softplus_example_expanded", "pending opset 18 support"},
{ "test_softplus_expanded", "pending opset 18 support"},
{ "test_optional_get_element_optional_sequence", "pending opset 18 support"},
{ "test_optional_get_element_optional_tensor", "pending opset 18 support"},
{ "test_optional_has_element_empty_optional_input", "pending opset 18 support"},
{ "test_optional_has_element_optional_input", "pending opset 18 support"},
{ "test_optional_has_element_tensor_input", "pending opset 18 support"},
};

// The following models fails on nocontribops win CI
Expand Down

0 comments on commit ea12b67

Please sign in to comment.