Es/lpt/lpt to ngraph fixes2 with master #2671

eshoguli · 2020-10-14T23:27:19Z

No description provided.

…ation

…n LowPrecisionTransformer

…/eshoguli/openvino into es/lpt/lpt_to_ngraph_integration

…aph_integration

…/eshoguli/openvino into es/lpt/lpt_to_ngraph_integration

vladimir-paramuzov · 2020-10-20T14:46:38Z

inference-engine/src/cldnn_engine/cldnn_engine.cpp

+                LayerTransformation::QuantizedTensorAlignment::UpdateLevel,  // quantizedTensorAlignmentOnActivations
+                LayerTransformation::QuantizedTensorAlignment::None,  // quantizedTensorAlignmentOnWeights
+                true);  // supportAsymmetricQuantization
+            LowPrecisionTransformer transformer(LowPrecisionTransformer::getAllTransformations(params)


@eshoguli, @slyalin, @GlebKazantaev why the API for low precision transformation is completely differs from common transformations?
We use common transforms as follows:

manager.register_pass<ngraph::pass::ConvertOpSet2ToOpSet1>(); auto pass_config = manager.get_pass_config(); pass_config->disable<ngraph::pass::ConvertGELU>(); pass_config->set_callback<ngraph::pass::ConvertReduceMaxToPooling>( [](const_node_ptr &node) -> bool { return disableReduceDecomposition<ngraph::opset1::ReduceMax>(node); }); manager.run_passes(nGraphFunc); // etc

So I'd expect LPT to be used as follows:

manager.register_pass<ngraph::pass::LowPrecisionTransformations>(); pass_config.set_callback<ngraph::pass:MatMulTransformation>(/* callback that checks asymmetric quantization */); or pass_config.disable<ngraph::pass:MatMulTransformationAsymmetric>(); manager.run_passes(nGraphFunc);

It it possible to align LPT with other transforms?

vladimir-paramuzov · 2020-10-20T15:11:16Z

inference-engine/src/cldnn_engine/cldnn_program.cpp


-        // [WA part2] Try to find non-quantized layers and convert them back to FP16
+    // [WA part2] Try to find non-quantized layers and convert them back to FP16
+    if (config.enableInt8) {


If I understand correctly, you run this fp16 fallback stuff only for USE_CNNNETWORK_LPT path which will lead to regressions on GPU with new LPT.

vladimir-paramuzov · 2020-10-20T15:12:21Z

inference-engine/src/cldnn_engine/cldnn_engine.cpp

@@ -72,8 +77,10 @@ cldnn::device_info clDNNEngine::GetDeviceInfo(const std::map<std::string, std::s
    return device_info;
 }

-InferenceEngine::ICNNNetwork::Ptr clDNNEngine::CloneAndTransformNetwork(const InferenceEngine::ICNNNetwork& network) const {
+InferenceEngine::ICNNNetwork::Ptr clDNNEngine::CloneAndTransformNetwork(const InferenceEngine::ICNNNetwork& network, CLDNNPlugin::Config config) const {


Use const ref for the config.

vladimir-paramuzov · 2020-10-20T15:14:14Z

inference-engine/src/cldnn_engine/cldnn_program.cpp

+    auto input0 = getInputTo(inputsMap.begin()->second->getInputData());
+
+    bool baselineIsFP16 = false;
+    if (input0.begin()->second->params.count("FP16") != 0) {


If the results of getInputTo is empty map, then this code will crash with segmentation fault.

vladimir-paramuzov · 2020-10-20T15:18:04Z

...y_api/include/legacy/transformations/convert_opset1_to_legacy/convert_mul_or_add_finally.hpp

@@ -77,7 +80,10 @@ ngraph::graph_rewrite_callback get_callback() {
        const auto output_shape = lin_op->output(0).get_partial_shape();
        const auto output_shape_rank = output_shape.rank().get_length();

-        if (!lin_op->get_element_type().is_real()) {
+        const auto intInputs = !lin_op->get_input_element_type(0).is_real() &&


Why you didn't use is_integral_number()?

vladimir-paramuzov · 2020-10-20T15:22:53Z

inference-engine/src/mkldnn_plugin/mkldnn_exec_network.cpp

 #include <threading/ie_cpu_streams_executor.hpp>
 #include <ie_system_conf.h>
 #include <threading/ie_thread_affinity.hpp>
 #include <algorithm>
 #include <unordered_set>
 #include <utility>
+#include <cstring>


vladimir-paramuzov · 2020-10-20T15:43:16Z

inference-engine/src/transformations/include/transformations/low_precision/iparams_manager.hpp

+/**
+ * @brief low precision transformation component interface.
+  */
+class TRANSFORMATIONS_API IParamsManager {


Do we really need this interface? What's the purpose of it? I found 1 class that implements this interface, so why do we need extra abstraction level?

vladimir-paramuzov · 2020-10-20T15:53:13Z

inference-engine/src/transformations/include/transformations/low_precision/network_helper.hpp

+class TRANSFORMATIONS_API NetworkHelper {
+public:
+    // Return true if `type` can be castable to at least one of `type`
+    static bool is_castable_to_one_of(NodeTypeInfo type, const std::unordered_set<NodeTypeInfo>& types);


Do we really need all these helper methods here?
As I can see is_castable_to_one_of is used only once in getParentsRecursivelyExceptTypes method which is always called with empty types container. So why do we need this helper?

Can't consumer_inputs and consumers with default ngraph::Node methods such as inputs() and outputs?

and so on.

vladimir-paramuzov · 2020-10-20T15:56:01Z

...rc/transformations/include/transformations/low_precision/weightable_layer_transformation.hpp

+
+protected:
+    DataPrecision decomposeFakeQuantizeForWeightsPath(std::shared_ptr<Node> weightableLayer) const;
+    static bool isGroup(const std::shared_ptr<Node>& node);


isGrouped maybe?

vladimir-paramuzov · 2020-10-20T15:57:13Z

...ence-engine/src/transformations/include/transformations/rt_info/dequantization_attribute.hpp

+
+/**
+ * @brief Defines fused names attribute
+ * @file fused_names_attribute.hpp


The docs above must be updated.

vladimir-paramuzov · 2020-10-20T16:02:28Z

inference-engine/src/transformations/src/transformations/low_precision/clamp.cpp

+void ClampTransformation::registerMatcherIn(GraphRewrite& pass, TransformationContext& context) const {
+    addPattern(pass,
+               context,
+               make_op_pattern<opset1::Clamp>({ make_op_label<opset1::Multiply>() }));


And once again a lot of custom stuff is used for pattern matching. Can it be unified with other transformations?

vladimir-paramuzov · 2020-10-21T06:42:19Z

@eshoguli folks already mentioned that it's absolutely impossible to review these changes properly (39k changed lines, ~800 commits, no docs, etc). I just want to give some tips for the future:

Such a huge changes with a lot of complex logic must be split into several patches. Ideally several PRs should be created, but if we can't merge some parts of the changes separately, then you can just maintain several commits with more-or-less independent changes (i.e. tests in separate commit and stuff like that).
Use git rebase instead of merge to avoid useless merge commits in the branch history
Squash useless commits (code style fix, compilation errors fix, other kind of minor fixes)
Use interactive rebase to clean-up history on your branch (squash commits, change patches order, rename something)

ilya-lavrenov · 2020-10-21T09:40:55Z

Use git rebase instead of merge to avoid useless merge commits in the branch history
Squash useless commits (code style fix, compilation errors fix, other kind of minor fixes)
Use interactive rebase to clean-up history on your branch (squash commits, change patches order, rename something)

After merge of this branch we will have a single commit anyway.

vladimir-paramuzov · 2020-10-21T10:20:24Z

Use git rebase instead of merge to avoid useless merge commits in the branch history
Squash useless commits (code style fix, compilation errors fix, other kind of minor fixes)
Use interactive rebase to clean-up history on your branch (squash commits, change patches order, rename something)

After merge of this branch we will have a single commit anyway.

Sure, but good commits structure in the branch might significantly simplify review process as we can use filters by commit in PR and check different logical parts of these changes separately.

… is not fused

…_master_review fixes to master merge branch

will be fixed

* [LPT] Replace creation of dequantization with factory * [ngraph][LPT] Add ScaleShift replace for dequantization operations * [LPT] SubtractMultiplyToMultiplyAdd refactoring * [LPT] Code style fix * [LPT] Edit SubtractMultiplyToMultiplyAdd transformation for dequantization * [LPT] Linux compilation quick fix * [LPT] [WIP] runtime info applying * [LPT] Concat transformation functional tests extending * [LPT] MultiplyToConvolution + Subtract to add fusing + improvements in LowPrecisionTransformer * [LPT] linux compilation error fix * [LPT] compilation error * [LPT] MultiplyToGroupConvolution fix: 5D support * [LPT] Multiply transformation extending: FQ weights support - wip * [LPT] FQ folding & precision selection * [LPT] code style fixes * [LPT] code style fixes * [LPT] Linux compilation error fix * [LPT] SubtractMultiplyToMultiplyAdd: refactoring * [LPT] Tests fixes * [LPT] MultiplyToGroupConvolution tests * [LPT] Convert subtract with int inputs to Eltwise sub * [LPT] Constant folding fix for quant models * [LPT] 1) Asymmetric quantization improvement 2) tests extending * [LPT] 2 fixes for se_resnext_50 * [LPT] Add transformation priority branch selection test * [LPT] AddMultiplyFusion: legacy transformation quick fix * [LPT] nGraph tests temporary disabling * [LPT] Fix for eltwise inputs with multiple outputs * [LPT] Fix for FQ fuse * [LPT] Reshape by channel, batch temporary disabled * [nGraph][LPT] MatMul fix for reading FP16 models * [LPT] 1) Add (not after Convolution/GroupConvolution/MatMul with Constant) to Subtract 2) precision selection fix: MultiplyToGroupConvolution quick fix * [LPT] DenseNet improvments: AddTransformation: Add to Subtract + tests * [LPT] AddTransformarion refactoring * [LPT] AddTransformation tests temporay disabled * [LPT] ReshapeTransformation improvements: degradation fix * [LPT] code style fix * [LPT] Concat tests temporary disabling * [LPT] tests unification 1) plugin tests: added test-cases and nGraph-validation for clamp, split and variadic split 2) func tests: added test-cases 3) transformNGraph: added the ability to run additional transformations * [LPT] split & variadic split merge fix * [LPT] Clamp: added support for asymmetric quantization * [LPT] added DequantizationAttr run-time attribute * [LPT] debug info removal * [LPT] ConcatTransformation: zero point fix * [LPT] CNNNetwork ReLU transformation quick fix * [LPT] 1) Concat fix 2) ConcatMultiChannels fix 3) Added "Concat with Split" test-cases 4) Subgraph fix * [LPT] 1) Concat fix 2) Added "Concat with different precision on childs" test-case * [LPT] concat fix Ubuntu18 * [LPT] Concat test fixes * [LPT] Not fp32 FQ input support * [LPT] MatMul Fix + separateInStandaloneBranch Fix * [LPT] Fix reference input types in mish fusion tests * [LPT] Fix cpuFuncTests on CentOS building * [nGraph][LPT] ScaleShift 2d, 3d nGraph conversion enabling * [LPT] 1) FullyConnected workaround removing 2) validate_nodes_and_infer_types for LPT * [ngraph] Add check for childs for ConvertSubtract * [LPT] Squeeze/Unsqueeze tests unification * [LPT] Squeeze/Unsqueeze change signature for getReference/getOriginal * [LPT] Mul & Add -> ScaleShift quick fix * [LPT] nGraph tests emporary disabling * [LPT] code style fix * [LPT] code style fix #2 * [LPT] nGraph tests temporary disabling * [LPT] code styl fix #3 * [LPT] shared plugin tests temporary disabling * [LPT] cleanup * [LPT] nGraph unit_tests tests temproary disabling * [LPT] nGraph unit tests disabling #2 * [LPT] nGraph tests disabling * [LPT] nGraph tests temporary disabling * [LPT] WA removing * [LPT] CentOS compilation fix * [LPT] KMB wa to avoid compilation error * [LPT] functional test temporary disabling * [nGraph] code style fixes * [LPT] ConcatTransformation: data movement operation as intermediate handling * [LPT] FuseSubtractToFakeQuantize after VariadicSplit * [LPT] ConcatWithSplitTransformation functional test temporary disabling * [LPT] Clamp and ConcatWithDifferentPrecisionsOnChilds: tests fix * [LPT] MatMul: bert-nv-mlperf-quantized fix * [LPT] Add to convolution biases fuse fix * [LPT] GPU plugin tests fixes * [LPT] Normalize GPU plugin tests fix * [LPT] test-commit * [LPT] CLDNN Plugin FP16 conversion * [LPT] AvgPool update precision if there is not FQ after + convolution precision limitation on activation * [LPT] Convolution fixes * [LPT] FuseSubtractToFakequantize & FuseMultiplyToFakeQuantize improvement * [LPT] FuseSubtractToFakeQuantize test fix * [LPT] FuseSubtractToFakeQuantizeTransformation tests * [LPT] code style fix * [LPT] AvgPool child recursive extend * [LPT] AvgPool tests + fix * [LPT] compilation quick fix * [LPT] Add to convolution biases fuse fix * [LPT] Linux issues: MatMulWithOptimizedConstantFakeQuantizeTransformation temporary disabled * [LPT] Normalize GPU plugin tests fix * [LPT] test-commit * [LPT] 1) added the ability to create sub without dequantizationAttribute 2) fixed optimizeMulAfter: added copying rt_info 3) Tests Unification: Convolution transformation 4) added cleanRunTimeInfo into Network Helper * [LPT] Tests Unification: GroupConvolution * [LPT] removed debug info * [LPT] functional tests for Convolution & GroupConvolution extending * [LPT] [MatMul] Quick fix ubuntu error * [LPT] MatMulTransformation quick test fix: one constant for both intervals * [nGraph] code style fix * [LPT] added output_precision to NormalizeIE * [nGraph] NormalizeIE fix for LPT support * [LPT] nGraph WA removal * [LPT] fixed fillSubgraph for concat multi channels * [LPT] MatMul fix * [nGraph] WA removal: 1) nGraph tests enabling 2) LPT extanding: not handle in FP32 * [LPT] nGraph WA removal: function tests skip config rollback * [LPT] WA removal: precision propagation fix * [LPT] ConvertMulOrAddFinally transformation extending * [nGraph] ConvolutionMultiplyFusion rollback (move from legacy to common) * [nGraph] ConvertMulAddToScaleShiftOrPower: WA removal * [nGraph] TypeRelaxed: WA removal * [nGraph] WA removal: TypeRelaxed * [LPT] WA removal: ConcatTransformation * [nGraph] WA removal: Eltwise & ConvertMulOrAddFinally fixes to support LPT * [nGraph] MulAddConversion fix: 2D & 3D ScaleShift are supproted * [nGraph] VisualizeTree extending * [LPT] FakeQuantizeDequantization extending: check element wise dequantization operation * [LPT] FakeQuantizeDequantization extending: SubtractMultiplyToMultiplyAddTransformation & WeightableLayerTransformation * [LPT] Convolution + test infrastructure update * [LPT] GPU compilation error * [nGraph] BatchNorm plugin tests: input tensor definition * [LPT] LowPrecisionTransformer::isFunctionQuantized was added * [nGraph] WA final cleanup * [nGraph] ScaleShiftIE quick fix * [LPT] Functional tests: added test-cases "Concat with intermediate with constant" * [LPT] Transformer::isNetworkquantized fix * [LPT] SubtractMultiplyToMultiplyAdd zero Add remove: fix for ssd300 on gpu * [LPT] MultiplyToGroupConvolution not transform on Const * [LPT] workaround for negative scales * [LPT] Convert standalone dequantization Mul,Sub,Add to ScaleShift * [LPT] SubtractMultiplyToMultiplyAdd test fix * [LPT] Clamp transformation: GPU tests fix * [LPT] Transformer tests * [LPT] FakeQuantizePrecisionSelectionTransformation was disabled for GPU * [LPT] TransformerIsFunctionQuantized refactoring * [nGraph] code style fix * [LPT] mobilenet_v2_tf_depthwise test update * [LPT] TMP: dequantization folding * [LPT] Elementwise transformation fix: dequantization operations constant folding * [LPT] cleanup * [LPT] denormal values fix * [LPT] FuseFakeQuantize test fixed + negative multiply case * [LPT] FP32 -> FP16 conversion info * [LPT] FQ dot interval support + swapMultiplyAdd safely division * [LPT] test fix * [LPT] Tests for dot interval on FQ + tests for addTransformation enabling * [LPT] Clamp transformation fix * [LPT] FQ prec selection test fix * [LPT] Clamp test case * [LPT] Concat division precision fix * [LPT] cleanup * [LPT] merge fix * [LPT] WIP: MatMul asymmetric quantization fix (BERT) * [LPT] MatMulWithOptimizedConstantFakeQuantizeTransformation disabled * [LPT] GPU Plugin set config fix * [LPT] Fix merge mistakes * [LPT] Rollback device specific INT8 * [LPT] ReshapeFullyConnected fix: FullyConnected output fix * [LPT] bert-base-chinese GPU fix * [ngraph/LPT] Tests for fix convert_mul_or_add_finally with dequantization [ngraph/LPT] Fix convert mul_or_add_finally with dequantization * [LPT] ScaleShift dim < 4 only dequantization conversion * [LPT] MatMul transformation tests extensing * [LPT] ReshapeFullyConnected legacy transformation: LPT test case addition * [nGraph] VisualizeTree extending: property names displying to simplify search * [LPT] getDequantization extending * [LPT] MulAddToScaleshiftOrPower: out precision fix & tests * [LPT] Multiply to ScaleShiftIE: Multiply transformation: remove DEQUANTIZATION if not valid * [LPT] Concat test case * [nGraph] try to fix opencv compatibility * [nGraph] nGraph code style fix * [LPT] InPlace dequantization folding * [LPT] Multiply constant folding test * [LPT] Fix plugin test case for MatMulWithOptimizedConstantFakeQuantize [LPT] Enable MatMulWithOptimizedConstantFakeQuantize plugin test * [LPT] Convolution transformation: mulConst shape fix * [LPT] INT8 Constant folding branch for elementwise ops optimization removal * [LPT] eltwise for const branch fix * [LPT] linux fix * [LPT] Multiply test refactoring * [LPT] Convert Fuse in Constant + tests * [LPT] function comparation: runtime info comparation rollback * [LPT] linux build fix * [LPT] linux build fix2 * [LPT] MatMul transformation limitation was added to be similar as CNNNetwork LPT * [LPT] Reshape transformation update: don't broadcast by batch * [LPT] MatMul transformation limitation was added to be similar as CNNNetwork LPT - refactoring * [LPT] MatMul transformation: transpose input tensors fix * [LPT] checkElementwise for AddTransformation WA: should be moved to getDequantization * [LPT] merge fix * [LPT] MatMul fix & tests * [LPT] AddTransformation tests * [LPT] Interpolate transformation enabled * [LPT] constant folding before LPT * [LPT] WIP: not completed tests * [LPT] GPU degradation fix * [LPT] FuseConvert workaround * [LPT] code cleanup * [LPT] Interpolate GPU test quick fix * [LPT] GroupConvolution fix * [LPT] Fix fusing multiply for non-dequantization layers * [LPT] GPU pipeline update: enableInt8 initialization place update * [LPT] tests compilation fix * [LPT] merge fix * [LPT] tests enabling * [LPT] merge issue resolving * [LPT] LPT CNNNetwork usage macros: part #1: source code * [LPT] LPT CNNNetwork usage macros: part #2: cmake files update and tests addoption * [LPT] LPT workaround from nGraph core removing * [LPT] previous LPT version tests * [LPT] inference_engine_lp_transformations was returned back * [LPT] replace_node rollback * [LPT] ConvertSubtract fix * [LPT] GPU: baselineIsFP16 reuse fix * [LPT] FakeQuantizeTransformation: GPU workaround: I32 -> FP32 Convert is not fused * [LPT] AvgPool output precision workaround * [LPT] Group convolution precision + Subtract to ScaleShift const fix * [LPT] SubMulToMulAdd & Transpose: action-recognition-0001 fix * [LPT] Transpose: added test with per-tensor quantization Co-authored-by: Aleksandr Pertovsky <[email protected]> Co-authored-by: Zinoviev, Vladimir <[email protected]> Co-authored-by: Vladislav Golubev <[email protected]> Co-authored-by: Gorokhov Dmitriy <[email protected]>

* [LPT] Replace creation of dequantization with factory * [ngraph][LPT] Add ScaleShift replace for dequantization operations * [LPT] SubtractMultiplyToMultiplyAdd refactoring * [LPT] Code style fix * [LPT] Edit SubtractMultiplyToMultiplyAdd transformation for dequantization * [LPT] Linux compilation quick fix * [LPT] [WIP] runtime info applying * [LPT] Concat transformation functional tests extending * [LPT] MultiplyToConvolution + Subtract to add fusing + improvements in LowPrecisionTransformer * [LPT] linux compilation error fix * [LPT] compilation error * [LPT] MultiplyToGroupConvolution fix: 5D support * [LPT] Multiply transformation extending: FQ weights support - wip * [LPT] FQ folding & precision selection * [LPT] code style fixes * [LPT] code style fixes * [LPT] Linux compilation error fix * [LPT] SubtractMultiplyToMultiplyAdd: refactoring * [LPT] Tests fixes * [LPT] MultiplyToGroupConvolution tests * [LPT] Convert subtract with int inputs to Eltwise sub * [LPT] Constant folding fix for quant models * [LPT] 1) Asymmetric quantization improvement 2) tests extending * [LPT] 2 fixes for se_resnext_50 * [LPT] Add transformation priority branch selection test * [LPT] AddMultiplyFusion: legacy transformation quick fix * [LPT] nGraph tests temporary disabling * [LPT] Fix for eltwise inputs with multiple outputs * [LPT] Fix for FQ fuse * [LPT] Reshape by channel, batch temporary disabled * [nGraph][LPT] MatMul fix for reading FP16 models * [LPT] 1) Add (not after Convolution/GroupConvolution/MatMul with Constant) to Subtract 2) precision selection fix: MultiplyToGroupConvolution quick fix * [LPT] DenseNet improvments: AddTransformation: Add to Subtract + tests * [LPT] AddTransformarion refactoring * [LPT] AddTransformation tests temporay disabled * [LPT] ReshapeTransformation improvements: degradation fix * [LPT] code style fix * [LPT] Concat tests temporary disabling * [LPT] tests unification 1) plugin tests: added test-cases and nGraph-validation for clamp, split and variadic split 2) func tests: added test-cases 3) transformNGraph: added the ability to run additional transformations * [LPT] split & variadic split merge fix * [LPT] Clamp: added support for asymmetric quantization * [LPT] added DequantizationAttr run-time attribute * [LPT] debug info removal * [LPT] ConcatTransformation: zero point fix * [LPT] CNNNetwork ReLU transformation quick fix * [LPT] 1) Concat fix 2) ConcatMultiChannels fix 3) Added "Concat with Split" test-cases 4) Subgraph fix * [LPT] 1) Concat fix 2) Added "Concat with different precision on childs" test-case * [LPT] concat fix Ubuntu18 * [LPT] Concat test fixes * [LPT] Not fp32 FQ input support * [LPT] MatMul Fix + separateInStandaloneBranch Fix * [LPT] Fix reference input types in mish fusion tests * [LPT] Fix cpuFuncTests on CentOS building * [nGraph][LPT] ScaleShift 2d, 3d nGraph conversion enabling * [LPT] 1) FullyConnected workaround removing 2) validate_nodes_and_infer_types for LPT * [ngraph] Add check for childs for ConvertSubtract * [LPT] Squeeze/Unsqueeze tests unification * [LPT] Squeeze/Unsqueeze change signature for getReference/getOriginal * [LPT] Mul & Add -> ScaleShift quick fix * [LPT] nGraph tests emporary disabling * [LPT] code style fix * [LPT] code style fix #2 * [LPT] nGraph tests temporary disabling * [LPT] code styl fix openvinotoolkit#3 * [LPT] shared plugin tests temporary disabling * [LPT] cleanup * [LPT] nGraph unit_tests tests temproary disabling * [LPT] nGraph unit tests disabling #2 * [LPT] nGraph tests disabling * [LPT] nGraph tests temporary disabling * [LPT] WA removing * [LPT] CentOS compilation fix * [LPT] KMB wa to avoid compilation error * [LPT] functional test temporary disabling * [nGraph] code style fixes * [LPT] ConcatTransformation: data movement operation as intermediate handling * [LPT] FuseSubtractToFakeQuantize after VariadicSplit * [LPT] ConcatWithSplitTransformation functional test temporary disabling * [LPT] Clamp and ConcatWithDifferentPrecisionsOnChilds: tests fix * [LPT] MatMul: bert-nv-mlperf-quantized fix * [LPT] Add to convolution biases fuse fix * [LPT] GPU plugin tests fixes * [LPT] Normalize GPU plugin tests fix * [LPT] test-commit * [LPT] CLDNN Plugin FP16 conversion * [LPT] AvgPool update precision if there is not FQ after + convolution precision limitation on activation * [LPT] Convolution fixes * [LPT] FuseSubtractToFakequantize & FuseMultiplyToFakeQuantize improvement * [LPT] FuseSubtractToFakeQuantize test fix * [LPT] FuseSubtractToFakeQuantizeTransformation tests * [LPT] code style fix * [LPT] AvgPool child recursive extend * [LPT] AvgPool tests + fix * [LPT] compilation quick fix * [LPT] Add to convolution biases fuse fix * [LPT] Linux issues: MatMulWithOptimizedConstantFakeQuantizeTransformation temporary disabled * [LPT] Normalize GPU plugin tests fix * [LPT] test-commit * [LPT] 1) added the ability to create sub without dequantizationAttribute 2) fixed optimizeMulAfter: added copying rt_info 3) Tests Unification: Convolution transformation 4) added cleanRunTimeInfo into Network Helper * [LPT] Tests Unification: GroupConvolution * [LPT] removed debug info * [LPT] functional tests for Convolution & GroupConvolution extending * [LPT] [MatMul] Quick fix ubuntu error * [LPT] MatMulTransformation quick test fix: one constant for both intervals * [nGraph] code style fix * [LPT] added output_precision to NormalizeIE * [nGraph] NormalizeIE fix for LPT support * [LPT] nGraph WA removal * [LPT] fixed fillSubgraph for concat multi channels * [LPT] MatMul fix * [nGraph] WA removal: 1) nGraph tests enabling 2) LPT extanding: not handle in FP32 * [LPT] nGraph WA removal: function tests skip config rollback * [LPT] WA removal: precision propagation fix * [LPT] ConvertMulOrAddFinally transformation extending * [nGraph] ConvolutionMultiplyFusion rollback (move from legacy to common) * [nGraph] ConvertMulAddToScaleShiftOrPower: WA removal * [nGraph] TypeRelaxed: WA removal * [nGraph] WA removal: TypeRelaxed * [LPT] WA removal: ConcatTransformation * [nGraph] WA removal: Eltwise & ConvertMulOrAddFinally fixes to support LPT * [nGraph] MulAddConversion fix: 2D & 3D ScaleShift are supproted * [nGraph] VisualizeTree extending * [LPT] FakeQuantizeDequantization extending: check element wise dequantization operation * [LPT] FakeQuantizeDequantization extending: SubtractMultiplyToMultiplyAddTransformation & WeightableLayerTransformation * [LPT] Convolution + test infrastructure update * [LPT] GPU compilation error * [nGraph] BatchNorm plugin tests: input tensor definition * [LPT] LowPrecisionTransformer::isFunctionQuantized was added * [nGraph] WA final cleanup * [nGraph] ScaleShiftIE quick fix * [LPT] Functional tests: added test-cases "Concat with intermediate with constant" * [LPT] Transformer::isNetworkquantized fix * [LPT] SubtractMultiplyToMultiplyAdd zero Add remove: fix for ssd300 on gpu * [LPT] MultiplyToGroupConvolution not transform on Const * [LPT] workaround for negative scales * [LPT] Convert standalone dequantization Mul,Sub,Add to ScaleShift * [LPT] SubtractMultiplyToMultiplyAdd test fix * [LPT] Clamp transformation: GPU tests fix * [LPT] Transformer tests * [LPT] FakeQuantizePrecisionSelectionTransformation was disabled for GPU * [LPT] TransformerIsFunctionQuantized refactoring * [nGraph] code style fix * [LPT] mobilenet_v2_tf_depthwise test update * [LPT] TMP: dequantization folding * [LPT] Elementwise transformation fix: dequantization operations constant folding * [LPT] cleanup * [LPT] denormal values fix * [LPT] FuseFakeQuantize test fixed + negative multiply case * [LPT] FP32 -> FP16 conversion info * [LPT] FQ dot interval support + swapMultiplyAdd safely division * [LPT] test fix * [LPT] Tests for dot interval on FQ + tests for addTransformation enabling * [LPT] Clamp transformation fix * [LPT] FQ prec selection test fix * [LPT] Clamp test case * [LPT] Concat division precision fix * [LPT] cleanup * [LPT] merge fix * [LPT] WIP: MatMul asymmetric quantization fix (BERT) * [LPT] MatMulWithOptimizedConstantFakeQuantizeTransformation disabled * [LPT] GPU Plugin set config fix * [LPT] Fix merge mistakes * [LPT] Rollback device specific INT8 * [LPT] ReshapeFullyConnected fix: FullyConnected output fix * [LPT] bert-base-chinese GPU fix * [ngraph/LPT] Tests for fix convert_mul_or_add_finally with dequantization [ngraph/LPT] Fix convert mul_or_add_finally with dequantization * [LPT] ScaleShift dim < 4 only dequantization conversion * [LPT] MatMul transformation tests extensing * [LPT] ReshapeFullyConnected legacy transformation: LPT test case addition * [nGraph] VisualizeTree extending: property names displying to simplify search * [LPT] getDequantization extending * [LPT] MulAddToScaleshiftOrPower: out precision fix & tests * [LPT] Multiply to ScaleShiftIE: Multiply transformation: remove DEQUANTIZATION if not valid * [LPT] Concat test case * [nGraph] try to fix opencv compatibility * [nGraph] nGraph code style fix * [LPT] InPlace dequantization folding * [LPT] Multiply constant folding test * [LPT] Fix plugin test case for MatMulWithOptimizedConstantFakeQuantize [LPT] Enable MatMulWithOptimizedConstantFakeQuantize plugin test * [LPT] Convolution transformation: mulConst shape fix * [LPT] INT8 Constant folding branch for elementwise ops optimization removal * [LPT] eltwise for const branch fix * [LPT] linux fix * [LPT] Multiply test refactoring * [LPT] Convert Fuse in Constant + tests * [LPT] function comparation: runtime info comparation rollback * [LPT] linux build fix * [LPT] linux build fix2 * [LPT] MatMul transformation limitation was added to be similar as CNNNetwork LPT * [LPT] Reshape transformation update: don't broadcast by batch * [LPT] MatMul transformation limitation was added to be similar as CNNNetwork LPT - refactoring * [LPT] MatMul transformation: transpose input tensors fix * [LPT] checkElementwise for AddTransformation WA: should be moved to getDequantization * [LPT] merge fix * [LPT] MatMul fix & tests * [LPT] AddTransformation tests * [LPT] Interpolate transformation enabled * [LPT] constant folding before LPT * [LPT] WIP: not completed tests * [LPT] GPU degradation fix * [LPT] FuseConvert workaround * [LPT] code cleanup * [LPT] Interpolate GPU test quick fix * [LPT] GroupConvolution fix * [LPT] Fix fusing multiply for non-dequantization layers * [LPT] GPU pipeline update: enableInt8 initialization place update * [LPT] tests compilation fix * [LPT] merge fix * [LPT] tests enabling * [LPT] merge issue resolving * [LPT] LPT CNNNetwork usage macros: part #1: source code * [LPT] LPT CNNNetwork usage macros: part #2: cmake files update and tests addoption * [LPT] LPT workaround from nGraph core removing * [LPT] previous LPT version tests * [LPT] inference_engine_lp_transformations was returned back * [LPT] replace_node rollback * [LPT] ConvertSubtract fix * [LPT] GPU: baselineIsFP16 reuse fix * [LPT] FakeQuantizeTransformation: GPU workaround: I32 -> FP32 Convert is not fused * [LPT] AvgPool output precision workaround * [LPT] Group convolution precision + Subtract to ScaleShift const fix * [LPT] SubMulToMulAdd & Transpose: action-recognition-0001 fix * [LPT] Transpose: added test with per-tensor quantization Co-authored-by: Aleksandr Pertovsky <[email protected]> Co-authored-by: Zinoviev, Vladimir <[email protected]> Co-authored-by: Vladislav Golubev <[email protected]> Co-authored-by: Gorokhov Dmitriy <[email protected]>

eshoguli and others added 30 commits August 29, 2020 16:08

[LPT] Replace creation of dequantization with factory

75a6414

[ngraph][LPT] Add ScaleShift replace for dequantization operations

4cc1109

[LPT] SubtractMultiplyToMultiplyAdd refactoring

0ee5e9a

[LPT] Code style fix

6840fb8

[LPT] Edit SubtractMultiplyToMultiplyAdd transformation for dequantiz…

e83e0d9

…ation

[LPT] Linux compilation quick fix

c25028b

[LPT] [WIP] runtime info applying

ff35f58

[LPT] Concat transformation functional tests extending

10d2a4a

[LPT] MultiplyToConvolution + Subtract to add fusing + improvements i…

88ed077

…n LowPrecisionTransformer

[LPT] linux compilation error fix

5e3b364

[LPT] compilation error

09a8c7b

Merge branch 'es/lpt/lpt_to_ngraph_integration' of https://github.com…

178fbeb

…/eshoguli/openvino into es/lpt/lpt_to_ngraph_integration

[LPT] MultiplyToGroupConvolution fix: 5D support

db0c33c

[LPT] Multiply transformation extending: FQ weights support - wip

20bc449

[LPT] FQ folding & precision selection

0e9f6de

[LPT] code style fixes

c92a2aa

[LPT] code style fixes

f4ffccf

[LPT] Linux compilation error fix

d956008

[LPT] SubtractMultiplyToMultiplyAdd: refactoring

7bfdf66

[LPT] Tests fixes

36119b5

[LPT] MultiplyToGroupConvolution tests

3155060

[LPT] Convert subtract with int inputs to Eltwise sub

9a3b0e2

[LPT] Constant folding fix for quant models

6d8b12b

[LPT] 1) Asymmetric quantization improvement 2) tests extending

715b8c4

[LPT] 2 fixes for se_resnext_50

8092c93

Merge remote-tracking branch 'upstream/master' into es/lpt/lpt_to_ngr…

1e316cc

…aph_integration

[LPT] Add transformation priority branch selection test

6bb3a8e

[LPT] AddMultiplyFusion: legacy transformation quick fix

0b6d600

Merge branch 'es/lpt/lpt_to_ngraph_integration' of https://github.com…

fd7d49b

…/eshoguli/openvino into es/lpt/lpt_to_ngraph_integration

[LPT] nGraph tests temporary disabling

7396092

vladimir-paramuzov reviewed Oct 20, 2020

View reviewed changes

eshoguli force-pushed the es/lpt/lpt_to_ngraph_fixes2_with_master branch from da449bb to e87eb4f Compare October 20, 2020 21:44

eshoguli marked this pull request as ready for review October 20, 2020 23:02

eshoguli requested a review from ilyachur as a code owner October 20, 2020 23:02

eshoguli requested review from a team October 20, 2020 23:02

eshoguli requested review from a team as code owners October 20, 2020 23:02

eshoguli requested review from a team October 20, 2020 23:02

[LPT] ConvertSubtract fix

5126095

eshoguli force-pushed the es/lpt/lpt_to_ngraph_fixes2_with_master branch from 5beecf6 to 5126095 Compare October 21, 2020 11:42

eshoguli and others added 7 commits October 21, 2020 17:23

[LPT] GPU: baselineIsFP16 reuse fix

6c52ee7

[LPT] FakeQuantizeTransformation: GPU workaround: I32 -> FP32 Convert…

3c1b711

… is not fused

[LPT] AvgPool output precision workaround

840937e

[LPT] Group convolution precision + Subtract to ScaleShift const fix

00aef0d

[LPT] SubMulToMulAdd & Transpose: action-recognition-0001 fix

a29524d

[LPT] Transpose: added test with per-tensor quantization

2a4ca0f

Merge pull request #15 from eshoguli/es/lpt/lpt_to_ngraph_fixes2_with…

0a57b9d

…_master_review fixes to master merge branch

eshoguli force-pushed the es/lpt/lpt_to_ngraph_fixes2_with_master branch from 7d993a8 to 0a57b9d Compare October 21, 2020 18:05

dmitry-gorokhov approved these changes Oct 23, 2020

View reviewed changes

ababushk merged commit c2271da into openvinotoolkit:master Oct 23, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Es/lpt/lpt to ngraph fixes2 with master #2671

Es/lpt/lpt to ngraph fixes2 with master #2671

eshoguli commented Oct 14, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov Oct 20, 2020

vladimir-paramuzov commented Oct 21, 2020

ilya-lavrenov commented Oct 21, 2020

vladimir-paramuzov commented Oct 21, 2020

Es/lpt/lpt to ngraph fixes2 with master #2671

Es/lpt/lpt to ngraph fixes2 with master #2671

Conversation

eshoguli commented Oct 14, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vladimir-paramuzov commented Oct 21, 2020

ilya-lavrenov commented Oct 21, 2020

vladimir-paramuzov commented Oct 21, 2020