Skip to content

Commit

Permalink
doc updates (openvinotoolkit#8268)
Browse files Browse the repository at this point in the history
* Various doc changes

* theme changes

* remove font-family (openvinotoolkit#8211)

* fix  css

* Update uninstalling-openvino.md

* fix css

* fix

* Fixes for Installation Guides

Co-authored-by: Andrey Zaytsev <[email protected]>
Co-authored-by: kblaszczak-intel <[email protected]>
# Conflicts:
#	docs/IE_DG/Bfloat16Inference.md
#	docs/IE_DG/InferenceEngine_QueryAPI.md
#	docs/IE_DG/OnnxImporterTutorial.md
#	docs/IE_DG/supported_plugins/AUTO.md
#	docs/IE_DG/supported_plugins/HETERO.md
#	docs/IE_DG/supported_plugins/MULTI.md
#	docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md
#	docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_YOLO_From_Tensorflow.md
#	docs/install_guides/installing-openvino-macos.md
#	docs/install_guides/installing-openvino-windows.md
#	docs/ops/opset.md
#	inference-engine/samples/benchmark_app/README.md
#	inference-engine/tools/benchmark_tool/README.md
#	thirdparty/ade
  • Loading branch information
ntyukaev authored and andrew-zaytsev committed Dec 15, 2021
1 parent 8fe4a65 commit ff36118
Show file tree
Hide file tree
Showing 74 changed files with 2,646 additions and 1,318 deletions.
12 changes: 6 additions & 6 deletions docs/HOWTO/Custom_Layers_Guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Representation (IR) for this model.
This guide illustrates the workflow for running inference on topologies featuring custom operations, allowing you to
plug in your own implementation for existing or completely new operation.

> **NOTE:** *Layer* — The legacy term for an *operation* which came from Caffe\* framework. Currently it is not used.
> **NOTE**: *Layer* — The legacy term for an *operation* which came from Caffe\* framework. Currently it is not used.
> Refer to the [Deep Learning Network Intermediate Representation and Operation Sets in OpenVINO™](../MO_DG/IR_and_opsets.md)
> for more information on the topic.
Expand Down Expand Up @@ -44,7 +44,7 @@ plugins to support inference of this operation using a particular target hardwar
To see the operations that are supported by each device plugin for the Inference Engine, refer to the
[Supported Devices](../IE_DG/supported_plugins/Supported_Devices.md).

> **NOTE:** If a device doesn't support a particular operation, an alternative to creating a new operation is to target
> **NOTE**: If a device doesn't support a particular operation, an alternative to creating a new operation is to target
> an additional device using the HETERO plugin. The [Heterogeneous Plugin](../IE_DG/supported_plugins/HETERO.md) may be
> used to run an inference model on multiple devices allowing the unsupported operations on one device to "fallback" to
> run on another device (e.g., CPU) that does support those operations.
Expand All @@ -63,7 +63,7 @@ operation and uses corresponding operation class to update graph node attributes
operation. Refer to the "Operation Extractor" section of
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for detailed instructions on how to implement it.

> **NOTE:** In some cases you may need to implement some transformation to support the operation. This topic is covered in the "Graph Transformation Extensions" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
> **NOTE**: In some cases you may need to implement some transformation to support the operation. This topic is covered in the "Graph Transformation Extensions" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
## Custom Operations Extensions for the Inference Engine

Expand Down Expand Up @@ -140,7 +140,7 @@ mo --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1

@endsphinxdirective

> **NOTE:** This conversion guide is applicable for the 2021.3 release of OpenVINO and that starting from 2021.4
> **NOTE**: This conversion guide is applicable for the 2021.3 release of OpenVINO and that starting from 2021.4
> the OpenVINO supports this model out of the box.
Model Optimizer produces the following error:
Expand Down Expand Up @@ -222,7 +222,7 @@ following snippet provides two extractors: one for "IFFT2D", another one for "FF

@snippet FFT_ext.py fft_ext:extractor

> **NOTE:** The graph is in inconsistent state after extracting node attributes because according to original operation
> **NOTE**: The graph is in inconsistent state after extracting node attributes because according to original operation
> "IFFT2D" semantic it should have an input consuming a tensor of complex numbers, but the extractor instantiated an
> operation "FFT" which expects a real tensor with specific layout. But the inconsistency will be resolved during
> applying front phase transformations discussed below.
Expand All @@ -240,7 +240,7 @@ information on how this type of transformation works. The code snippet should be

@snippet Complex.py complex:transformation

> **NOTE:** The graph is in inconsistent state because the "ComplexAbs" operation consumes complex value tensor but
> **NOTE**: The graph is in inconsistent state because the "ComplexAbs" operation consumes complex value tensor but
> "FFT" produces real value tensor.
Now lets implement a transformation which replace a "ComplexAbs" operation with a sub-graph of primitive operations
Expand Down
186 changes: 150 additions & 36 deletions docs/IE_DG/Bfloat16Inference.md

Large diffs are not rendered by default.

23 changes: 9 additions & 14 deletions docs/IE_DG/Deep_Learning_Inference_Engine_DevGuide.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,19 @@

openvino_docs_IE_DG_Integrate_with_customer_application_new_API
openvino_docs_deployment_optimization_guide_dldt_optimization_guide
openvino_docs_deployment_optimization_guide_dldt_optimization_guide_additional
openvino_docs_IE_DG_API_Changes
openvino_docs_IE_DG_Memory_primitives
openvino_docs_IE_DG_InferenceEngine_QueryAPI
openvino_docs_IE_DG_Model_caching_overview
openvino_docs_IE_DG_Extensibility_DG_Intro
openvino_docs_IE_DG_Migration_CoreAPI
openvino_inference_engine_ie_bridges_python_docs_api_overview
openvino_docs_IE_DG_ONNX_Support
openvino_docs_IE_DG_OnnxImporterTutorial
openvino_docs_IE_DG_Device_Plugins
Direct ONNX Format Support <openvino_docs_IE_DG_ONNX_Support>
openvino_docs_IE_DG_Int8Inference
openvino_docs_IE_DG_Bfloat16Inference
openvino_docs_IE_DG_DynamicBatching
openvino_docs_IE_DG_ShapeInference
openvino_docs_IE_DG_Bfloat16Inference
openvino_docs_IE_DG_Int8Inference
openvino_docs_IE_DG_network_state_intro
openvino_docs_IE_DG_Model_caching_overview
openvino_docs_IE_DG_Extensibility_DG_Intro
openvino_docs_IE_DG_Memory_primitives
openvino_docs_IE_DG_network_state_intro
openvino_docs_IE_DG_API_Changes
openvino_docs_IE_DG_Known_Issues_Limitations
openvino_docs_IE_DG_Glossary


@endsphinxdirective

Expand Down
112 changes: 80 additions & 32 deletions docs/IE_DG/DynamicBatching.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Using Dynamic Batching {#openvino_docs_IE_DG_DynamicBatching}

## Dynamic Batching Introduction (C++)
## Using Dynamic Batching (C++)

@sphinxdirective
.. raw:: html
Expand All @@ -9,50 +9,98 @@
@endsphinxdirective

The Dynamic Batching feature allows you to dynamically change batch size for inference calls
within a preset batch size limit.
This feature might be useful when batch size is unknown beforehand and using an extra-large batch size is
undesirable or impossible due to resource limitations.
For example, applying face detection and then mood labeling to a video, you won't know in advance how many frames will contain a face when you pass inferencing results to a secondary model.
within a preset batch size limit. This feature might be useful when batch size is unknown beforehand and using an extra-large batch size is undesirable or impossible due to resource limitations. For example, applying face detection and then mood labeling to a video, you won't know in advance how many frames will contain a face when you pass inferencing results to a secondary model.

## Usage

You can activate Dynamic Batching by setting <code>KEY_DYN_BATCH_ENABLED</code> flag to <code>YES</code> in a configuration map that is
You can activate Dynamic Batching by setting `KEY_DYN_BATCH_ENABLED` flag to `YES` in a configuration map that is
passed to the plugin while loading a network.
This configuration creates an <code>ExecutableNetwork</code> object that will allow setting batch size
dynamically in all of its infer requests using <code>SetBatch()</code> method.
The batch size that was set in the passed <code>CNNNetwork</code> object will be used as a maximum batch size limit.
This configuration creates an `ExecutableNetwork` object that will allow setting batch size
dynamically in all of its infer requests using `SetBatch()` method.
The batch size that was set in the passed `CNNNetwork` object will be used as a maximum batch size limit.

Here is a code example:

@snippet snippets/DynamicBatching.cpp part0


## Limitations
### Limitations

Currently, there are certain limitations for using Dynamic Batching:
Currently, there are certain limitations for the use of Dynamic Batching exist:

* Use Dynamic Batching with CPU and GPU plugins only.
* Use Dynamic Batching on topologies that consist of certain layers only:
* Convolution
* Deconvolution
* Activation
* LRN
* Pooling
* FullyConnected
* SoftMax
* Split
* Concatenation
* Power
* Eltwise
* Crop
* BatchNormalization
* Copy

The following types of layers are not supported:

* Layers that might arbitrary change tensor shape (such as Flatten, Permute, Reshape)
* Layers specific to object detection topologies (ROIPooling, ProirBox, DetectionOutput)
* Custom layers

Topology analysis is performed during the process of loading a network into plugin, and if the topology is not supported, an exception is generated.

## Using Dynamic Batching (Python)

@sphinxdirective
.. raw:: html

<div id="switcher-python" class="switcher-anchor">Python</div>
@endsphinxdirective

Dynamic Batching is a feature that allows you to dynamically change batch size for inference calls within a preset batch size limit. This feature might be useful when batch size is unknown beforehand, and using extra large batch size is not desired or impossible due to resource limitations. For example, face detection with person age, gender, or mood recognition is a typical usage scenario.

You can activate Dynamic Batching by setting the "DYN_BATCH_ENABLED" flag to "YES" in a configuration map that is passed to the plugin while loading a network. This configuration creates an `ExecutableNetwork` object that will allow setting batch size dynamically in all of its infer requests using the [ie_api.batch_size](api/ie_python_api/_autosummary/openvino.inference_engine.IENetwork.html#openvino.inference_engine.IENetwork.batch_size) method. The batch size that was set in the passed CNNNetwork object will be used as a maximum batch size limit.

```python
from openvino.inference_engine import IECore

ie = IECore()
dyn_config = {"DYN_BATCH_ENABLED": "YES"}
ie.set_config(config=dyn_config, device_name=device)
# Read a network in IR or ONNX format
net = ie.read_network(path_to_model)
net.batch_size = 32 # set the maximum batch size to 32
exec_net = ie.load_network(network=net, device_name=device)
```

### Limitations

Currently, certain limitations for the use of Dynamic Batching exist:

* Use Dynamic Batching with CPU and GPU plugins only.
* Use Dynamic Batching on topologies that consist of certain layers only:
* Convolution
* Deconvolution
* Activation
* LRN
* Pooling
* FullyConnected
* SoftMax
* Split
* Concatenation
* Power
* Eltwise
* Crop
* BatchNormalization
* Copy

The following types of layers are not supported:

* Convolution
* Deconvolution
* Activation
* LRN
* Pooling
* FullyConnected
* SoftMax
* Split
* Concatenation
* Power
* Eltwise
* Crop
* BatchNormalization
* Copy

Do not use layers that might arbitrary change tensor shape (such as Flatten, Permute, Reshape),
layers specific to object detection topologies (ROIPooling, ProirBox, DetectionOutput), and
custom layers.
Topology analysis is performed during the process of loading a network into plugin, and if topology is
not applicable, an exception is generated.
* Layers that might arbitrary change tensor shape (such as Flatten, Permute, Reshape)
* Layers specific to object detection topologies (ROIPooling, ProirBox, DetectionOutput)
* Custom layers

Topology analysis is performed during the process of loading a network into plugin, and if the topology is not supported, an exception is generated.
2 changes: 1 addition & 1 deletion docs/IE_DG/Extensibility_DG/GPU_Kernel.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ __kernel void example_relu_kernel(
}
```
> **NOTE:** As described in the previous section, all items like
> **NOTE**: As described in the previous section, all items like
> `INPUT0_TYPE` are actually defined as OpenCL (pre-)compiler inputs by
> the Inference Engine for efficiency reasons. See [Debugging
> Tips](#debugging-tips) for information on debugging the results.
Expand Down
18 changes: 17 additions & 1 deletion docs/IE_DG/Extensibility_DG/Intro.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,21 @@
# Inference Engine Extensibility Mechanism {#openvino_docs_IE_DG_Extensibility_DG_Intro}

@sphinxdirective

.. toctree::
:maxdepth: 1
:hidden:

openvino_docs_IE_DG_Extensibility_DG_AddingNGraphOps
openvino_docs_IE_DG_Extensibility_DG_Custom_ONNX_Ops
CPU Kernels Extensibility <openvino_docs_IE_DG_Extensibility_DG_CPU_Kernel>
GPU Kernels Extensibility <openvino_docs_IE_DG_Extensibility_DG_GPU_Kernel>
VPU Kernels Extensibility <openvino_docs_IE_DG_Extensibility_DG_VPU_Kernel>
openvino_docs_IE_DG_Extensibility_DG_Extension
openvino_docs_IE_DG_Extensibility_DG_Building

@endsphinxdirective

If your model contains operations not normally supported by OpenVINO, the Inference Engine Extensibility API lets you add support for those custom operations in a library containing custom nGraph operation sets, corresponding extensions to the Model Optimizer, and a device plugin extension. See the overview in the [Custom Operations Guide](../../HOWTO/Custom_Layers_Guide.md) to learn how these work together.

To load the Extensibility library to the `InferenceEngine::Core` object, use the `InferenceEngine::Core::AddExtension` method.
Expand All @@ -17,7 +33,7 @@ An Inference Engine Extension dynamic library contains the following components:
- Enables the creation of `ngraph::Function` with unsupported operations
- Provides a shape inference mechanism for custom operations

> **NOTE**: This documentation is written based on the [`Template extension`](https://github.com/openvinotoolkit/openvino/tree/master/docs/template_extension), which demonstrates extension development details. You can review the complete code, which is fully compilable and up-to-date, to see how it works.
> **NOTE**: This documentation is written based on the [Template extension](https://github.com/openvinotoolkit/openvino/tree/master/docs/template_extension), which demonstrates extension development details. You can review the complete code, which is fully compilable and up-to-date, to see how it works.
## Execution Kernels

Expand Down
4 changes: 2 additions & 2 deletions docs/IE_DG/Extensibility_DG/VPU_Kernel.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ To customize your topology with an OpenCL layer, carry out the tasks described o

## Compile OpenCL code for VPU (Intel® Neural Compute Stick 2)

> **NOTE:** OpenCL compiler, targeting Intel® Neural Compute Stick 2 for the SHAVE* processor only, is redistributed with OpenVINO.
> **NOTE**: OpenCL compiler, targeting Intel® Neural Compute Stick 2 for the SHAVE* processor only, is redistributed with OpenVINO.
OpenCL support is provided by ComputeAorta* and is distributed under a license agreement between Intel® and Codeplay* Software Ltd.

The OpenCL toolchain for the Intel® Neural Compute Stick 2 supports offline compilation only, so first compile OpenCL C code using the standalone `clc` compiler. You can find the compiler binary at `<INSTALL_DIR>/tools/cl_compiler`.

> **NOTE:** By design, custom OpenCL layers support any OpenCL kernels written assuming OpenCL version 1.2. It also supports half float extension and is optimized for this type, because it is a native type for Intel® Movidius™ VPUs.
> **NOTE**: By design, custom OpenCL layers support any OpenCL kernels written assuming OpenCL version 1.2. It also supports half float extension and is optimized for this type, because it is a native type for Intel® Movidius™ VPUs.
1. Prior to running a compilation, make sure that the following variables are set:
* `SHAVE_MA2X8XLIBS_DIR=<INSTALL_DIR>/tools/cl_compiler/lib/`
Expand Down
Loading

0 comments on commit ff36118

Please sign in to comment.