Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/azaytsev/g api docs #3731

1 change: 0 additions & 1 deletion docs/HOWTO/Custom_Layers_Guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -360,7 +360,6 @@ python3 $INTEL_OPENVINO_DIR/deployment_tools/tools/benchmark_tool/benchmark_app.
- [Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md)
- [Inference Engine Samples Overview](../IE_DG/Samples_Overview.md)
- [Overview of OpenVINO™ Toolkit Pre-Trained Models](@ref omz_models_intel_index)
- [Inference Engine Tutorials](https://github.com/intel-iot-devkit/inference-tutorials-generic)
- For IoT Libraries and Code Samples see the [Intel® IoT Developer Kit](https://github.com/intel-iot-devkit).

## Converting Models:
Expand Down
1 change: 0 additions & 1 deletion docs/IE_DG/protecting_model_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,5 +59,4 @@ should be called with `weights` passed as an empty `Blob`.
- Inference Engine Developer Guide: [Inference Engine Developer Guide](Deep_Learning_Inference_Engine_DevGuide.md)
- For more information on Sample Applications, see the [Inference Engine Samples Overview](Samples_Overview.md)
- For information on a set of pre-trained models, see the [Overview of OpenVINO™ Toolkit Pre-Trained Models](@ref omz_models_intel_index)
- For information on Inference Engine Tutorials, see the [Inference Tutorials](https://github.com/intel-iot-devkit/inference-tutorials-generic)
- For IoT Libraries and Code Samples see the [Intel® IoT Developer Kit](https://github.com/intel-iot-devkit).
13 changes: 13 additions & 0 deletions docs/doxygen/openvino_docs.xml
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,19 @@
<tab type="user" title="DL Streamer API Reference" url="https://openvinotoolkit.github.io/dlstreamer_gst/"/>
<!-- DL Streamer Examples -->
<tab type="usergroup" title="DL Streamer Examples" url="@ref gst_samples_README">
</tab>
<!-- G-API -->
<tab type="usergroup" title="Graph API (G-API) Developer Guide" url="@ref openvino_docs_gapi_gapi_intro">
<tab type="user" title="Introduction to G-API" url="@ref openvino_docs_gapi_gapi_intro"/>
<tab type="user" title="G-API Kernel API" url="@ref openvino_docs_gapi_kernel_api"/>
<tab type="user" title="Use Cases: Implementing a Face Beautification Algorithm" url="@ref openvino_docs_gapi_face_beautification"/>
<tab type="user" title="Use Cases: Building a Face Analytics Pipeline" url="@ref openvino_docs_gapi_gapi_face_analytics_pipeline"/>
<tab type="usergroup" title="API Reference" url="">
<tab type="user" title="G-API Core functionality" url="https://docs.opencv.org/4.2.0/df/d1f/group__gapi__core.html"/>
<tab type="user" title="G-API Image processing functionality" url="https://docs.opencv.org/4.2.0/d2/d00/group__gapi__imgproc.html"/>
<tab type="user" title="G-API Drawing and composition functionality" url="https://docs.opencv.org/4.2.0/df/de4/group__gapi__draw.html"/>
<tab type="user" title="G-API Framework" url="https://docs.opencv.org/4.2.0/d7/d0d/group__gapi.html"/>
</tab>
</tab>
<!-- OpenVX -->
<tab type="user" title="OpenVX Developer Guide" url="https://software.intel.com/en-us/openvino-ovx-guide"/>
Expand Down
435 changes: 435 additions & 0 deletions docs/gapi/face_beautification.md

Large diffs are not rendered by default.

325 changes: 325 additions & 0 deletions docs/gapi/gapi_face_analytics_pipeline.md

Large diffs are not rendered by default.

52 changes: 52 additions & 0 deletions docs/gapi/gapi_intro.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Introduction to OpenCV Graph API (G-API) {#openvino_docs_gapi_gapi_intro}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need all this stuff in our docs? It's OpenCV module and all documentation must be there

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The G-API documentation was integrated by request from Dmitry @dmatveev .

Copy link
Contributor

@ilya-lavrenov ilya-lavrenov Dec 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, question is redirected to @dmatveev - why do we need this docs inside OpenVINO? Such intro must be a part of OpenCV docs, we can just refer to it, I suppose

Otherwise we can have custom intro into opencv_core, opencv_imgproc and so. Why G-API is an exception?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Customers asked for some starting point at the OpenVINO docs as G-API was exposed and served there under the OpenVINO umbrella.

OpenCV docs have this already & the operation/kernel reference will remain there. The OpenVINO-side part will be more Intel/OpenVINO-specific as

  • more *PU support comes in the future;
  • more OMZ samples get ported on G-API.

Comparison with core/imgproc is not very correct as those parts are quite stable, well known, and not evolve that fast as we do.

OpenCV Graph API (G-API) is an OpenCV module targeted to make regular image and video processing fast and portable. G-API is a special module in OpenCV – in contrast with the majority of other main modules, this one acts as a framework rather than some specific CV algorithm.

G-API is positioned as a next level optimization enabler for computer vision, focusing not on particular CV functions but on the whole algorithm optimization.

G-API provides means to define CV operations, construct graphs (in form of expressions) using it, and finally implement and run the operations for a particular backend.

The idea behind G-API is that if an algorithm can be expressed in a special embedded language (currently in C++), the framework can catch its sense and apply a number of optimizations to the whole thing automatically. Particular optimizations are selected based on which [kernels](kernel_api.md) and [backends](https://docs.opencv.org/4.5.0/dc/d1c/group__gapi__std__backends.html) are involved in the graph compilation process, for example, the graph can be offloaded to GPU via the OpenCL backend, or optimized for memory consumption with the Fluid backend. Kernels, backends, and their settings are parameters to the graph compilation, so the graph itself does not depend on any platform-specific details and can be ported easily.

> **NOTE**: Graph API (G-API) was introduced in the most recent major OpenCV 4.0 release and now is being actively developed. The API is volatile at the moment and there may be minor but compatibility-breaking changes in the future.

## G-API Concepts

* *Graphs* are built by applying operations to data objects.
* API itself has no "graphs", it is expression-based instead.
* *Data objects* do not hold actual data, only capture dependencies.
* *Operations* consume and produce data objects.
* A graph is defined by specifying its boundaries with data objects:
* What data objects are inputs to the graph?
* What are its outputs?

The paragraphs below explain the G-API programming model and development workflow.

## Programming Model
Building graphs is easy with G-API. In fact, there is no notion of graphs exposed in the API, so the user doesn’t need to operate in terms of “nodes” and “edges” — instead, graphs are constructed implicitly via expressions in a "functional" way. Expression-based graphs are built using two major concepts: *[operations](kernel_api.md)* and *[data objects](https://docs.opencv.org/4.2.0/db/df1/group__gapi__data__objects.html)*.

In G-API, every graph begins and ends with data objects; data objects are passed to operations which produce (“return”) their results — new data objects, which are then passed to other operations, and so on. You can declare their own operations, G-API does not distinguish user-defined operations from its own predefined ones in any way.

After the graph is defined, it needs to be compiled for execution. During the compilation, G-API figures out what the graph looks like, which kernels are available to run the operations in the graph, how to manage heterogeneity and to optimize the execution path. The result of graph compilation is a so-called “compiled” object. This object encapsulates the execution sequence for the graph inside and operates on real image data. You can set up the compilation process using various [compilation arguments](https://docs.opencv.org/4.5.0/dc/d1c/group__gapi__std__backends.html). Backends expose some of their options as these arguments; also, actual kernels and DL network settings are passed into the framework this way.

G-API supports graph compilation for two execution modes, *regular* and *streaming*, producing different types of compiled objects as the result.
* <strong>Regular</strong> compiled objects are represented with class GCompiled, which follows functor-like semantics and has an overloaded operator(). When called for execution on the given input data, the GCompiled functor blocks the current thread and processes the data immediately — like a regular C++ function. By default, G-API tries to optimize the execution time for latency in this compilation mode.
* Starting with OpenCV 4.2, G-API can also produce GStreamingCompiled objects that better fit the asynchronous pipelined execution model. This compilation mode is called **streaming mode**, and G-API tries to optimize the overall throughput by implementing the pipelining technique as described above. We will use both in our example.

The overall process for the regular case is summarized in the diagram below:

![G-API Programming Model](../img/gapi_programming_model.png)

The graph is built with operations so having operations defined (**0**) is a basic prerequisite; a constructed expression graph (**1**) forms a `cv::GComputation` object; kernels (**2**) which implement operations are the basic requirement to the graph compilation (**3**); the actual execution (**4**) is handled by a `cv::GCompiled` object with takes input and produces output data.

## Development Workflow
One of the ways to organize a G-API development workflow is presented in the diagram below:

![G-API development workflow](../img/gapi_development_workflow.png)

Basically, it is a derivative from the programming model illustrated in the previous chapter. You start with an algorithm or a data flow in mind (**0**), mapping it to a graph model (**1**), then identifying what operations you need (**2**) to construct this graph. These operations may already exist in G-API or be missing, in the latter case we implement the missing ones as kernels (**3**). Then decide which execution model fits our case better, pass kernels and DL networks as arguments to the compilation process (**4**), and finally switch to the execution (**5**). The process is iterative, so if you want to change anything based on the execution results, get back to steps (**0**) or (**1**) (a dashed line).






Loading