-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/azaytsev/g api docs #3731
Merged
evgenytalanin-intel
merged 11 commits into
openvinotoolkit:releases/2021/2
from
andrew-zaytsev:feature/azaytsev/g-api-docs
Dec 25, 2020
Merged
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
4f0d313
Initial commit
andrew-zaytsev b6436e6
Added content
andrew-zaytsev 85b04b8
Added new content for g-api documentation. Removed obsolete links thr…
andrew-zaytsev b358134
Merge remote-tracking branch 'upstream/releases/2021/2' into feature/…
andrew-zaytsev 028c3b0
Fixed layout
andrew-zaytsev d6f99c1
Fixed layout
andrew-zaytsev 3e3998b
Added new topics
andrew-zaytsev c0cfa7a
Added new info
andrew-zaytsev aeae5b6
added a note
andrew-zaytsev 176a4f7
Merge remote-tracking branch 'upstream/releases/2021/2' into feature/…
andrew-zaytsev fdc2f0d
Removed redundant .svg
andrew-zaytsev File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
# Introduction to OpenCV Graph API (G-API) {#openvino_docs_gapi_gapi_intro} | ||
OpenCV Graph API (G-API) is an OpenCV module targeted to make regular image and video processing fast and portable. G-API is a special module in OpenCV – in contrast with the majority of other main modules, this one acts as a framework rather than some specific CV algorithm. | ||
|
||
G-API is positioned as a next level optimization enabler for computer vision, focusing not on particular CV functions but on the whole algorithm optimization. | ||
|
||
G-API provides means to define CV operations, construct graphs (in form of expressions) using it, and finally implement and run the operations for a particular backend. | ||
|
||
The idea behind G-API is that if an algorithm can be expressed in a special embedded language (currently in C++), the framework can catch its sense and apply a number of optimizations to the whole thing automatically. Particular optimizations are selected based on which [kernels](kernel_api.md) and [backends](https://docs.opencv.org/4.5.0/dc/d1c/group__gapi__std__backends.html) are involved in the graph compilation process, for example, the graph can be offloaded to GPU via the OpenCL backend, or optimized for memory consumption with the Fluid backend. Kernels, backends, and their settings are parameters to the graph compilation, so the graph itself does not depend on any platform-specific details and can be ported easily. | ||
|
||
> **NOTE**: Graph API (G-API) was introduced in the most recent major OpenCV 4.0 release and now is being actively developed. The API is volatile at the moment and there may be minor but compatibility-breaking changes in the future. | ||
|
||
## G-API Concepts | ||
|
||
* *Graphs* are built by applying operations to data objects. | ||
* API itself has no "graphs", it is expression-based instead. | ||
* *Data objects* do not hold actual data, only capture dependencies. | ||
* *Operations* consume and produce data objects. | ||
* A graph is defined by specifying its boundaries with data objects: | ||
* What data objects are inputs to the graph? | ||
* What are its outputs? | ||
|
||
The paragraphs below explain the G-API programming model and development workflow. | ||
|
||
## Programming Model | ||
Building graphs is easy with G-API. In fact, there is no notion of graphs exposed in the API, so the user doesn’t need to operate in terms of “nodes” and “edges” — instead, graphs are constructed implicitly via expressions in a "functional" way. Expression-based graphs are built using two major concepts: *[operations](kernel_api.md)* and *[data objects](https://docs.opencv.org/4.2.0/db/df1/group__gapi__data__objects.html)*. | ||
|
||
In G-API, every graph begins and ends with data objects; data objects are passed to operations which produce (“return”) their results — new data objects, which are then passed to other operations, and so on. You can declare their own operations, G-API does not distinguish user-defined operations from its own predefined ones in any way. | ||
|
||
After the graph is defined, it needs to be compiled for execution. During the compilation, G-API figures out what the graph looks like, which kernels are available to run the operations in the graph, how to manage heterogeneity and to optimize the execution path. The result of graph compilation is a so-called “compiled” object. This object encapsulates the execution sequence for the graph inside and operates on real image data. You can set up the compilation process using various [compilation arguments](https://docs.opencv.org/4.5.0/dc/d1c/group__gapi__std__backends.html). Backends expose some of their options as these arguments; also, actual kernels and DL network settings are passed into the framework this way. | ||
|
||
G-API supports graph compilation for two execution modes, *regular* and *streaming*, producing different types of compiled objects as the result. | ||
* <strong>Regular</strong> compiled objects are represented with class GCompiled, which follows functor-like semantics and has an overloaded operator(). When called for execution on the given input data, the GCompiled functor blocks the current thread and processes the data immediately — like a regular C++ function. By default, G-API tries to optimize the execution time for latency in this compilation mode. | ||
* Starting with OpenCV 4.2, G-API can also produce GStreamingCompiled objects that better fit the asynchronous pipelined execution model. This compilation mode is called **streaming mode**, and G-API tries to optimize the overall throughput by implementing the pipelining technique as described above. We will use both in our example. | ||
|
||
The overall process for the regular case is summarized in the diagram below: | ||
|
||
![G-API Programming Model](../img/gapi_programming_model.png) | ||
|
||
The graph is built with operations so having operations defined (**0**) is a basic prerequisite; a constructed expression graph (**1**) forms a `cv::GComputation` object; kernels (**2**) which implement operations are the basic requirement to the graph compilation (**3**); the actual execution (**4**) is handled by a `cv::GCompiled` object with takes input and produces output data. | ||
|
||
## Development Workflow | ||
One of the ways to organize a G-API development workflow is presented in the diagram below: | ||
|
||
![G-API development workflow](../img/gapi_development_workflow.png) | ||
|
||
Basically, it is a derivative from the programming model illustrated in the previous chapter. You start with an algorithm or a data flow in mind (**0**), mapping it to a graph model (**1**), then identifying what operations you need (**2**) to construct this graph. These operations may already exist in G-API or be missing, in the latter case we implement the missing ones as kernels (**3**). Then decide which execution model fits our case better, pass kernels and DL networks as arguments to the compilation process (**4**), and finally switch to the execution (**5**). The process is iterative, so if you want to change anything based on the execution results, get back to steps (**0**) or (**1**) (a dashed line). | ||
|
||
|
||
|
||
|
||
|
||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do we need all this stuff in our docs? It's OpenCV module and all documentation must be there
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CC @dmatveev @alalek
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The G-API documentation was integrated by request from Dmitry @dmatveev .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, question is redirected to @dmatveev - why do we need this docs inside OpenVINO? Such intro must be a part of OpenCV docs, we can just refer to it, I suppose
Otherwise we can have custom intro into opencv_core, opencv_imgproc and so. Why G-API is an exception?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Customers asked for some starting point at the OpenVINO docs as G-API was exposed and served there under the OpenVINO umbrella.
OpenCV docs have this already & the operation/kernel reference will remain there. The OpenVINO-side part will be more Intel/OpenVINO-specific as
Comparison with core/imgproc is not very correct as those parts are quite stable, well known, and not evolve that fast as we do.