update python

Signed-off-by: Hu, Yuan2 <[email protected]>

docs/OV_Runtime_UG/multi_device.md

@@ -13,13 +13,13 @@ The Multi-Device plugin automatically assigns inference requests to available co
 * Improved throughput from using multiple devices (compared to single-device execution)
 * More consistent performance, since the devices share the inference burden (if one device is too busy, another can take more of the load)
 
-Note that with Multi-Device the application logic is left unchanged, so you don't need to explicitly load the network to every device, create and balance the inference requests and so on. From the application point of view, this is just another device that handles the actual machinery. The only thing that is required to leverage performance is to provide the multi-device (and hence the underlying devices) with enough inference requests to process. For example, if you were processing 4 cameras on the CPU (with 4 inference requests), it might be desirable to process more cameras (with more requests in flight) to keep CPU and GPU busy via Multi-Device.
+Note that with Multi-Device the application logic is left unchanged, so you don't need to explicitly compile the model on every device, create and balance the inference requests and so on. From the application point of view, this is just another device that handles the actual machinery. The only thing that is required to leverage performance is to provide the multi-device (and hence the underlying devices) with enough inference requests to process. For example, if you were processing 4 cameras on the CPU (with 4 inference requests), it might be desirable to process more cameras (with more requests in flight) to keep CPU and GPU busy via Multi-Device.
 
 The setup of Multi-Device can be described in three major steps:
 
 1. Configure each device as usual.
-2. Load the network to the Multi-Device plugin created on top of a (prioritized) list of the configured devices. This is the only change needed in the application.
-3. As with any other ExecutableNetwork call (resulting from `InferenceEngine::ov::compile_model`), you create as many requests as needed to saturate the devices.
+2. Compile the model on the Multi-Device plugin created on top of a (prioritized) list of the configured devices. This is the only change needed in the application.
+3. As with any other CompiledModel call (resulting from `compile_model`), you create as many requests as needed to saturate the devices.
 
 These steps are covered below in detail.
 
@@ -35,11 +35,25 @@ You can set the priorities directly as a string.
 
 Basically, there are three ways to specify the devices to be use by the "MULTI":
 
-@snippet snippets/MULTI0.cpp part0
+@sphinxdirective
+
+.. tab:: C++
+    .. doxygensnippet:: docs/snippets/MULTI0.cpp
+       :language: cpp
+       :fragment: [part0]
+
+@endsphinxdirective
 
 Notice that the priorities of the devices can be changed in real time for the executable network:
 
-@snippet snippets/MULTI1.cpp part1
+@sphinxdirective
+
+.. tab:: C++
+    .. doxygensnippet:: docs/snippets/MULTI1.cpp
+       :language: cpp
+       :fragment: [part1]
+
+@endsphinxdirective
 
 Finally, there is a way to specify number of requests that the Multi-Device will internally keep for each device. Suppose your original app was running 4 cameras with 4 inference requests. You would probably want to share these 4 requests between 2 devices used in MULTI. The easiest way is to specify a number of requests for each device using parentheses: "MULTI:CPU(2),GPU(2)" and use the same 4 requests in your app. However, such an explicit configuration is not performance-portable and hence not recommended. Instead, the better way is to configure the individual devices and query the resulting number of requests to be used at the application level (see [Configuring the Individual Devices and Creating the Multi-Device On Top](#configuring-the-individual-devices-and-creating-the-multi-device-on-top)).
 
@@ -60,7 +74,14 @@ The OpenVINO Runtime API features a dedicated methods to enumerate devices and t
 
 A simple programmatic way to enumerate the devices and use with the multi-device is as follows:
 
-@snippet snippets/MULTI2.cpp part2
+@sphinxdirective
+
+.. tab:: C++
+    .. doxygensnippet:: docs/snippets/MULTI2.cpp
+       :language: cpp
+       :fragment: [part2]
+
+@endsphinxdirective
 
 Beyond the trivial "CPU", "GPU", "HDDL" and so on, when multiple instances of a device are available the names are more qualified. For example, this is how two Intel® Movidius™ Myriad™ X sticks are listed with the hello_query_sample:
 ```
@@ -72,13 +93,26 @@ Beyond the trivial "CPU", "GPU", "HDDL" and so on, when multiple instances of a
 
 So the explicit configuration to use both would be "MULTI:MYRIAD.1.2-ma2480,MYRIAD.1.4-ma2480". Accordingly, the code that loops over all available devices of "MYRIAD" type only is below:
 
-@snippet snippets/MULTI3.cpp part3
+@sphinxdirective
 
+.. tab:: C++
+    .. doxygensnippet:: docs/snippets/MULTI3.cpp
+       :language: cpp
+       :fragment: [part3]
+
+@endsphinxdirective
 
 ### Configuring the Individual Devices and Creating the Multi-Device On Top
 As discussed in the first section, you shall configure each individual device as usual and then just create the "MULTI" device on top:
 
-@snippet snippets/MULTI4.cpp part4
+@sphinxdirective
+
+.. tab:: C++
+    .. doxygensnippet:: docs/snippets/MULTI4.cpp
+       :language: cpp
+       :fragment: [part4]
+
+@endsphinxdirective
 
 An alternative is to combine all the individual device settings into a single config file and load that, allowing the Multi-Device plugin to parse and apply settings to the right devices. See the code example in the next section.
 
@@ -88,7 +122,14 @@ See the [Using the Multi-Device with OpenVINO samples and benchmarking the perfo
 ### Querying the Optimal Number of Inference Requests
 You can use the [configure devices](supported_plugins/config_properties.md) to query the optimal number of requests. Similarly, when using the Multi-Device you don't need to sum over included devices yourself, you can query property directly:
 
-@snippet snippets/MULTI5.cpp part5
+@sphinxdirective
+
+.. tab:: C++
+    .. doxygensnippet:: docs/snippets/MULTI5.cpp
+       :language: cpp
+       :fragment: [part5]
+
+@endsphinxdirective
 
 ### Using the Multi-Device with OpenVINO Samples and Benchmarking the Performance
 
@@ -125,13 +166,13 @@ The Multi-Device plugin automatically assigns inference requests to available co
 * Improved throughput from using multiple devices (compared to single-device execution)
 * More consistent performance, since the devices share the inference burden (if one device is too busy, another can take more of the load)
 
-Note that with Multi-Device the application logic is left unchanged, so you don't need to explicitly load the network to every device, create and balance the inference requests and so on. From the application point of view, this is just another device that handles the actual machinery. The only thing that is required to leverage performance is to provide the multi-device (and hence the underlying devices) with enough inference requests to process. For example, if you were processing 4 cameras on the CPU (with 4 inference requests), it might be desirable to process more cameras (with more requests in flight) to keep CPU and GPU busy via Multi-Device.
+Note that with Multi-Device the application logic is left unchanged, so you don't need to explicitly compile the model on every device, create and balance the inference requests and so on. From the application point of view, this is just another device that handles the actual machinery. The only thing that is required to leverage performance is to provide the multi-device (and hence the underlying devices) with enough inference requests to process. For example, if you were processing 4 cameras on the CPU (with 4 inference requests), it might be desirable to process more cameras (with more requests in flight) to keep CPU and GPU busy via Multi-Device.
 
 The setup of Multi-Device can be described in three major steps:
 
-1. Configure each device as usual (using the conventional [ie_api.IECore.set_config](api/ie_python_api/_autosummary/openvino.inference_engine.IECore.html#openvino.inference_engine.IECore.set_config) method
-2. Load the network to the Multi-Device plugin created on top of a (prioritized) list of the configured devices. This is the only change needed in the application.
-3. As with any other ExecutableNetwork call (resulting from `load_network`), you create as many requests as needed to saturate the devices.
+1. Configure each device as usual (using the conventional [configure devices](supported_plugins/config_properties.md) method
+2. Compile the model on the Multi-Device plugin created on top of a (prioritized) list of the configured devices. This is the only change needed in the application.
+3. As with any other CompiledModel call (resulting from `compile_model`), you create as many requests as needed to saturate the devices.
 
 These steps are covered below in detail.
 
@@ -148,34 +189,27 @@ You can set the configuration directly as a string, or use the metric key `MULTI
 #### The Three Ways to Specify Devices Targets for the MULTI plugin
 
 * Option 1 - Pass a Prioritized List as a Parameter in ie.load_network()
-   ```python
-   from openvino.inference_engine import IECore
 
-   ie = IECore()
-   # Read a network in IR or ONNX format
-   net = ie.read_network(model=path_to_model)
-   exec_net = ie.load_network(network=net, device_name="MULTI:CPU,GPU")
-   ```
+@sphinxdirective
+
+.. tab:: Python
+    .. doxygensnippet:: docs/snippets/ov_multi.py
+       :language: python
+       :fragment: [Option_1]
+
+@endsphinxdirective
 
 * Option 2 - Pass a List as a Parameter, and Dynamically Change Priorities during Execution
    Notice that the priorities of the devices can be changed in real time for the executable network:
-   ```python
-   from openvino.inference_engine import IECore
 
-   # Init the Inference Engine Core
-   ie = IECore()
-
-   # Read a network in IR or ONNX format
-   net = ie.read_network(model=path_to_model)
+@sphinxdirective
 
-   ie.set_config( config={"MULTI_DEVICE_PRIORITIES":"HDDL,GPU"}, device_name="MULTI")
+.. tab:: Python
+    .. doxygensnippet:: docs/snippets/ov_multi.py
+       :language: python
+       :fragment: [Option_2]
 
-   # Change priorities
-   ie.set_config( config={"MULTI_DEVICE_PRIORITIES":"GPU,HDDL"}, device_name="MULTI")
-   ie.set_config( config={"MULTI_DEVICE_PRIORITIES":"GPU"}, device_name="MULTI")
-   ie.set_config( config={"MULTI_DEVICE_PRIORITIES":"HDDL,GPU"}, device_name="MULTI")
-   ie.set_config( config={"MULTI_DEVICE_PRIORITIES":"CPU,HDDL,GPU"}, device_name="MULTI")
-   ```
+@endsphinxdirective
 
 * Option 3 - Use Explicit Hints for Controlling Request Numbers Executed by Devices
    There is a way to specify the number of requests that Multi-Device will internally keep for each device. If the original app was running 4 cameras with 4 inference requests, it might be best to share these 4 requests between 2 devices used in the MULTI. The easiest way is to specify a number of requests for each device using parentheses: “MULTI:CPU(2),GPU(2)” and use the same 4 requests in the app. However, such an explicit configuration is not performance-portable and not recommended. The better way is to configure the individual devices and query the resulting number of requests to be used at the application level. See [Configuring the Individual Devices and Creating the Multi-Device On Top](#configuring-the-individual-devices-and-creating-the-multi-device-on-top).
@@ -198,16 +232,14 @@ The OpenVINO Runtime API features a dedicated methods to enumerate devices and t
 
 A simple programmatic way to enumerate the devices and use with the multi-device is as follows:
 
-```python
+@sphinxdirective
 
-from openvino.inference_engine import IECore
+.. tab:: Python
+    .. doxygensnippet:: docs/snippets/ov_multi.py
+       :language: python
+       :fragment: [available_devices_1]
 
-all_devices = "MULTI:"
-ie = IECore()
-net = ie.read_network(model=path_to_model)
-all_devices += ",".join(ie.available_devices)
-exec_net = ie.load_network(network=net, device_name=all_devices)
-```
+@endsphinxdirective
 
 Beyond the trivial "CPU", "GPU", "HDDL" and so on, when multiple instances of a device are available the names are more qualified. For example, this is how two Intel® Movidius™ Myriad™ X sticks are listed with the hello_query_sample:
 
@@ -220,46 +252,27 @@ Beyond the trivial "CPU", "GPU", "HDDL" and so on, when multiple instances of a
 
 So the explicit configuration to use both would be "MULTI:MYRIAD.1.2-ma2480,MYRIAD.1.4-ma2480". Accordingly, the code that loops over all available devices of "MYRIAD" type only is below:
 
-```python
-from openvino.inference_engine import IECore
-
-ie = IECore()
-match_list = []
-all_devices = "MULTI:"
-dev_match_str = "MYRIAD"
-net = ie.read_network(model=path_to_model)
+@sphinxdirective
 
-for d in ie.available_devices:
-    if dev_match_str in d:
-        match_list.append(d)
+.. tab:: Python
+    .. doxygensnippet:: docs/snippets/ov_multi.py
+       :language: python
+       :fragment: [available_devices_2]
 
-all_devices += ",".join(match_list)
-exec_net = ie.load_network(network=net, device_name=all_devices)
-```
+@endsphinxdirective
 
 ### Configuring the Individual Devices and Creating the Multi-Device On Top
 
 It is possible to configure each individual device as usual and then create the "MULTI" device on top:
 
-```python
-from openvino.inference_engine import IECore
-
-ie = IECore()
-net = ie.read_network(model=path_to_model)
-
-cpu_config = {}
-gpu_config = {}
+@sphinxdirective
 
-ie.set_config(config=cpu_config, device_name="CPU")
-ie.set_config(config=gpu_config, device_name="GPU")
+.. tab:: Python
+    .. doxygensnippet:: docs/snippets/ov_multi.py
+       :language: python
+       :fragment: [set_property]
 
-# Load the network to the multi-device, specifying the priorities
-exec_net = ie.load_network(
-    network=net, device_name="MULTI", config={"MULTI_DEVICE_PRIORITIES": "CPU,GPU"}
-)
-# Query the optimal number of requests
-nireq = exec_net.get_metric("OPTIMAL_NUMBER_OF_INFER_REQUESTS")
-```
+@endsphinxdirective
 
 An alternative is to combine all the individual device settings into a single config file and load that, allowing the Multi-Device plugin to parse and apply settings to the right devices. See the code example in the next section.
 

docs/snippets/ov_multi.py

@@ -0,0 +1,70 @@
+import sys
+from openvino.runtime import Core
+model_path = "/openvino_CI_CD/result/install_pkg/tests/test_model_zoo/core/models/ir/add_abc.xml"
+path_to_model = "/openvino_CI_CD/result/install_pkg/tests/test_model_zoo/core/models/ir/add_abc.xml"
+def Option_1():
+#! [option_1]
+    core = Core()
+
+    # Read a network in IR or ONNX format
+    model = core.read_model(model_path)
+    compiled_model = core.compile_model(model=model, device_name="MULTI:CPU,GPU")
+#! [option_1]
+
+def Option_2():
+#! [option_2]
+    core = Core()
+
+    # Read a network in IR or ONNX format
+    model = core.read_model(model_path)
+    core.set_property(device_name="MULTI", properties={"MULTI_DEVICE_PRIORITIES":"HDDL,GPU"})
+    # Change priorities
+    core.set_property(device_name="MULTI", properties={"MULTI_DEVICE_PRIORITIES":"GPU,HDDL"})
+    core.set_property(device_name="MULTI", properties={"MULTI_DEVICE_PRIORITIES":"GPU"})
+    core.set_property(device_name="MULTI", properties={"MULTI_DEVICE_PRIORITIES":"HDDL,GPU"})
+    core.set_property(device_name="MULTI", properties={"MULTI_DEVICE_PRIORITIES":"CPU,HDDL,GPU"})
+#! [option_2]
+
+def available_devices_1():
+#! [available_devices_1]
+    all_devices = "MULTI:"
+    core = Core()
+    model = core.read_model(model_path)
+    all_devices += ",".join(core.available_devices)
+    compiled_model = core.compile_model(model=model, device_name=all_devices)
+#! [available_devices_1]
+
+def available_devices_2():
+#! [available_devices_2]
+    match_list = []
+    all_devices = "MULTI:"
+    dev_match_str = "MYRIAD"
+    core = Core()
+    model = core.read_model(model_path)
+    for d in core.available_devices:
+        if dev_match_str in d:
+            match_list.append(d)
+    all_devices += ",".join(match_list)
+    compiled_model = core.compile_model(model=model, device_name=all_devices)
+#! [available_devices_2]
+
+def set_property():
+#! [set_property]
+    core = Core()
+    cpu_config = {}
+    gpu_config = {}
+    model = core.read_model(model_path)
+    core.set_property(device_name="CPU", properties=cpu_config)
+    core.set_property(device_name="GPU", properties=gpu_config)
+    compiled_model = core.compile_model(model=model, device_name="MULTI:GPU,CPU")
+#! [set_property]
+
+def main():
+    Option_1()
+    Option_2()
+    available_devices_1()
+    available_devices_2()
+    set_property()
+
+if __name__ == '__main__':
+    sys.exit(main())