notebooks/101-tensorflow-to-openvino/101-tensorflow-to-openvino.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JwEAhQVzkAwA"
   },
   "source": [
    "# TensorFlow to OpenVINO\n",
    "\n",
    "This short tutorial shows how to convert a TensorFlow MobilenetV3 model to OpenVINO IR format.\n",
    "Model documentation: https://docs.openvinotoolkit.org/latest/omz_models_model_mobilenet_v3_small_1_0_224_tf.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QB4Yo-rGGLmV"
   },
   "source": [
    "### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2ynWRum4iiTz"
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import sys\n",
    "import time\n",
    "from pathlib import Path\n",
    "\n",
    "import cv2\n",
    "import matplotlib.pyplot as plt\n",
    "import mo_tf\n",
    "import numpy as np\n",
    "from openvino.inference_engine import IECore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The paths of the source and converted models\n",
    "model_path = Path(\"model/v3-small_224_1.0_float.pb\")\n",
    "ir_path = Path(model_path).with_suffix(\".xml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6JSoEIk60uxV"
   },
   "source": [
    "## Convert the Model to OpenVINO IR Format\n",
    "\n",
    "### Convert TensorFlow model to OpenVINO IR Format\n",
    "\n",
    "Call the OpenVINO Model Optimizer tool to convert the TensorFlow model to OpenVINO IR, with FP16 precision. The models are saved to the current directory. We add the mean values to the model and scale the output with the standard deviation with `--scale_values`. With these options, it is not necessary to normalize input data before propagating it through the network. The original model expects input images in RGB format. The converted model also expects images in RGB format. If you want the converted model to work with BGR images, you can use the `--reverse-input-channels` option. See the [Model Optimizer Developer Guide](https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html) for more information about Model Optimizer, including a description of the command line options. Check the [model documentation](https://docs.openvinotoolkit.org/latest/omz_models_model_mobilenet_v3_small_1_0_224_tf.html) for information about the model, including input shape, expected color order and mean values. \n",
    "\n",
    "We first construct the command for Model Optimizer, and then execute this command in the notebook by prepending the command with a `!`. There may be some errors or warnings in the output. Model Optimization was succesful if the last lines of the output include `[ SUCCESS ] Generated IR version 10 model.\n",
    "`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Get the path to the Model Optimizer script\n",
    "mo_path = str(Path(mo_tf.__file__))\n",
    "\n",
    "# Construct the command for Model Optimizer\n",
    "mo_command = f\"\"\"\"{sys.executable}\"\n",
    "                 \"{mo_path}\" \n",
    "                 --input_model \"{model_path}\" \n",
    "                 --input_shape \"[1,224,224,3]\" \n",
    "                 --mean_values=\"[127.5,127.5,127.5]\"\n",
    "                 --scale_values=\"[127.5]\" \n",
    "                 --data_type FP16 \n",
    "                 --output_dir \"{model_path.parent}\"\n",
    "                 \"\"\"\n",
    "mo_command = \" \".join(mo_command.split())\n",
    "print(\"Model Optimizer command to convert TensorFlow to OpenVINO:\")\n",
    "print(mo_command)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run Model Optimizer if the IR model file does not exist\n",
    "if not ir_path.exists():\n",
    "    print(\"Exporting TensorFlow model to IR... This may take a few minutes.\")\n",
    "    ! $mo_command\n",
    "else:\n",
    "    print(f\"IR model {ir_path} already exists.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test Inference on the Converted Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ie = IECore()\n",
    "net = ie.read_network(str(ir_path))\n",
    "exec_net = ie.load_network(net, \"CPU\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get model information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_key = list(exec_net.input_info)[0]\n",
    "output_key = list(exec_net.outputs.keys())[0]\n",
    "network_input_shape = exec_net.input_info[input_key].tensor_desc.dims"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load image\n",
    "\n",
    "Load an image, resize it, and convert it to network input shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image = cv2.cvtColor(cv2.imread(\"data/coco.jpg\"), cv2.COLOR_BGR2RGB)\n",
    "# Resize image to network input image shape\n",
    "resized_image = cv2.resize(image, (224, 224))\n",
    "# Transpose image to network input shape\n",
    "input_image = np.reshape(resized_image, network_input_shape) / 255\n",
    "input_image = np.expand_dims(np.transpose(resized_image, (2, 0, 1)), 0)\n",
    "plt.imshow(image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Do inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = exec_net.infer(inputs={input_key: input_image})[output_key]\n",
    "result_index = np.argmax(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "imagenet_classes = json.loads(open(\"utils/imagenet_class_index.json\").read())\n",
    "imagenet_classes = {\n",
    "    int(key) + 1: value for key, value in imagenet_classes.items()\n",
    "}\n",
    "imagenet_classes[result_index]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Timing\n",
    "\n",
    "Measure the time it takes to do inference on thousand images. This gives an indication of performance. For more accurate benchmarking, use the [OpenVINO benchmark tool](https://github.com/openvinotoolkit/openvino/tree/master/inference-engine/tools/benchmark_tool). Note that many optimizations are possible to improve the performance. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_images = 1000\n",
    "start = time.perf_counter()\n",
    "for _ in range(num_images):\n",
    "    exec_net.infer(inputs={input_key: input_image})\n",
    "end = time.perf_counter()\n",
    "time_ir = end - start\n",
    "print(\n",
    "    f\"IR model in Inference Engine/CPU: {time_ir/num_images:.4f} \"\n",
    "    f\"seconds per image, FPS: {num_images/time_ir:.2f}\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "OpenVINO 2021.3 PIP installer - PyTorch Image Segmentation.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "openvino_env",
   "language": "python",
   "name": "openvino_env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}