notebooks/Step5_Deployment_Pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy the model to Azure\n",
    "The goal of the notebook is to deploy the model from Step 4 to Azure. The deployment notebook can be extended to other models. \n",
    "\n",
    "The deploy pipelines have the following steps:\n",
    "1. Build docker image\n",
    "2. Deploy azure container registry\n",
    "3. Push docker image to registry\n",
    "4. Deploy the model in Kubernetes cluster by pulling images from the ACR registry.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "\n",
    "### Test images\n",
    "- An image URL is required to test the published Web service.\n",
    "\n",
    "### Pretrained models\n",
    "- [Frontal face landmark model](https://github.com/AKSHAYUBHAT/TensorFace/blob/master/openface/models/dlib/shape_predictor_68_face_landmarks.dat) in the same directory as this jupyter notebook.\n",
    "- [One Eye model](https://raw.githubusercontent.com/opencv/opencv/master/data/haarcascades/haarcascade_eye.xml) in the same directory as this jupyter notebook.\n",
    "- [ResNet152\\_ImageNet\\_Caffe.model](https://www.cntk.ai/Models/Caffe_Converted/ResNet152_ImageNet_Caffe.model)\n",
    "- Trained full connected neural network regression model from ***[Step 3. Training_Pipeline](Step3_Training_Pipeline.ipynb)***\n",
    "\n",
    "All these models are stored in the models folder.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Write main python file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "mainfile = \"\"\"\\\n",
    "\n",
    "import os\n",
    "import flask\n",
    "import numpy as np\n",
    "from flask import jsonify, request\n",
    "from model import extract_patches, score_patch, del_cache\n",
    "\n",
    "app = flask.Flask(__name__)\n",
    "\n",
    "@app.route('/url/<path:argument>')\n",
    "def url(argument):\n",
    "    # create a patch folder\n",
    "    patch_path = './patches'\n",
    "    if not os.path.exists(patch_path):\n",
    "        os.mkdir(patch_path)\n",
    "    \n",
    "    # get image url from the query string\n",
    "    imageURL = request.url.split('=',1)[1]\n",
    "    \n",
    "    # extract patches from imageURL\n",
    "    dimension, face_loc, image_dim = extract_patches(imageURL)\n",
    "    \n",
    "    # score each patch\n",
    "    patch_score= score_patch(patch_path)\n",
    "    \n",
    "    # delete the downloaded image and the patches from local\n",
    "    del_cache(patch_path)\n",
    "    if os.path.exists('temp.jpg'):\n",
    "        os.remove('temp.jpg')\n",
    "    \n",
    "    data = dict()\n",
    "    data['patch_score'] = []\n",
    "    for key in dimension:\n",
    "        tmp = []\n",
    "        tmp[:] = dimension[key]\n",
    "        tmp.append(patch_score[key])\n",
    "        data['patch_score'].append(tmp)\n",
    "   \n",
    "    data['image_score'] = round(np.mean(list(patch_score.values())), 2) \n",
    "    data['face_loc'] = face_loc['face_loc']\n",
    "    data['img_dim'] = image_dim\n",
    "\n",
    "    return jsonify(patch_score = str(data['patch_score']), image_score = str(data['image_score']), face_loc = str(data['face_loc']), image_dim = str(data['img_dim']))\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    app.run(host='0.0.0.0', port = 9580) # port number can be changed in your case\n",
    "\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "mainpy_path = './main.py'\n",
    "with open(mainpy_path,'w') as f:\n",
    "    f.write(mainfile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Write docker file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "dockerfile = \"\"\"\\\n",
    "\n",
    "FROM microsoft/cntk:2.5.1-cpu-python3.5\n",
    " \n",
    "RUN dpkg --add-architecture i386 \\\n",
    "&& apt-get update && apt-get install -y \\\n",
    "build-essential \\\n",
    "git \\\n",
    "python3-pip \\\n",
    "python3-tk \\\n",
    "libglib2.0-0:i386 \\\n",
    "&& rm -rf /var/lib/apt/lists/*\n",
    "\n",
    "RUN pip3 install -U scikit-learn\n",
    "RUN pip3 install numpy pandas sklearn matplotlib\n",
    "RUN pip3 install scipy\n",
    "RUN pip3 install opencv-python-headless\n",
    "RUN pip3 install dlib\n",
    "RUN pip3 install scikit-image\n",
    "RUN pip3 install matplotlib\n",
    "RUN pip3 install cntk\n",
    "RUN pip3 install \\\n",
    "flask \\\n",
    "pillow\n",
    "\n",
    "RUN mkdir /usr/src/nestle\n",
    "WORKDIR /workspace\n",
    "RUN chmod -R a+w /workspace\n",
    "ADD ../models /workspace/models\n",
    "COPY main.py /workspace\n",
    "COPY regressionModel.py /workspace\n",
    "COPY getPatches.py /workspace\n",
    "COPY model.py /workspace\n",
    "RUN chmod +x /workspace/main.py\n",
    "RUN ls /workspace\n",
    "CMD python3 /workspace/main.py\n",
    " \n",
    "EXPOSE 9580\n",
    "\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "docker_path = './Dockerfile'\n",
    "with open(docker_path,'w') as f:\n",
    "    f.write(dockerfile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build docker image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run docker build command to build the nestleapi image.\n",
    "!sudo docker build -t 'nestleapi' ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To see the built image, use the docker images command.\n",
    "!sudo docker images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ACS parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "resourceGroupName = '<resource group name>' \n",
    "location ='<location>'\n",
    "acrName = '<acr name>'\n",
    "aksClusterName = '<cluster name>'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo $resourceGroupName\n",
    "!echo $location\n",
    "!echo $acrName\n",
    "!echo $aksClusterName"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy Azure Container Registry for NSH"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# login to Azure\n",
    "!az login"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create resource group\n",
    "!az group create --name $resourceGroupName --location $location"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create azure container registory under the resource group\n",
    "!az acr create --name $acrName --resource-group $resourceGroupName  --sku Basic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Login Succeeded\r\n"
     ]
    }
   ],
   "source": [
    "!sudo az acr login --name $acrName"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# list acr, resource group, location ......\n",
    "!az acr list --resource-group $resourceGroupName --output table "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Push images to registry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# tag the docker image \n",
    "!sudo docker tag nestleapi $acrName\".azurecr.io/nestleapi:v1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Once tagged, run docker images to verify the operation.\n",
    "!sudo docker images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Push the nestleapi image to the registry\n",
    "!sudo docker push $acrName\".azurecr.io/nestleapi:v1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result\r\n",
      "---------\r\n",
      "nestleapi\r\n"
     ]
    }
   ],
   "source": [
    "# To verify the above operation\n",
    "!az acr repository list --name $acrName --output table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Kubernetes cluster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a Kubernetes cluster\n",
    "!sudo az aks create --resource-group $resourceGroupName --name $aksClusterName --node-count 1 --generate-ssh-keys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Configure kubectl to connect to your Kubernetes cluster\n",
    "!sudo az aks get-credentials --resource-group $resourceGroupName --name $aksClusterName"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configure ACR authentication\n",
    "\n",
    "Authentication needs to be configured between the AKS cluster and the ACR registry. This involves granting the AKS identity the proper rights to pull images from the ACR registry."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the ID of the service principal configured for AKS.\n",
    "clientID = !az aks show --resource-group $resourceGroupName --name $aksClusterName --query \"servicePrincipalProfile.clientId\" --output tsv\n",
    "clientID = clientID.n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the ACR registry resource ID\n",
    "arcID = !az acr show --name $acrName --resource-group $resourceGroupName --query \"id\" --output tsv\n",
    "arcID = arcID.n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo $clientID\n",
    "!echo $arcID"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the role assignment with the values gathered in the last two steps, which grants the proper access \n",
    "!az role assignment create --assignee $clientID --role Reader --scope $arcID"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Write yml file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "ymlfile = \"\"\"\n",
    "apiVersion: apps/v1beta1\n",
    "kind: Deployment\n",
    "metadata:\n",
    "  name: nestleapi\n",
    "spec:\n",
    "  replicas: 1\n",
    "  template:\n",
    "    metadata:\n",
    "      labels:\n",
    "        app: nestleapi\n",
    "    spec:\n",
    "      containers:\n",
    "      - name: nestleapi\n",
    "        image: {}.azurecr.io/nestleapi:v1\n",
    "        ports:\n",
    "        - containerPort: 9580\n",
    "        resources:\n",
    "          requests:\n",
    "            cpu: 250m\n",
    "          limits:\n",
    "            cpu: 500m\n",
    "        env:\n",
    "        - name: LD_LIBRARY_PATH\n",
    "          value: \"$LD_LIBRARY_PATH:/usr/local/nvidia/lib64:/opt/conda/envs/py3.6/lib\"\n",
    "\n",
    "---\n",
    "apiVersion: v1\n",
    "kind: Service\n",
    "metadata:\n",
    "  name: nestleapi\n",
    "spec:\n",
    "  type: LoadBalancer\n",
    "  ports:\n",
    "  - port: 80\n",
    "    targetPort: 9580\n",
    "  selector:\n",
    "    app: nestleapi\n",
    "\n",
    "\"\"\".format(acrName)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "yml_path = './nestledeploy.yml'\n",
    "with open(yml_path,'w') as f:\n",
    "    f.write(ymlfile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy application"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment.apps \"nestleapi\" created\n",
      "service \"nestleapi\" created\n"
     ]
    }
   ],
   "source": [
    "# Deploy the application to the Kubernetes cluster we created in the last step\n",
    "!sudo kubectl apply -f nestledeploy.yml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check status until STATUS = Running\n",
    "!sudo kubectl get pods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test Application\n",
    "\n",
    "A Kubernetes service is created which exposes the application to the internet. This process can take a few minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To monitor progress, use the kubectl get service command with the --watch argument\n",
    "# Initially the EXTERNAL-IP for the nestleapi service appears as pending. \n",
    "# Once the EXTERNAL-IP address has changed from pending to an IP address, use CTRL-C to stop the kubectl watch process.\n",
    "!sudo kubectl get service nestleapi --watch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Replace external_ip with the return result from the above\n",
    "!curl -i -H \"Accept: application/json\" -H \"Content-Type: application/json\" http://<external_ip>/url/ImageURL='<image URL>'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}