Oil tanks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to count the number of oil tanks in Houston TX, a major global oil storage facility.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a GBDX interface using [gbdxtools](https://github.com/digitalglobe/gbdxtools). You need your credentials to do this; you can find them under your profile on gbdx.geobigdata.io."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['GBDX_USERNAME'] = \n",
    "os.environ['GBDX_PASSWORD'] = \n",
    "os.environ['GBDX_CLIENT_ID'] =  \n",
    "os.environ['GBDX_CLIENT_SECRET'] = \n",
    "\n",
    "import gbdxtools\n",
    "gbdx = gbdxtools.Interface()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the catalog to query for imagery over Houston."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "catid: 104001001838A000\n",
      "cloud cover: 0\n",
      "off nadir: 7.41784572601\n",
      "catid: 10400100183BD500\n",
      "cloud cover: 0\n",
      "off nadir: 21.641210556\n",
      "catid: 1040010023558B00\n",
      "cloud cover: 0\n",
      "off nadir: 11.3405036926\n",
      "catid: 1040010018412D00\n",
      "cloud cover: 0\n",
      "off nadir: 10.6088581085\n"
     ]
    }
   ],
   "source": [
    "W, S, E, N = (-95.14483630657196, 29.696617936567343, -94.98828113079071, 29.773830057098092)\n",
    "\n",
    "wkt_string = \"POLYGON(({} {}, {} {}, {} {}, {} {}, {} {}))\".format(W, S, W, N, E, N, E, S, W, S)\n",
    "\n",
    "filters = [\"sensorPlatformName = 'WORLDVIEW03_VNIR'\",\n",
    "           \"cloudCover < 10\",\n",
    "           \"imageBands = 'Pan_MS1_MS2'\"]\n",
    "\n",
    "results = gbdx.catalog.search(searchAreaWkt=wkt_string,\n",
    "                              startDate=\"2016-01-01T00:00:00.000Z\",\n",
    "                              endDate=\"2016-11-01T00:00:00.000Z\",\n",
    "                              filters=filters)\n",
    "\n",
    "# Results is a dictionary with multiple entries. Each entry corresponds to an image over the area of interest.\n",
    "for result in results:\n",
    "    properties = result['properties']\n",
    "    print 'catid: ' + properties['catalogID']\n",
    "    print 'cloud cover: ' + str(properties['cloudCover'])\n",
    "    print 'off nadir: ' + str(properties['offNadirAngle'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pick an image; the lowest the cloud cover and the off nadir angle, the better the quality of the results.\n",
    "We've selected 104001001838A000 (WV3, 2016).\n",
    "\n",
    "The following batch GBDX workflow orders the raw image from the DG factory, and then produces an orthorectified panchromatic image via the AOP_Strip_Processor. Once the workflow completes, the image is stored under platform-stories/oil-tanks/image-houston-pan. We've already run this workflow for you so you can skip this step if you want and go straight to the oil tank detection part."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "catid = '104001001838A000'\n",
    "\n",
    "# create order task\n",
    "# it the images are not on GBDX, this task will order them from the DG factory\n",
    "order = gbdx.Task('Auto_Ordering')\n",
    "order.inputs.cat_id = catid\n",
    "# for this particular task, we need to set this flag to true\n",
    "order.impersonation_allowed = True\n",
    "\n",
    "# panchromatic\n",
    "aop_pan = gbdx.Task('AOP_Strip_Processor')\n",
    "aop_pan.inputs.data = order.outputs.s3_location.value\n",
    "aop_pan.inputs.bands = 'PAN'\n",
    "aop_pan.inputs.enable_acomp = False\n",
    "aop_pan.inputs.enable_pansharpen = False\n",
    "aop_pan.inputs.enable_dra = False\n",
    "aop_pan.inputs.ortho_epsg='UTM'\n",
    "\n",
    "# define preprocessing workflow\n",
    "preprocess_wf = gbdx.Workflow([order, aop_pan])\n",
    "\n",
    "# set output location \n",
    "output_location = 'platform-stories/oil-tanks/image-houston'\n",
    "preprocess_wf.savedata(aop_pan.outputs.data, output_location + '-pan')\n",
    "\n",
    "# execute\n",
    "preprocess_wf.execute()\n",
    "preprocess_wf.status"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll now use oil-tank-detector to detect oil tanks on this image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "u'4721672104198972138'"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "otd = gbdx.Task('oil-tank-detector')\n",
    "otd.inputs.image = 's3://gbd-customer-data/32cbab7a-4307-40c8-bb31-e2de32f940c2/platform-stories/oil-tanks/image-houston-pan'\n",
    "otd.inputs.bbox = ','.join(map(str, (W, S, E, N)))\n",
    "\n",
    "wf = gbdx.Workflow([otd])\n",
    "wf.savedata(otd.outputs.detections, 'platform-stories/trial-runs/oil-tanks')\n",
    "wf.execute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{u'event': u'succeeded', u'state': u'complete'}"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wf.status"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the workflow is complete, download the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "from os.path import join\n",
    "gbdx.s3.download(location='platform-stories/trial-runs/oil-tanks/detections', local_dir='detections')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Put the detections on a map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 1428 oil tanks in the area of interest\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8d1a6d4727f344199c831a59345abd63"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create slippy map\n",
    "from ipyleaflet import Map, TileLayer, GeoJSON\n",
    "import json\n",
    "\n",
    "m = Map(center=[29.71, -95.03], zoom=13)\n",
    "\n",
    "# This is the Mapbox TMS URL\n",
    "mapbox_token = 'pk.eyJ1IjoicGxhdGZvcm1zdG9yaWVzIiwiYSI6ImNpeTZkeWlvOTAwNm0yeHFocHFyaGFleDcifQ.wOsbVsS0NXKrWeX2bQwc-g'\n",
    "url = 'https://a.tiles.mapbox.com/v4/platformstories.oil-tanks-houston/{z}/{x}/{y}.png?access_token=' + mapbox_token\n",
    "\n",
    "# add raster layer\n",
    "m.add_layer(TileLayer(url=url))\n",
    "  \n",
    "with open('detections/detections.geojson') as f:\n",
    "    data = json.load(f)\n",
    "        \n",
    "n_oil_tanks = len(data['features'])\n",
    "print 'There are ' + str(n_oil_tanks) + ' oil tanks in the area of interest'\n",
    "\n",
    "g = GeoJSON(data=data)\n",
    "\n",
    "# add vector layer\n",
    "m.add_layer(g)\n",
    "\n",
    "m        "
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}