shuttle_three_days.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sfmta\n",
    "import os\n",
    "\n",
    "os.environ['SHUTTLE_DB_USER'] = \"postgres\"\n",
    "os.environ['SHUTTLE_DB_PASSWORD'] =''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "conn = sfmta.db_connect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sfmta.main(conn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "# WIP--------\n",
    "\n",
    "## summary table\n",
    "- Summary table by CNN \n",
    "- Summary table by stops (every second within 50 feet of a stop point, it will be a stop)\n",
    "- periodic calculations (populate the summary tables, cron jobs?, another server - lambda), how often will they be peformed\n",
    "- cron job to run in postgres dd?\n",
    "- python function to start\n",
    "\n",
    "\n",
    "## SQL queries\n",
    "- volume per CNN\n",
    "- route(trip) identification\n",
    "- sopping and update stoping table\n",
    "- point inside/near CNN polygon\n",
    "- distance from polygon\n",
    "-\n",
    "\n",
    "## dropdown UI\n",
    "- (done) timestamp with date fields\n",
    "- how to update parameters with each field\n",
    "\n",
    "\n",
    "## Implementation\n",
    "1. upsert? function to tie each individual point to cnn, use postgres windowing fucntion. gropupby and min on distance to cnn (to associate cnn to points)\n",
    "2. stops, companies and vehicles added to the db schema\n",
    "3. summary of stops and frequency of stops, add table of stops\n",
    "4. self report stops by changing a field, initially look at self reporting and see how accurate it is\n",
    "- query to look at min/max time stamp with every group of points to classify stops\n",
    "- flter for specific stop\n",
    "\n",
    "\n",
    "\n",
    "### When should we persist data?\n",
    "- summaries by CNN (every half hour at a 15min lag)\n",
    "- \n",
    "\n",
    "- We should compare apples to apples both solutions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "sfmta",
   "language": "python",
   "name": "sfmta"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}