Practical-4-Point Patterns.ipynb

{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Point Patterns: Part 1 - Generating CSR\n",
    "\n",
    "In this practical we are going to \n",
    "(1) explore how random numbers and random points can be generated.\n",
    "(2) appreciate the nature of uniform random points and how they are distributed.\n",
    "(3) measure the centrality and the spatial dispersion of random points.\n",
    "(4) use Excel to quickly check the outcomes of basic exploratory data analysis.\n",
    "(5) follow a point pattern practical using PySAL to familiarise ourselves with the process.\n",
    "[Bonus: Install and use GeoDa.]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (1) Try generating a random integer between 0 and N (where N is an integer and N>0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pysal as ps\n",
    "# import pandas as pd\n",
    "# import geopandas as gpd\n",
    "# import seaborn as sns\n",
    "# from shapely.geometry import Point\n",
    "# from pysal.contrib.viz import mapping as maps # For maps.plot_choropleth\n",
    "\n",
    "import random\n",
    "random.randint(0,N)\n",
    "\n",
    "# Make sure output is written into notebook\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This should give you an integer randomly selected between 0 and N. Note that the range from which a random integer is selected INCLUDES both 0 and N (i.e. the boundaries are included, as in [0…N] ).\n",
    "\n",
    "Bonus Question 1: What would you change to prevent from either 0 or N being selected?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (2) Let us now think of generating a random number between 0.0 and 1.0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print random.random()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To specify a range other than 0.0 to 1.0, try random.uniform(a,b) where a < b.\n",
    "\n",
    "Try generating five random numbers in a row: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for i in range(5):\n",
    "print ‘%.6f’ % random.random()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bonus Question 2: How do we know if these are purely random? Or are they merely pseudo-random? If so, how do these programmes decide where to start and what do they do to randomise the values? Would that be good enough for the purpose of (most) applications in geocomputation and spatial analysis?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (3) Let us move on to producing random points.\n",
    "We use the same principle as above except, this time, we must specify both the x and the y values. By default, they will be generated between (0,0) and (1,1), but can be transformed to adapt to any value range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print ('%.6f' % random.random()),('%.6f' % random.random())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Let us generate a set of random points.\n",
    "  for i in range(10):\n",
    "      print ('%.6f' % random.random()),('%.6f' % random.random())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Question: Think about how these points are distributed. Are they different from one set to another? Why?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (4) Generate files containing random points and plot them within a square area of [(0,0), (1,1)].\n",
    "\n",
    "Bonus Question 3: Try using randn(num) instead and see what happens. It produces a normal or Gaussian distribution of random numbers with a mean of 0 and a standard deviation of 1 (i.e. its histogram should form a bell curve). \n",
    "What kind of geographical phenomena or events do you think can be simulated by normal random distribution?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 2: Thinking about the Spatial Patterns of Random Points\n",
    "\n",
    "(1) Let us think of the mean centre of the random points.\n",
    "Exercise 1(a): Can you calculate the spatial mean?\n",
    "\n",
    "Question: If the points are uniformly randomly distributed, between (0,0) and (1,1), where would the spatial mean most likely be found? Why?\n",
    "\n",
    "(2) Repeat producing a set of random points.\n",
    "For each set, calculate the spatial mean, and see how they change their position from one set to another.\n",
    "\n",
    "Question: Is there a way to make the spatial mean more stable?\n",
    "\n",
    "(3) Just as we do in ordinary statistics, we measure the scatteredness of observations on a single variable about their mean by the standard deviation. In other words, we can measure the degree of spatial dispersion of a point pattern by its standard distance ds. It is the square root of the average square of the distances from every point to the spatial mean: i.e.   where lic is the distance of the ith point from the spatial mean. Try calculating this and produce the circle of influence around the spatial mean."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 3: Using an Excel Spreadsheet\n",
    "\n",
    "Coding in Python (in an open source environment) is great, as it gives you (1) the complete control and flexibility over what you do and (2) not having to pay the license/package fees of commercial products, especially when it comes to specialist software such as GIS (e.g. ArcGIS). At the same time, it can be also tedious if you simply wanted to quickly process something that is rudimentary. \n",
    "Here is an example of what you can do with a standard spreadsheet package:\n",
    "\n",
    "(1) Using MS-Excel, can you repeat the above (generating random numbers and random points and finding the mean centre, as well as its spatial dispersion)?\n",
    "\n",
    "Note that Excel uses a set of simple formulae. Random values can be produced with “=rand()”, while formulas like “average([cell 1]:[cell x])” would give you the average, and the cell position can be fixed by designating them with a $ sign; e.g. =+SQRT(([cell x1] - [cell $x$m])^2 + ([cell y1] - [cell $y$m])^2)\n",
    "where cells [cell $x$m] and [cell $y$m] can respectively hold the x and y values of the mean centre.\n",
    "\n",
    "The other handy feature is that we can easily copy formula over to the next cell, and drape the same feature/formula along rows or columns.\n",
    "\n",
    "(2) Can you plot the random points, as well as the mean centre in a chart?\n",
    "(Note that you should use an xy-scatter plot.)\n",
    "\n",
    "(3) Question: Can you produce histograms their distribution (one for the x-values and another for y-values)? How are the bins (or the range) determined? If you change the size and number of bins, how would that affect the outcome? What does that tell you?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bonus Task: Install GeoDa and Try It Out\n",
    "GeoDa and PySal are some of the open-source packages for spatial analysis, developed and offered by a group of researchers at Arizona State University, including Prof. Luc Anselin and Prof. Sergio Rey.\n",
    "From GeoDa project page: “GeoDa is a free, open source, cross-platform software program that serves as an introduction to spatial data analysis. It runs on different versions of Windows (including XP, Vista, 7 and 8), Mac OS, and Linux.” \n",
    "You can download GeoDa from here:https://spatial.uchicago.edu/software\n",
    "\n",
    "If you are installing in the GeoCUP environment, please make sure that you are downloading the package for 32bit Ubuntu Linux version.\n",
    "\n",
    "(1) Open/Read some of the files (containing a set of random points) you have created earlier. \n",
    "(2) Plot these points and find out the basic EDAs.\n",
    "(3) Produce histograms of the distribution of their x-values, and that of their y-values. Then change the number of bins and see how that affects the outcome.\n",
    "\n",
    "When you are done, please make sure to save all files on your GeoCUP.\n",
    "Don’t forget to log out (and take your GeoCUP with you!)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Take Home Task\n",
    "Download the dataset on the location of the AirBnb. Plot them on a map. Do you think they are randomly distributed, or are they forming some patterns? How can we find out for sure?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore Point Patterns in PySAL\n",
    "\n",
    "Go to \"Planar Point Patterns in PySAL\" by Serge Rey (sjsrey@gmail.com) to explore point patterns in PySAL. (http://nbviewer.jupyter.org/github/sjsrey/giasp16/blob/master/content/pages/notebooks/pointpattern.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Point Patterns: Part 2 - Methods of Point Pattern Analysis in PySAL\n",
    "\n",
    "The points module in PySAL implements basic methods of point pattern analysis organized into the following groups:\n",
    "\n",
    "Point Processing\n",
    "Centrography and Visualization\n",
    "Quadrat Based Methods\n",
    "Distance Based Methods\n",
    "\n",
    "In the remainder of this notebook we shall focus on point processing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pysal as ps\n",
    "import numpy as np\n",
    "from pysal.contrib.points.pointpattern import PointPattern"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating Point Patterns\n",
    "\n",
    "From lists\n",
    "\n",
    "We can build a point pattern by using Python lists of coordinate pairs (s0,s1,…,sm)(s0,s1,…,sm) as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "points = [[66.22, 32.54], [22.52, 22.39], [31.01, 81.21],\n",
    "          [9.47, 31.02],  [30.78, 60.10], [75.21, 58.93],\n",
    "          [79.26,  7.68], [8.23, 39.93],  [98.73, 77.17],\n",
    "          [89.78, 42.53], [65.19, 92.08], [54.46, 8.48]]\n",
    "p1 = PointPattern(points)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "p1.mbb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "array([  8.23,   7.68,  98.73,  92.08])\n",
    "Thus s0=(66.22,32.54), s11=(54.46,8.48)s0=(66.22,32.54), s11=(54.46,8.48).\n",
    "\n",
    "In [4]:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "p1.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Point Pattern\n",
    "12 points\n",
    "Bounding rectangle [(8.23,7.68), (98.73,92.08)]\n",
    "Area of window: 7638.2\n",
    "Intensity estimate for window: 0.00157105077112\n",
    "       x      y\n",
    "0  66.22  32.54\n",
    "1  22.52  22.39\n",
    "2  31.01  81.21\n",
    "3   9.47  31.02\n",
    "4  30.78  60.10\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "type(p1.points)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "pandas.core.frame.DataFrame\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.asarray(p1.points)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "array([[ 66.22,  32.54],\n",
    "       [ 22.52,  22.39],\n",
    "       [ 31.01,  81.21],\n",
    "       [  9.47,  31.02],\n",
    "       [ 30.78,  60.1 ],\n",
    "       [ 75.21,  58.93],\n",
    "       [ 79.26,   7.68],\n",
    "       [  8.23,  39.93],\n",
    "       [ 98.73,  77.17],\n",
    "       [ 89.78,  42.53],\n",
    "       [ 65.19,  92.08],\n",
    "       [ 54.46,   8.48]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "p1.mbb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "array([  8.23,   7.68,  98.73,  92.08])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### From numpy arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "points = np.asarray(points)\n",
    "points"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "array([[ 66.22,  32.54],\n",
    "       [ 22.52,  22.39],\n",
    "       [ 31.01,  81.21],\n",
    "       [  9.47,  31.02],\n",
    "       [ 30.78,  60.1 ],\n",
    "       [ 75.21,  58.93],\n",
    "       [ 79.26,   7.68],\n",
    "       [  8.23,  39.93],\n",
    "       [ 98.73,  77.17],\n",
    "       [ 89.78,  42.53],\n",
    "       [ 65.19,  92.08],\n",
    "       [ 54.46,   8.48]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "p1_np = PointPattern(points)\n",
    "p1_np.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Point Pattern\n",
    "12 points\n",
    "Bounding rectangle [(8.23,7.68), (98.73,92.08)]\n",
    "Area of window: 7638.2\n",
    "Intensity estimate for window: 0.00157105077112\n",
    "       x      y\n",
    "0  66.22  32.54\n",
    "1  22.52  22.39\n",
    "2  31.01  81.21\n",
    "3   9.47  31.02\n",
    "4  30.78  60.10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### From shapefiles\n",
    "This example uses 200 randomly distributed points within the counties of Virginia. Coordinates are for UTM zone 17 N."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "f = ps.examples.get_path('vautm17n_points.shp')\n",
    "fo = ps.open(f)\n",
    "pp_va = PointPattern(np.asarray([pnt for pnt in fo]))\n",
    "fo.close()\n",
    "pp_va.summary()\n",
    "Point Pattern\n",
    "200 points\n",
    "Bounding rectangle [(273959.664381,4049220.90341), (972595.989578,4359604.85978)]\n",
    "Area of window: 2.16845506675e+11\n",
    "Intensity estimate for window: 9.22315629531e-10\n",
    "               x             y\n",
    "0  865322.486181  4.150317e+06\n",
    "1  774479.213103  4.258993e+06\n",
    "2  308048.692232  4.054700e+06\n",
    "3  670711.529980  4.258864e+06\n",
    "4  666254.475614  4.256514e+06"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "* Please refer to http://nbviewer.jupyter.org/github/sjsrey/giasp16/blob/master/content/pages/notebooks/pointpattern.ipynb for further study on the attribute of PySAL Point Patterns."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [spats]",
   "language": "python",
   "name": "Python [spats]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}