2-Classification (face-to-emoji).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Computer Vision Classification\n",
    "\n",
    "In this workshop, we'll explore the concept of **supervised learning**, where our dataset isn't just data, it includes **labels**. \n",
    "\n",
    "## Face-to-Emoji\n",
    "\n",
    "Let's imagine that you want to make a Face-To-Emoji converter, so that whenever you smile, a smiley emoji appears. When you're frowning, a frowny emoji appears. \n",
    "\n",
    "## This is similar to classification of Farsi\n",
    "\n",
    "This is similar to the Farsi Flashcard game. How??? You might ask. Remember how to trained on a *training dataset* and then got tested on a *test set*?\n",
    "\n",
    "## How it would work\n",
    "\n",
    "Here's how the flow would look:\n",
    "\n",
    "**Step 1: Train with Training Set**\n",
    "- Get lots of images of people's facial expressions, make sure they're labeled (smile/frown/surprise, etc.)\n",
    "- Convert the images to feature representations\n",
    "- Train your classifier (brain) to learn the correspondences between features and labels\n",
    "\n",
    "**Step 2: Test with Test Set**\n",
    "- Get a test image (no label)\n",
    "- Convert it to a feature representation\n",
    "- Using your trained classifer (brain), guess its the label (smile/frown/surprise/other) based on its features\n",
    "\n",
    "Once you have the label, then you can convert it to the right emoji! 😲😦😊 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "\n",
    "Let's start by taking this dataset, called the CK+ (Cohn Kanade) dataset. As a reminder, the images look as below:\n",
    "\n",
    "|S078_005_00000013.png | S088_004_00000011.png | S094_004_00000012.png | S098_004_00000012.png | S112_001_00000020.png|\n",
    "|---|---|---|---|---|\n",
    "|  <img width=\"120px\" src=\"img/S078_005_00000013.png\"> |  <img width=\"120px\" src=\"img/S088_004_00000011.png\">  | <img width=\"120px\" src=\"img/S094_004_00000012.png\">  |  <img width=\"120px\" src=\"img/S098_004_00000012.png\"> | <img width=\"120px\" src=\"img/S112_001_00000020.png\">  |\n",
    "\n",
    "But this time, we're going to take advantage of an extra piece of information. \n",
    "\n",
    "## Annotation\n",
    "You see, there were some **annotators** that were hired to look at each photo and give it a label. Now our dataset looks like this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas\n",
    "import matplotlib.pyplot as plot\n",
    "from sklearn.cluster import KMeans\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "# Import the data\n",
    "raw_data = pandas.read_csv('data/facs-labels.csv') \n",
    "dataset = raw_data[:20]  # Let's just get 20 photos\n",
    "dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to Classify with K Nearest Neighbours\n",
    "\n",
    "So let's remember again how we did classification with the Farsi dataset. We could simply say \"hey, which one out of these training images looks the most similar to our test image?\"\n",
    "\n",
    "And then you could check the back of the card, find its label, and assume that the test image has the same label. \n",
    "\n",
    "This is called **1-Nearest Neighbour**. Can you guess what **2-Nearest Neighbours** might mean?\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare the Training and Test data\n",
    "\n",
    "Now let's split up our big dataset into training and testing data, just like we did with the Farsi dataset.\n",
    "\n",
    "## 1. Create a training dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import pandas\n",
    "\n",
    "# Read the data\n",
    "raw_data = pandas.read_csv('data/facs-labels.csv')\n",
    "train = raw_data[:100]  # Training set: get 100 photos\n",
    "\n",
    "# Let's see the training data\n",
    "train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Create a testing example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's just try it out with one test example for now\n",
    "test = raw_data[101:102] # Test set: just 1 photo for now\n",
    "test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Train a K-Nearest Neighbours Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This classifier is basically just like a piece of memory that remembers reaaaaally well. Like a photographic memory. It can pick out things it has already seen (from the training set) that look similar to your test photo."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's train and test our K-Nearest Neighbors classifier!\n",
    "\n",
    "Now, we can run our classifier on our test data to see how well it worked."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy\n",
    "from sklearn import neighbors\n",
    "\n",
    "# Prepare the training data and labels\n",
    "features = pandas.DataFrame(train).loc[:,'AU1: Inner Brow Raiser':'AU64: Eyes down']\n",
    "X = features.values\n",
    "y = train['LBL']\n",
    "\n",
    "# Set parameters for our classifier, in this case set K=1\n",
    "n_neighbors = 1\n",
    "\n",
    "######## TRAINING ########\n",
    "# Create the classifier with 100 training data images\n",
    "classifier = neighbors.KNeighborsClassifier(n_neighbors)\n",
    "classifier.fit(X, y)\n",
    "\n",
    "######## TESTING ##########\n",
    "# Run the classifer on the 101st test image\n",
    "testcase = pandas.DataFrame(test).loc[:, 'AU1: Inner Brow Raiser':'AU64: Eyes down']\n",
    "print(classifier.predict(testcase))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Your Turn\n",
    "\n",
    "Try testing your classifier with other photos from the dataset. Make sure your testing example is not in the training set! Why?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write if you were able to successfully recognize/classify each image you tested. Any that didn't work?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- What happens if your training dataset is small? Try it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What values did you try? What happened?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculating Accuracy\n",
    "\n",
    "Now, we did this only on one sample. But how can we tell how good our classifier is? We need to take an average over many different photos.\n",
    "\n",
    "Let's break up our labeled data into training and test sets. Then we can calculate the classifier's accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "minitrain = train[1:50]\n",
    "minitest = train[50:100] # Test set: just 1 photo for now"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}