10. Convolutional Neural Networks - R.ipynb

{
  "cells": [
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Convolutional Neural Networks\n======\n\nConvolutional neural networks (CNNs) are a class of deep neural networks, most commonly used in computer vision applications.\n\nConvolutional refers the network pre-processing data for you - traditionally this pre-processing was performed by data scientists. The neural network can learn how to do pre-processing *itself* by applying filters for things such as edge detection.\n\nThis exercise uses keras and another library, which we need to load before we begin.\n\n**Run the code below to start loading the required libraries for this exercise.**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box to load libraries\n\n# Load libraries\nsuppressMessages(install.packages(\"keras\"))\nsuppressMessages(install.packages(\"stringr\"))\nsuppressMessages(library(keras))\nsuppressMessages(install_keras())\nsuppressMessages(library(stringr))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 1\n-----\n\nIn this exercise we will train a CNN to recognise handwritten digits, using the MNIST digit dataset.\n\nThis is a very common exercise and data set to learn from.\n\nLet's start by load our dataset and setting up our training and test sets (keras will automatically assign a validation set from the training set for us).\n\n### In the cell below replace:\n#### 1. `<selectTrainingSetX>` with `1:1000,,`\n#### 2. `<selectTrainingSetY>` with `1:1000`\n#### 3. `<selectTestSetX>` with `1001:1500,,`\n#### 4. `<selectTestSetY>` with `1001:1500`\n#### then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box to load the dataset and split it into training and test sets\n\n# Here we import the dataset.\nmnist <- dataset_mnist()\n\n\n# This stores our features and labels for both our training and test sets as local variables\n###\n# REPLACE <selectTrainingSetX> WITH 1:6000,, AND <selectTrainingSetY> WITH 1:6000\n###\nraw_x_train <- mnist$train$x[<selectTrainingSet>]\nraw_y_train <- mnist$train$y[<selectTrainingSet>]\n###\n\n###\n# REPLACE <selectTestSetX> WITH 6001:8000,, AND <selectTestSetY> WITH 6001:8000\n###\nraw_x_test <- mnist$test$x[<selectTestSet>]\nraw_y_test <- mnist$test$y[<selectTestSet>]\n###\n\n# This tells us the dimensions of our training set's features\ndim(raw_x_train)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Expected output:  \n`1000  28  28`\n\nSo we have 1,000 training samples.\n\n\nThe two 28's after the 1,000 tell us each sample is 28 pixels wide and 28 pixels high.\n\nEach pixel is really just a number from 0 to 255 - 0 being fully black, 255 being fully white - so the images are greyscale. When we graph the 28x28 numbers, we can see the image.\n\nStep 2\n============\n\nSo, let's have a look at one of our samples.\n\n**Run the code below**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box to look at one of our images\nim <- raw_x_train[1,,]\nim <- t(apply(im, 2, rev)) \nimage(1:28, 1:28, im, col=gray((0:255)/255), xaxt='n', main=paste(raw_y_train[1]))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Our first training image is `5`.\n\nNext, let's check out our test set.\n\n**Run the code below**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this to see the dimensions of our test set.\ndim(raw_x_test)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Expected output:  \n`500  28  28`\n\nAnd we have 500 test images!\n\nLet's take a look at the first image in the test set.\n\n**Run the code below**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this to look at the first image in the test set\nim <- raw_x_test[1,,]\nim <- t(apply(im, 2, rev)) \nimage(1:28, 1:28, im, col=gray((0:255)/255), xaxt='n', main=paste(raw_y_test[1]))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "You should see a 9 above. Looking good - next we will prepare our data for another neural network.\n\nStep 3\n---\n\nThe neural network will use the 28x28 values of each image to predict what each image represents.\n\nWe need to reshape our data to get it working well with our neural network. \n\n**Run the code below**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Read then run this code\n\n# First off, let's reshape our X sets so that they fit the convolutional layers.\nx_train <- array_reshape(raw_x_train, c(nrow(raw_x_train), 28, 28, 1))\nx_test <- array_reshape(raw_x_test, c(nrow(raw_x_test), 28, 28, 1))\n\n# Next up - feature scaling.\n# We scale the values so they are between 0 and 1, instead of 0 and 255.\nx_train <- x_train / 255\nx_test <- x_test / 255\n\n# Print the label associated with the first element in the training data set\nprint(raw_y_train[1])",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Expected output:  \n`5`\n\nThe label is a number - the number we see when we view the image.\n\nWe need represent this number as a category by using a one-hot vector, rather than an integer (a number). This is the same as if we were still trying to predict the breed of a dog.\n\nKeras can convert these numeric labels into one-hot vectors easily with the function - `to_categorical`\n\n#### Replace `<addCategorical>` with `to_categorical` and run the code."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# The 10 means that there are 10 different categories - 0 to 9\n###\n# REPLACE THE <addCategorical> BELOW WITH to_categorical\n###\ny_train <- <addCategorical>(raw_y_train, 10)\ny_test <- <addCategorical>(raw_y_test, 10)\n###\n\n# Print the label for the first element\nprint(y_train[1,])",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Expected output:  \n`[1] 0 0 0 0 0 1 0 0 0 0`\n\nStep 4\n-----\n\nAll ready! Time to build another neural network.\n\nWe need to add in convolutional layers. We have 2D images, so we want 2D layers. We also will use a few additional techniques which you can read about in the code comments.\n\n### In the cell below replace:\n#### 1. `<shape1>` with `28 `\n#### 2. `<shape2>` with `28`\n#### 3. `<shape3>` with `1`\n#### 4. `<numberOfClasses>` with `10`\n\n#### and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "suppressMessages(use_session_with_seed(1))\n###\n# REPLACE THE <shape1> WITH 28 AND <shape2> WITH 28 AND <shape3> WITH 1\n###\ninput_shape <- c(<shape1>, <shape2>, <shape3>)\n###\n\n###\n# REPLACE THE <numberOfClasses> WITH 10\n###\nnum_classes <- <numberOfClasses>\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Time to set up our model.\n\n### In the cell below replace:\n#### 1. `<convolutionalLayer>` with `layer_conv_2d `\n#### 2. `<convolutionalLayer>` with `layer_conv_2d`\n#### 3. `<poolingLayer>` with `layer_max_pooling_2d`\n#### 4. `<dropout>` with `layer_dropout`\n#### 5. `<flatten>` with `layer_flatten()`\n#### 6. `<dropout>` with `layer_dropout`\n\n#### and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# This box sets up a new convolutional neural network and prints a summary         \n\nuse_session_with_seed(1)\nset.seed(1)\n\nmodel <- keras_model_sequential() %>%\n# Here we start with the convolutional layers\n###\n# REPLACE THE TWO <convolutionalLayer>'s BELOW WITH layer_conv_2d\n###\n  <convolutionalLayer>(filters = 28, kernel_size = c(3,3), activation = 'relu',\n                input_shape = input_shape) %>% \n  <convolutionalLayer>(filters = 28, kernel_size = c(3,3), activation = 'relu') %>%\n###\n\n# Pooling layers help speed up training time and make features it detects more robust.\n# They act by downsampling the data - reducing the data size and complexity.\n###\n# REPLACE <poolingLayer> WITH layer_max_pooling_2d\n###\n  <poolingLayer>(pool_size = c(2, 2)) %>%\n###\n\n# Dropout is a technique to help prevent overfitting\n# It makes nodes 'dropout' - turning them off randomly.\n###\n# REPLACE <dropout> WITH layer_dropout\n###\n  <dropout>(rate = 0.125) %>% \n###\n\n# Next the data is flattened to a vector\n###\n# REPLACE <flatten> WITH layer_flatten()\n###\n  <flatten> %>% \n###\n\n# Dense layers perform classification - we have extracted the features with the convolutional pre-processing\n  layer_dense(units = 64, activation = 'relu') %>% \n###\n# REPLACE <dropout> WITH layer_dropout\n###\n  <dropout>(rate = 0.25) %>% \n###\n\n# Next is our output layer\n# Softmax outputs the probability for each category\n  layer_dense(units = num_classes, activation = 'softmax')\n\n\n# Let's print out the structure of our model\nsummary(model)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this cell!\n# Time to compile the model, ready for training\n\nmodel %>% compile(\n  loss = 'categorical_crossentropy',\n  optimizer = 'Adamax',\n  metrics = c('accuracy')\n)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 5\n============\n\nTime to train our model!\n\nIf it's taking a while you can lower the number of epochs. If you want to leave it running in the background and see how accurate you can get, you can increase the number of epochs.\n\n### In the cell below replace:\n#### 1. `<numberOfEpochs>` with `25`\n#### 2. `<validationPercentage>` with `0.2`\n\n#### and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this code to train the convolutional neural network and print out its accuracy\n\nhistory <- model %>% fit(\n  x_train, y_train, \n###\n# REPLACE <numberOfEpochs> WITH 25 AND <validationPercentage> WITH 0.2\n###    \n  epochs = <numberOfEpochs>, batch_size = 32, \n  validation_split = <validationPercentage>\n###    \n)\n\n# Make a graph of loss and accuracy\nplot(history)\n\n# Let's take a look at the loss and accuracy on the test set\nmodel %>% evaluate(x_test, y_test)\n\npredictions <- model %>% predict_classes(x_test)\nscores <- model %>% evaluate(\n  x_test, y_test, verbose = 0\n)\n\n# Output metrics\ncat('Test loss:', scores[[1]], '\\n')\ncat('Test accuracy:', scores[[2]], '\\n')",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 6\n============\n\nLet's take a look at an actual prediction, and what the image in the test set looks like.\n\n**Run the code below**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box to print how the  convolutional neural network predicts the label for an image\nprint(\"prediction:\")\nprint(predictions[1])\nprint(\"Test image:\")\nim <- x_test[1,,,]\nim <- t(apply(im, 2, rev)) \nimage(1:28, 1:28, im, col=gray((0:255)/255), xaxt='n')",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "How is the prediction? Does it look right?\n\nConclusion\n------\n\nCongratulations! We've built a convolutional neural network that is able to recognise handwritten digits with very high accuracy.\n\nCNN's are very complex - you're not expected to understand everything (or most things) we covered here. They take a lot of time and practice to properly understand each aspect of them.\n\nHere we used:  \n* __Feature scaling__ - reducing the range of the values. This helps improve training time.\n* __Convolutional layers__ - network layers that pre-process the data for us. These apply filters to extract features for the neural network to analyze.\n* __Pooling layers__ - part of the Convolutional layers. They apply filters to downsample the data - extracting features.\n* __Dropout__ - a regularization technique to help prevent overfitting.\n* __Dense layers__ - neural network layers which perform classification on the features extracted by the convolutional layers and downsampled by the pooling layers.\n* __Softmax__ - an activation function which outputs the probability for each category."
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "r",
      "display_name": "R",
      "language": "R"
    },
    "language_info": {
      "mimetype": "text/x-r-source",
      "name": "R",
      "pygments_lexer": "r",
      "version": "3.4.1",
      "file_extension": ".r",
      "codemirror_mode": "r"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}