naive_bayes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Movie Reviews Sentiment Analysis with Scikit-Learn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### In this lab tutorial, we will build a sentiment classifier to classify movie reviews as either positive or negative."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load movie_reviews corpus data through sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn\n",
    "from sklearn.datasets import load_files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The variable moviedir is the location of the movie_reviews folder.\n",
    "# You might need to modify the path according to where you store the data.\n",
    "moviedir= 'E:/NaiveBayes/movie_reviews'\n",
    "\n",
    "# Loading all files.\n",
    "# Set shuffle to be true means to rearrange the order of the files randomly.\n",
    "movie = load_files(moviedir, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Total number of movie reviews\n",
    "len(movie.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Target names (\"classes\") are automatically generated from subfolder names.\n",
    "movie.target_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First 500 characters of the first file\n",
    "movie.data[0][:500]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print out the filename of the first file.\n",
    "movie.filenames[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First file is a negative review and is mapped to 0 index 'neg' in target_names.\n",
    "movie.target[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Number of negtive reviews\n",
    "sum(movie.target==0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A detour: try out CountVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import CountVectorizer\n",
    "from sklearn.feature_extraction.text import CountVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize CountVectorizer\n",
    "vectorizer = CountVectorizer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# corpus contains 3 documents\n",
    "corpus = ['the cat sat',\n",
    "          'the cat sat in the hat',\n",
    "          'the cat with the hat']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# It will perform both tokenization and word occurrence counting on the corpus and therefore transform each of the documents into a vector.\n",
    "X = vectorizer.fit_transform(corpus)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Returns a list of feature names, in our case, these are the unique words appear in the corpus.\n",
    "vectorizer.get_feature_names()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Produces a 3x6 matrix, 3 represent the number of the document, 6 represents the size of the feature,\n",
    "# each row vector corresponds to a vectorized document,\n",
    "# each value in the row vector represents the number of occurrence of the feature at the corresponding position in the feature names list.\n",
    "X.toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### we completed the feature extraction process for our toy corpus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "·What if we get new ones?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A new document\n",
    "newdocs = [\"the dog with the ball\"]\n",
    "\n",
    "# This time, no fitting needed: directly apply transform method on the new document to convert it into count-vectorized form\n",
    "# Unseen words ('dog', 'ball') are ignored\n",
    "newdocs_counts = vectorizer.transform(newdocs)\n",
    "newdocs_counts.toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Back to real data: movie reviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split data into training and test sets\n",
    "from sklearn.model_selection import train_test_split\n",
    "docs_train, docs_test, y_train, y_test = train_test_split(movie.data, movie.target, \n",
    "                                                          test_size = 0.20, random_state = 12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Write code to implement following:\n",
    "- initialize CountVectorizer, ignore words with frequency less than 2 and use top 3000 words only. \n",
    "- fit and tranform using training data\n",
    "- using the fitted vectorizer, tranform the test data\n",
    "- check out <a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html\">CountVectorizer documentation</a> for any help."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### START CODE HERE ###\n",
    "# initialize CountVectorizer\n",
    "movie_vectorizer = \n",
    "\n",
    "# fit and tranform using training data \n",
    "docs_train_counts = \n",
    "### END CODE HERE ###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# test you code \n",
    "docs_train_counts.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Expected output\n",
    "```\n",
    "(1600, 3000)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### START CODE HERE ###\n",
    "# Using the fitted vectorizer, tranform the test data\n",
    "docs_test_counts = \n",
    "### END CODE HERE ###"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training and testing a Naive Bayes classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now ready to build a classifier. \n",
    "# We will use Multinominal Naive Bayes as our model\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Write code to implement following:\n",
    "- initialize and train a Multinominal Naive Bayes classifier\n",
    "- predict the test data results, using <a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html\">accuracy_score</a> find accuracy\n",
    "- check out <a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html#sklearn.naive_bayes.MultinomialNB\">Multinomial Naive Bayes documentation</a> for any help."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = MultinomialNB()\n",
    "### START CODE HERE ###\n",
    "\n",
    "### END CODE HERE ###"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Expected accuracy\n",
    "```\n",
    "0.7775\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Trying the classifier on fake movie reviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# very short and fake movie reviews\n",
    "reviews_new = ['This movie was excellent', \n",
    "               'Absolute joy ride', \n",
    "               'Steven Seagal was terrible', \n",
    "               'Two thumbs up', \n",
    "               'I fell asleep halfway through', \n",
    "               'Steven Seagal was amazing. His performance was Oscar-worthy.']\n",
    "\n",
    "reviews_new_counts = movie_vectorizer.transform(reviews_new)         # turn text into count vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# have classifier make a prediction\n",
    "pred = clf.predict(reviews_new_counts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# print out results\n",
    "for review, category in zip(reviews_new, pred):\n",
    "    print(f'{review} => {movie.target_names[category]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}