2019-02-05_intro_to_python_for_finance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Notebook Author: [Trenton McKinney](https://trenton3983.github.io/)\n",
    "- Course: **[DataCamp: Introduction to Python for Finance](https://learn.datacamp.com/courses/introduction-to-python-for-finance)**\n",
    " - This [notebook](https://github.com/trenton3983/DataCamp/blob/master/2019-02-05_intro_to_python_for_finance.ipynb) was created as a reproducible reference.\n",
    " - The material is from the course\n",
    " - I completed the exercises\n",
    " - If you find the content beneficial, consider a [DataCamp Subscription](https://www.datacamp.com/pricing?period=yearly)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# My Comments about this course\n",
    "\n",
    "* Most of the course material is very basic so has not been included in the Notebook\n",
    "* Great course for those with little or not Python experience"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from numpy import NaN\n",
    "from glob import glob\n",
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.set_option('display.max_columns', 200)\n",
    "pd.set_option('display.max_rows', 300)\n",
    "pd.set_option('display.expand_frame_repr', True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Files Location\n",
    "\n",
    "* Most data files for the exercises can be found on the [course site](https://www.datacamp.com/courses/intro-to-python-for-finance)\n",
    "    * [Stocks data (I)](https://assets.datacamp.com/production/repositories/1715/datasets/2623c8037df0505d619c87a09131af9105e5883d/stock_data.csv)\n",
    "    * [Stocks data (II)](https://assets.datacamp.com/production/repositories/1715/datasets/d96bf818f1f6f52af429edcaaf9dd96d37ab7b0a/stock_data2.csv)\n",
    "    * [S&P 100 data](https://assets.datacamp.com/production/repositories/1715/datasets/0ef2a37a04b12d12368f060efd02b93cd110bd29/sector.txt)    \n",
    "* Other data files may be found in my [DataCamp repository](https://github.com/trenton3983/DataCamp/tree/master/data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data File Objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stocks1 = 'data/intro_to_python_for_finance/stock_data.csv'\n",
    "stocks2 = 'data/intro_to_python_for_finance/stock_data2.csv'\n",
    "sector_100 = 'data/intro_to_python_for_finance/sector.txt'\n",
    "exercises = 'data/intro_to_python_for_finance/exercise_data.csv'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "# Intro to Python for Finance\n",
    "\n",
    "***Course Description***\n",
    "\n",
    "The financial industry is increasingly adopting Python for general-purpose programming and quantitative analysis, ranging from understanding trading dynamics to risk management systems. This course focuses specifically on introducing Python for financial analysis. Using practical examples, you will learn the fundamentals of Python data structures such as lists and arrays and learn powerful ways to store and manipulate financial data to identify trends."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "## Welcome to Python\n",
    "\n",
    "This chapter is an introduction to basics in Python, including how to name variables and various data types in Python."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Welcome to Python for Finance!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comments and Variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using variables to evaluate stock trends\n",
    "\n",
    "$\\text{Price to earning ratio} = \\frac{\\text{Maket Price}}{\\text{Earnings per share}}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "price = 200\n",
    "earnings = 5\n",
    "\n",
    "pe_ratio = price/earnings\n",
    "pe_ratio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Types\n",
    "\n",
    "* integer\n",
    "* float\n",
    "* string\n",
    "* boolean\n",
    "\n",
    "***Check type with:***\n",
    "```python\n",
    "type(variable)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Booleans in Python\n",
    "\n",
    "Booleans are used to represent True or False statements in Python. Boolean comparisons include:\n",
    "\n",
    "| Operators |      Descriptions     |\n",
    "|:---------:|:---------------------:|\n",
    "|     >     | greater than          |\n",
    "|     >=    | greater than or equal |\n",
    "|     <     | less than             |\n",
    "|     <=    | less than or equal    |\n",
    "|     ==    | equal (compare)       |\n",
    "|     !=    | does not equal        |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lists\n",
    "\n",
    "This chapter introduces lists in Python and how they can be used to work with data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Nested Lists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### List methods and functions\n",
    "\n",
    "| Methods                                                     | Functions                        |\n",
    "|-------------------------------------------------------------|----------------------------------|\n",
    "| All methods are functions                                   | Not all functions are methods    |\n",
    "| List methods are a subset of built in functions in Python   |                                  |\n",
    "| Used on an object                                           | Requires an input of an object   |\n",
    "| prices.sort()                                               | type(prices)                     |\n",
    "\n",
    "* Functions take objects as inputs or are \"passed\" an object\n",
    "* Methods act on objects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Arrays in Python\n",
    "\n",
    "This chapter introduces packages in Python, specifically the NumPy package and how it can be efficiently used to manipulate arrays."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Arrays\n",
    "\n",
    "***Why use an array for financial analysis?***\n",
    "\n",
    "* Arrays can handle very large datasets efficiently\n",
    "    * Computationally memory efficient\n",
    "    * Faster calculations and analysis than lists\n",
    "    * Diverse functionality (many functions in Python packages)\n",
    "* All dtypes in a numpy array are the same\n",
    "* Each element of a python list keeps its dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Array operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Arrays - element-wise sum\n",
    "\n",
    "array_A = np.array([1, 2, 3])\n",
    "array_B = np.array([4, 5, 6])\n",
    "\n",
    "array_A + array_B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Lists - list concatenation\n",
    "\n",
    "list_A = [1, 2, 3]\n",
    "list_B = [4, 5, 6]\n",
    "\n",
    "list_A + list_B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2D arrays and functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using arrays for analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualization in Python\n",
    "\n",
    "In this chapter, you will be introduced to the Matplotlib package for creating line plots, scatter plots, and histograms."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualization in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Single Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(stocks1)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(df.Day, df.Price, color='red', linestyle='--')\n",
    "\n",
    "# Add x and y labels\n",
    "plt.xlabel('Days')\n",
    "plt.ylabel('Prices, $')\n",
    "\n",
    "# Add plot title\n",
    "plt.title('Company Stock Prices Over Time')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Multiple plots I"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(stocks2)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot two lines of varying colors \n",
    "plt.plot(df.day, df.company1, color='red')\n",
    "plt.plot(df.day, df.company2, color='green')\n",
    "\n",
    "# Add labels\n",
    "plt.xlabel('Days')\n",
    "plt.ylabel('Prices, $')\n",
    "plt.title('Stock Prices Over Time')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Multiple plots II"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[['company1', 'company2']].plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Scatterplots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.scatter(df.day, df.company1, color='green', s=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Histograms\n",
    "\n",
    "* Tell the distribution of the data\n",
    "* Uses in Finance\n",
    "    * Economic Indicators\n",
    "    * Stock Returns\n",
    "    * Commodity Prices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Why histograms for financial analysis?\n",
    "    \n",
    "![alt text](https://raw.githubusercontent.com/trenton3983/DataCamp/master/Images/2019-02-07_intro_to_python_for_finance/intro_to_python_for_finance_histogram.JPG?raw=true \"Histogram\")\n",
    "\n",
    "* Is you data skewed?\n",
    "* Is you data centered around the average?\n",
    "* Do you have any abnormal data points (outliers) in your data?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Histograms and matplotlib.pyplot\n",
    "\n",
    "```python\n",
    "import matplotlib.pyplot as plt\n",
    "plt.hist(x=prices, bins=3)\n",
    "plt.show()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Normalizing histogram data\n",
    "\n",
    "```python\n",
    "import matplotlib.pyplot as plt\n",
    "plt.hist(x=prices, bins=6, density=True)\n",
    "plt.show()\n",
    "```\n",
    "\n",
    "* At times it's beneficial to know the relative frequency or the percentage of observations (rather than frequency counts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Layering histograms on a plot\n",
    "\n",
    "```python\n",
    "plt.hist(x=prices, bins=6, density=True)\n",
    "plt.hist(x=prices2, bins=6, density=True)\n",
    "plt.show()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Alpha: Changing transparency of histograms\n",
    "\n",
    "```python\n",
    "plt.hist(x=prices, bins=6, density=True, alpha=0.5)\n",
    "plt.hist(x=prices2, bins=6, density=True, alpha=0.5)\n",
    "plt.show()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Adding a legend\n",
    "\n",
    "```python\n",
    "plt.hist(x=prices, bins=6, density=True, alpha=0.5, label='Prices 1')\n",
    "plt.hist(x=prices2, bins=6, density=True, alpha=0.5, label='Prices New')\n",
    "plt.legend()\n",
    "plt.show()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "### Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Is data normally distributed?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.hist(df.company2, bins=100, ec='black')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Comparing two histograms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_exercises = pd.read_csv(exercises)\n",
    "df_exercises.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### pd.DataFrame.hist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_exercises.hist(bins=100, alpha=0.4, ec='black')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### matplotlib.pyplt as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot histogram of stocks_A\n",
    "plt.hist(df_exercises.stock_A, bins=100, alpha=0.4, label='Stock A')\n",
    "\n",
    "# Plot histogram of stocks_B \n",
    "plt.hist(df_exercises.stock_B, bins=100, alpha=0.4, label='Stock B')\n",
    "\n",
    "# Add the legend\n",
    "plt.legend()\n",
    "\n",
    "# Display plot\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## S&P 100 Case Study\n",
    "\n",
    "In this chapter, you will get a chance to apply all the techniques you learned in the course on the S&P 100 data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Introducing the dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Overall Review\n",
    "\n",
    "* Python shell and scripts\n",
    "* Variables and data types\n",
    "* Lists\n",
    "* Arrays\n",
    "* Methods and functions\n",
    "* Indexing and subsetting\n",
    "* Matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### S&P 100 Companies\n",
    "\n",
    "***Standard and Poor's S&P 100:***\n",
    "\n",
    "* made up of major companies that span multiple industry groups\n",
    "* used to measure stock performance of large companies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### S&P 100 Sectors\n",
    "\n",
    "![alt text](https://raw.githubusercontent.com/trenton3983/DataCamp/master/Images/2019-02-07_intro_to_python_for_finance/intro_to_python_for_finance_s_and_p_sectors_pie.JPG?raw=true?raw=true \"Sectors\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The Data\n",
    "\n",
    "* EPS: earning per share"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(sector_100)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Price to Earnings Ratio\n",
    "\n",
    "$\\text{Price to earning ratio} = \\frac{\\text{Maket Price}}{\\text{Earnings per share}}$\n",
    "\n",
    "* The dollar amount one can expect to invest in a company in order to receive one dollar of the company's earnings\n",
    "* The ratio for valuing a company that measures its current share price relative to the per-share earnings\n",
    "* In general, higher P/E ratio idicates higher growth expectations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Case Study Objective I:\n",
    "\n",
    "***Given***\n",
    "\n",
    "* List of data describing the S&P 100: names, prices, earnigns, sectors\n",
    "\n",
    "***Objective Part I***\n",
    "\n",
    "* Explore and analyze the S&P 100 data, specifically the P/E ratios of S&P 100 companies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Methods\n",
    "\n",
    "* Step 1: examine the lists\n",
    "* Step 2: Convert lists to arrays\n",
    "* Step 3: Elementwise array operations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Project Explorations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "names = df.Name.values\n",
    "prices = df.Price.values\n",
    "earnings = df.EPS.values\n",
    "sectors = df.Sector.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Lists\n",
    "\n",
    "Stocks in the S&P 100 are selected to represent sector balance and market capitalization. To begin, let's take a look at what data we have associated with S&P companies.\n",
    "\n",
    "Four ***lists***, ***names***, ***prices***, ***earnings***, and ***sectors***, are available in your workspace.\n",
    "\n",
    "***Instructions***\n",
    "\n",
    "* Print the first four items in ***names***.\n",
    "* Print the name, price, earning, and sector associated with the last company in the lists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First four items of names\n",
    "print(names[:4])\n",
    "\n",
    "# Print information on last company\n",
    "print(names[-1])\n",
    "print(prices[-1])\n",
    "print(earnings[-1])\n",
    "print(sectors[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Arrays and NumPy\n",
    "\n",
    "NumPy is a scientific computing package in Python that helps you to work with arrays. Let's use array operations to calculate price to earning ratios of the S&P 100 stocks.\n",
    "\n",
    "The S&P 100 data is available as the lists: ***prices*** (stock prices per share) and ***earnings*** (earnings per share).\n",
    "\n",
    "***Instructions***\n",
    "\n",
    "* Import the ***numpy*** as ***np***.\n",
    "* Convert the ***prices*** and ***earnings*** lists to arrays, ***prices_array*** and ***earnings_array***, respectively.\n",
    "* Calculate the price to earnings ratio as ***pe***."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "# Convert lists to arrays\n",
    "prices_array = np.array(prices)\n",
    "earnings_array = np.array(earnings)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate P/E ratio \n",
    "pe = prices/earnings\n",
    "pe[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A closer look at sectors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Case Study Objective II:\n",
    "\n",
    "***Given***\n",
    "\n",
    "* Numpy arrays of data describing the S&P 100: names, prices, earnings, sectors\n",
    "\n",
    "***Objective Part II***\n",
    "\n",
    "* Explore and analyze sector-specific P/E ratios within companies of the S&P 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Methods\n",
    "\n",
    "* Step 1: Create a boolean filtering array\n",
    "* Step 2: Apply filtering array to subset another array\n",
    "* Step 3: Summarize P/E ratios\n",
    "    * Calculate the average and standard deviation of these sector-specific P/E ratios"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Project Explorations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Filtering arrays\n",
    "\n",
    "In this lesson, you will focus on two sectors:\n",
    "\n",
    "* Information Technology\n",
    "* Consumer Staples\n",
    "\n",
    "***numpy*** is imported as ***np*** and S&P 100 data is stored as arrays: ***names***, ***sectors***, and ***pe*** (price to earnings ratio).\n",
    "\n",
    "***Instructions 1/2***\n",
    "\n",
    "* Create a boolean array to determine which elements in ***sectors*** are ***'Information Technology'***.\n",
    "* Use the boolean array to subset ***names*** and ***pe*** in the Information Technology sector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create boolean array \n",
    "boolean_array = (sectors == 'Information Technology')\n",
    "\n",
    "# Subset sector-specific data\n",
    "it_names = names[boolean_array]\n",
    "it_pe = pe[boolean_array]\n",
    "\n",
    "# Display sector names\n",
    "print(it_names)\n",
    "print(it_pe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Instructions 2/2***\n",
    "\n",
    "* Create a boolean array to determine which elements in ***sectors*** are ***'Consumer Staples'***.\n",
    "* Use the boolean array to subset ***names*** and ***pe*** in the Consumer Staples sector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create boolean array \n",
    "boolean_array = (sectors == 'Consumer Staples')\n",
    "\n",
    "# Subset sector-specific data\n",
    "cs_names = names[boolean_array]\n",
    "cs_pe = pe[boolean_array]\n",
    "\n",
    "# Display sector names\n",
    "print(cs_names)\n",
    "print(cs_pe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Summarizing sector data\n",
    "\n",
    "In this exercise, you will calculate the mean and standard deviation of P/E ratios for Information Technology and Consumer Staples sectors. ***numpy*** is imported as ***np*** and the ***it_pe*** and ***cs_pe*** arrays from the previous exercise are available in your workspace.\n",
    "\n",
    "***Instructions 1/2***\n",
    "\n",
    "Calculate the mean and standard deviation of the P/E ratios (***it_pe***) for the Industrial Technology sector.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate mean and standard deviation\n",
    "it_pe_mean = np.mean(it_pe)\n",
    "it_pe_std = np.std(it_pe)\n",
    "\n",
    "print(it_pe_mean)\n",
    "print(it_pe_std)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Instructions 2/2***\n",
    "\n",
    "* Calculate the mean and standard deviation of the P/E ratios (***cs_pe***) for the Consumer Staples sector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate mean and standard deviation\n",
    "cs_pe_mean = np.mean(cs_pe)\n",
    "cs_pe_std = np.std(cs_pe)\n",
    "\n",
    "print(cs_pe_mean)\n",
    "print(cs_pe_std)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Plot P/E ratios\n",
    "Let's take a closer look at the P/E ratios using a scatter plot for each company in these two sectors.\n",
    "\n",
    "The arrays ***it_pe*** and ***cs_pe*** from the previous exercise are available in your workspace. Also, each company name has been assigned a numeric ID contained in the arrays ***it_id*** and ***cs_id***.\n",
    "\n",
    "***Instructions***\n",
    "\n",
    "* Draw a scatter plot of ***it_pe*** ratios with red markers and ***'IT'*** label.\n",
    "* On the same plot, add the ***cs_pe*** ratios with green markers and ***'CS'*** label.\n",
    "* Add a legend to this plot.\n",
    "* Display the plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "it_id = np.arange(0, 15)\n",
    "cs_id = np.arange(0, 12)\n",
    "\n",
    "# Make a scatterplot\n",
    "plt.scatter(it_id, it_pe, color='red', label='IT')\n",
    "plt.scatter(cs_id, cs_pe, color='green', label='CS')\n",
    "\n",
    "# Add legend\n",
    "plt.legend()\n",
    "\n",
    "# Add labels\n",
    "plt.xlabel('Company ID')\n",
    "plt.ylabel('P/E Ratio')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Notice that there is one company in the IT sector with an unusually high P/E ratio***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizating trends"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Case Study Objective III:\n",
    "\n",
    "***Objective Part III***\n",
    "\n",
    "* Investigate the outlier from the scatter plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Methods\n",
    "\n",
    "* Step 1: Make a histogram of the P/E ratios\n",
    "* Step 2:\n",
    "    * Identify the outlier P/E ratio\n",
    "    * Create a boolean array filter to subset this company\n",
    "    * Filter out this company information from the provided datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Project Explorations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Histogram of P/E ratios\n",
    "\n",
    "To visualize and understand the distribution of the P/E ratios in the IT sector, you can use a histogram.\n",
    "\n",
    "The array ***it_pe*** from the previous exercise is available in your workspace.\n",
    "\n",
    "***Instructions***\n",
    "\n",
    "* Selectively import the ***pyplot*** module of ***matplotlib*** as ***plt***.\n",
    "* Plot a histogram of ***it_pe*** with 8 bins.\n",
    "* Add the x-label as ***'P/E ratio'*** and y-label as ***'Frequency'***.\n",
    "* Display the plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot histogram \n",
    "plt.hist(it_pe, bins=8, ec='black')\n",
    "\n",
    "# Add x-label\n",
    "plt.xlabel('P/E ratio')\n",
    "\n",
    "# Add y-label\n",
    "plt.ylabel('Frequency')\n",
    "\n",
    "# Show plot\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Identify the outlier\n",
    "\n",
    "* Histograms can help you to identify outliers or abnormal data points. Which P/E ratio in this histogram is an example of an outlier?\n",
    "\n",
    "***A stock with P/E ratio > 50.***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Name the outlier\n",
    "\n",
    "You've identified that a company in the Industrial Technology sector has a P/E ratio of greater than 50. Let's identify this company.\n",
    "\n",
    "***numpy*** is imported as ***np***, and arrays ***it_pe*** (P/E ratios of Industrial Technology companies) and ***it_names*** (names of Industrial Technology companies) are available in your workspace.\n",
    "\n",
    "Instructions\n",
    "\n",
    "* Identify the P/E ratio greater than 50 and assign it to ***outlier_price***.\n",
    "* Identify the company with P/E ratio greater than 50 and assign it to ***outlier_name***."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Identify P/E ratio within it_pe that is > 50\n",
    "outlier_price = it_pe[it_pe > 50]\n",
    "\n",
    "# Identify the company with PE ratio > 50\n",
    "outlier_name = it_names[it_pe == outlier_price]\n",
    "\n",
    "# Display results\n",
    "print(f'In 2017 {outlier_name[0]} had an abnormally high P/E ratio of {round(outlier_price[0], 2)}.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Certificate\n",
    "\n",
    "![](https://raw.githubusercontent.com/trenton3983/DataCamp/master/Images/2019-02-07_intro_to_python_for_finance/2019-02-07_intro_to_python_for_finance_certificate.JPG)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py312",
   "language": "python",
   "name": "py312"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}