APOGEE_data_tutorial.ver3comp.ipynb

{
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  },
  "name": ""
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "APOGEE Data:  \"Zero to Hero\""
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### What this is\n",
      "\n",
      "  An exectuable, interactive Ipython Notebook designed to quickly get\n",
      "  novices up and running with APOGEE data. By the end of this tutorial,\n",
      "  you will be able to :\n",
      "  - Download the APOGEE Summary file\n",
      "  - Read the APOGEE Summary file into memory\n",
      "  - Define \"clean\" samples by making cuts on tageting and reduction\n",
      "    flags\n",
      "  - Make exciting plots of APOGEE data\n",
      "\n",
      "  Along the way, you will also learn more about the APOGEE data model\n",
      "  and bitmasks.\n",
      "\n",
      "\n",
      "### What this isn't\n",
      "\n",
      "  A comprehensive guide to the APOGEE survey and data products. Please\n",
      "  see the official APOGEE documenation at\n",
      "  [http://www.sdss.org/dr12/irspec/] for a more thorough understanding of\n",
      "  the APOGEE database. Below, we provide links to this documentation\n",
      "  where appropriate. [http://www.sdss.org/dr12/irspec/catalogs/] is the\n",
      "  best place to start further educating yourself about APOGEE\n",
      "  parameters. [Official papers?]\n",
      "\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Requirements\n",
      "==============\n",
      "\n",
      "  This tutorial is written in Python 2.7. Python 3+ *should* work with\n",
      "  minimal effort from the user. Additional python modules required:\n",
      "  - numpy\n",
      "  - pyfits\n",
      "  - matplotlib\n",
      "\n",
      "  This notebook will likely still be valuable to you, but less so, if\n",
      "  you are using a language other than python. There are official data\n",
      "  exploration examples using the CAS database, IDL, or python at\n",
      "  [http://www.sdss.org/dr12/irspec/catalogs/#ExamplesofselectingAPOGEEdatafromcatalogs]\n",
      "  .\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "4 The Summary File\n",
      "==================\n",
      "\n",
      "  The APOGEE *allStar* summary file is one of two APOGEE [parameter catalogs](http://www.sdss.org/dr12/irspec/catalogs/) available as FITS tables. The entire summary parameter catalog is less than 1 GB in size, making it very portable and easy to access.\n",
      "  The allStar file \n",
      "  >provides\n",
      "  >information for each individual star observed in APOGEE-1, which\n",
      "  >includes mean barycentric radial velocity, the standard deviation of\n",
      "  >the mean velocity, ASPCAP parameters and abundances as derived from\n",
      "  >the combined spectra, and a compilation of ancillary targeting data.\n",
      "  \n",
      "  >-[APOGEE official documentation](http://www.sdss.org/dr12/irspec/catalogs/)\n",
      "\n",
      "  The latest, official Data Release from SDSS is DR12. The APOGEE summary\n",
      "  file for this release is named 'allStar-v603.fits' (~683 MB). The rest of this tutorial will focus on interacting with this catalog.\n",
      "\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "5 Getting the Data\n",
      "==================\n",
      "\n",
      "  **Option 1**: Directly download the allStar file [HERE](http://data.sdss3.org/sas/dr12/apogee/spectro/redux/r5/allStar-v603.fits). Once the catalog is downloaded, move the file  into the path assigned to the `data_directory` variable in Section 6.2. The catalog is large (~683 MB) so it might be best to download the catalog first and then finish the tutorial.\n",
      "  \n",
      "  **Option 2**: Automatic Download. As you run the code, it will search for the *allStar-v603.fits* file in the path assigned to `data_directory`. If it is not found, the catalog will be downloaded."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##6 Loading the *allStar* Summary File\n",
      "\n",
      "### 6.1 Python Preamble \n",
      "You need to press `Shift+Enter` to execute the code cells below. The preamble just contains some useful modules, functions, and plot formatting snippets. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from __future__ import print_function\n",
      "import numpy as np\n",
      "import os\n",
      "import sys\n",
      "import matplotlib\n",
      "from matplotlib import pyplot as plt\n",
      "%matplotlib inline\n",
      "\n",
      "###Plot formatting commands\n",
      "plt.rc('figure', figsize=(10.0, 7.1)) #bigger figures\n",
      "plt.rc('xtick', labelsize=14)  #bigger labels\n",
      "plt.rc('ytick', labelsize=14)\n",
      "###End formatting commands\n",
      "\n",
      "\n",
      "  \n",
      "### Functions for automatic downloading of catalog\n",
      "import urllib  \n",
      "### Checking for python 2 or 3\n",
      "if sys.version_info > (3,0): # If python3\n",
      "    geturl_func = urllib.request.urlretrieve\n",
      "else: #python2\n",
      "    geturl_func = urllib.urlretrieve\n",
      "\n",
      "#Progress bar function for automatic download (You can ignore this)\n",
      "try:\n",
      "    from IPython.html.widgets import FloatProgress\n",
      "    from IPython.display import display\n",
      "    from time import sleep\n",
      "    progbar = FloatProgress(min=0, max=100, description='Download Progress...')\n",
      "    html_widget = True\n",
      "    def report(count, blockSize, totalSize):   #function to report download percentage\n",
      "        percent = count*blockSize*100/float(totalSize)\n",
      "        progbar.value = percent\n",
      "\n",
      "except ImportError:\n",
      "    html_widget = False\n",
      "    from IPython.display import clear_output   #function to clear cell output\n",
      "    def report(count, blockSize, totalSize):   #function to report download percentage\n",
      "        percent = count*blockSize*100/float(totalSize)\n",
      "        if (count % 100) ==0: #only update every 40 blocks \n",
      "            print('{0:.2f}'.format(percent), end=' ')\n",
      "\n",
      "#clear_output()\n",
      "#print('{}% complete\\r'.format(percent))\n",
      "#sys.stdout.flush()\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### 6.2 APOGEE Data Path\n",
      "\n",
      "   <span style=\"color:red\">**MAKE SURE TO DEFINE YOUR DATA PATH in the next cell**</span>\n",
      "\n",
      "  The `data_directory` variable is the diretory where your summary\n",
      "  file is located OR where you want the summary file downloaded. Make sure this\n",
      "  path exists on your local machine."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data_directory = \"/Users/jquark/obs_data/apogee\"  #### CHANGE THIS PATH\n",
      "\n",
      "if not os.path.exists(data_directory):\n",
      "    print(\"\\nMake sure the data_directory path exists!\\n\")\n",
      "else:\n",
      "    print(\"Your APOGEE data directory is: {}\\nWoohoo! Let's get started!\".format(data_directory))\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### 6.3 Reading the allStar File\n",
      "\n",
      "The *allStar* file is a FITS table, a common astronomical data format. To read it off disk and into memory, we will use the popular [astropy](http://www.astropy.org/) package which includes the latest version of `pyfits` FITS reader. The code below will first try to use `astropy` but will automatically fall back to `pyfits` if `astropy` is not installed. At least one of these packages must be installed to continue. Note that  [pyfits](http://www.stsci.edu/institute/software_hardware/pyfits) is the precursor to the `astropy.io.fits` package."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "try:\n",
      "    import astropy.io.fits as pyfits\n",
      "except ImportError:\n",
      "    import pyfits\n",
      "\n",
      "allStar_file = 'allStar-v603.fits'\n",
      "allStar_localpath = os.path.join(data_directory,allStar_file)\n",
      "\n",
      "allStar_file_link = \"http://data.sdss3.org/sas/dr12/apogee/spectro/redux/r5/allStar-v603.fits\"\n",
      "\n",
      "#Download allStar file if it does not exist (This will take a while!)\n",
      "if not os.path.exists(allStar_localpath):\n",
      "    print ('% Complete: ', end=' ' )\n",
      "    if html_widget:\n",
      "        display(progbar)\n",
      "    geturl_func(allStar_file_link, filename=allStar_localpath, reporthook=report)\n",
      "\n",
      "try:\n",
      "    allStar_fits = pyfits.open(allStar_localpath)\n",
      "    allStar = allStar_fits[1].data #Takes data is HDU 1 and assigns it to allStar\n",
      "except ValueError:\n",
      "    print('FITS reader error: The catalog file is likely not complete.')\n",
      "    print('Use the direct download link in Section 5 and copy file into the data_directory path.')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### 6.4 APOGEE *allStar* catalog as python variable\n",
      "\n",
      "\n",
      "  The allStar variable is now a numpy\n",
      "  record array, a wrapper around [numpy.ndarray](http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.html). The total number of stars in the catalog is equal to the number of rows in the table. Check the number of rows with \n",
      "\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "allStar.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "and the available data fields (This will print out every data field name in the `allStar` file)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "allStar.dtype.fields"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "7 *allStar* Data Model\n",
      "=================================\n",
      "\n",
      "A lot of your interaction with the APOGEE catalog will be through pulling out the information that interests you from the `allStar` file. It's a good idea to familiarize yourself with the catalog's contents.\n",
      "\n",
      "The APOGEE team provides a full description of the *allStar* file via the [data model](http://data.sdss3.org/datamodel/files/APOGEE_REDUX/APRED_VERS/APSTAR_VERS/ASPCAP_VERS/RESULTS_VERS/allStar.html). Some details, like the size of the catalog on disk, will likely be different than v603 (the version you downloaded).\n",
      " \n",
      " A non-exhaustive list of important fields (some of which we will use below):\n",
      "\n",
      "| Field          | dtype    | units     | description                                                                                                                             |\n",
      "| ---            | ----     | ----      | ----                                                                                                                                    |\n",
      "| APSTAR_ID      | char[37] |           | Unique apStar identifier: apogee.[ns].[sc].APSTAR_VERS.LOC.STAR, where [ns] is for APOGEE North/South, [sc] is for survey/commissioning |\n",
      "| ASPCAP_ID      | char[36] |           | Unique ASPCAP identifier: apogee.[ns].[sc].RESULTS_VERS.LOC.STAR                                                                        |\n",
      "| APOGEE_ID      | char[18] |           | TMASS-STYLE object name                                                                                                                 |\n",
      "| J              | float32  |           | 2MASS J mag                                                                                                                             |\n",
      "| H              | float32  |           | 2MASS H mag                                                                                                                             |\n",
      "| K              | float32  |           | 2MASS Ks                                                                                                                                |\n",
      "| GLON           | float64  | degrees   | Galactic                                                                                                                                |\n",
      "| GLAT           | float64  | degrees   | Galactic                                                                                                                                |\n",
      "| AK_TARG        | float32  |           | K-band extinction adopted for                                                                                                           |\n",
      "| APOGEE_TARGET1 | int32    |           | first APOGEE target flag, see bitmask definitions                                                                                       |\n",
      "| APOGEE_TARGET2 | int32    |           | second APOGEE target flag, see bitmask                                                                                                  |\n",
      "| COMMISS        | int16    |           | =1 for commissioning data (before July 2011), else 0                                                                                    |\n",
      "| SNR            | float32  |           | median S/N per pixel in combined frame (at apStar sampling                                                                              |\n",
      "| STARFLAG       | int32    |           | Flag for star condition taken from bitwise OR of individual visits, see bitmask                                                         |\n",
      "| VHELIO_AVG     | float32  | km/s      | average radial velocity, weighted by S/N, using RVs determined from cross-correlation of individual spectra with combined               |\n",
      "| VSCATTER       | float32  | km/s      | scatter of individual visit RVs around                                                                                                  |\n",
      "| TEFF           | float32  | K         | Teff from ASPCAP analysis of combined spectrum (from PARAM)                                                                             |\n",
      "| LOGG           | float32  | log (cgs) | log g from ASPCAP analysis of combined spectrum (from PARAM)                                                                            |\n",
      "| METALS         | float32  | dex       | [Z/H] from ASPCAP analysis of combined spectrum (from PARAM)                                                                            |\n",
      "| ALPHAFE        | float32  | dex       | [alpha/Fe] from ASPCAP analysis of combined spectrum (from PARAM)                                                                       |\n",
      "| ASPCAPFLAG     | int32    |           | Flag for  ASPCAP analysis, see bitmask                                                                                                  |\n",
      "| PARAMFLAG      | int32[7] |           | Individual parameter flag for ASPCAP analysis, see bitmask                                                                              |\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "7.1 Sky map of the APOGEE Catalog\n",
      "-----------------------------------\n",
      "\n",
      "  Now that you have loaded the catalog, we can make our first plot! Let's make a sky map of the APOGEE catalog.\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "fig =plt.figure()\n",
      "ax = plt.subplot(111) ## Ask Gail how she does the box around the projection, projection='mollweide') # set up projection\n",
      "# For pretty plotting, make longitude negative relative to Galactic center.\u2423\n",
      "l_corr_inds = allStar['GLON']>180 #indices for GLON correction\n",
      "l_corr = allStar['GLON'].copy() # just to be safe,\n",
      "l_corr[l_corr_inds] -=360\n",
      "ax.scatter(l_corr, allStar['GLAT'], edgecolor='none', alpha=0.3)\n",
      "ax.set_xlim(180,-180)\n",
      "ax.set_xlabel('Galactic Longitude')\n",
      "ax.set_ylabel('Galactic Latitude')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "8 Clean samples, Targeting / Reduction flags\n",
      "============================================\n",
      "\n",
      "  Before doing any science with APOGEE, you will likely want a \"clean\"\n",
      "  sample of stars. APOGEE has many special types of targets (calibration\n",
      "  clusters, commissioning data, etc.; you can read more about the types\n",
      "  of targets HERE). Below, we assume that you are interested in the\n",
      "  \"main\" survey targets (defined below).\n",
      "\n",
      "\n",
      "### 8.1 A Quick Aside: Bitmasks\n",
      "\n",
      "\n",
      "  If you are unfamiliar with bitmasks, SDSS has published a great\n",
      "  primer: [http://www.sdss.org/dr12/algorithms/bitmasks]. In short,\n",
      "  bitmasks provide an easy way to ask many True/False questions about an\n",
      "  object without the need for independent variables. Let's look at a\n",
      "  quick example.\n",
      "  \n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "allStar['APOGEE_TARGET1'][2]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Above is the APOGEE_TARGET1 bitmask for a single star. A\n",
      "  full description of the available bits in APOGEE_TARGET1 is available\n",
      "  at [http://www.sdss.org/dr12/algorithms/bitmasks/#APOGEE_TARGET1].\n",
      "\n",
      "  Each bit in the bitmask above contains information about the target. Note that all valid APOGEE targets have the 31st bit set."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This object is part of a 'short' cohort (see Zasowski+ for\n",
      "  description), so the 11th bit is also set. You can check the value of a particular bit using the bitwise `and` operator."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "allStar['APOGEE_TARGET1'][2] & 2**11"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The result will be non-zero if the bit is set; otherwise it will be zero."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "allStar['APOGEE_TARGET1'][2] & 2**15 #not set"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Below, we will be using a series of cuts on the APOGEE bitmasks to define our\n",
      "  \"clean\" sample. \n",
      "\n",
      "  Links to other APOGEE bitmasks that we will use can be found at:\n",
      "  [http://www.sdss.org/dr12/algorithms/bitmasks/#APOGEE_TARGET2]\n",
      "  [http://www.sdss.org/dr12/algorithms/bitmasks/#APOGEE_ASPCAPFLAG]\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Defining the \"Main\" Sample\n",
      "\n",
      "The `allStar` catalog contains all of the targets observed by the APOGEE survey, including engineering and calibration targets. Most scientific analysis will only use the 'main survey' targets as the parent sample of stars; the [APOGEE_EXTRATARG](http://www.sdss.org/dr12/algorithms/bitmasks/#APOGEE_EXTRATARG) bitmask (new in DR12) was specifically made for this purpose. The table below describes this bitmask.\n",
      "\n",
      "|Bit Name\t|Binary Digit|\tDescription|\n",
      "|--\n",
      "|NOT_MAIN\t|0|\tNot main survey target|\n",
      "|COMMISSIONING\t|1|\tCommissioning data|\n",
      "|TELLURIC\t|2|\tTelluric target|\n",
      "|APO1M\t|3|\t1m observation|\n",
      "|DUPLICATE\t|4|\tDuplicate observation|\n",
      "\n",
      "Full descriptions of these terms can be found in the online [APOGEE targeting documentation](http://www.sdss.org/dr12/irspec/targets/) and the APOGEE target selection paper ([Zasowski et al. 2013](http://arXiv.org/abs/1308.0351)). If none of the bits in `EXTRATARG` are set, the target is part of the 'main survey' sample."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "mainIndx = allStar['EXTRATARG'] == 0  #If the bitmask is 0, no bits are set"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "mainIndx # show what mainIndx actually is"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`mainIndx` is a [boolean index array](http://docs.scipy.org/doc/numpy/user/basics.indexing.html#boolean-or-mask-index-arrays). It has the same shape as the condition from which it was made (in this case, `allStar['EXTRATARG']` which has the same length as the other fields in the `allStar` table). The elements of `mainIndx` are true when `EXTRATARG`==0; else false. Boolean indices are a common way to select a subset of an data structure and we will encounter them in the rest of the tutorial."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.sum(mainIndx) #True=1 and False=0; sum gives us the number of Main survey targets in DR12"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's define a new variable to point towards all of the 'Main' targets in the APOGEE catalog."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "mainStar = allStar[mainIndx] #mainStar is now the main survey subsample"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "9 Plot: The RV distribution of the inner galaxy\n",
      "==========================================================\n",
      "\n",
      "Now that our main sample is defined, let's make some plots! First up, the line of sight velocity distribution for stars in sightlines towards the Galactic center.\n",
      "\n",
      "Step one is to define the spatial region of interest and then grab the subset of targets meeting the conditions. Here, let's pick all stars with\n",
      "    $-15^{\\circ} < l < 15^{\\circ}$ and close to the plane of the disk: $-5^{\\circ} < b < 5^{\\circ}$. Remember, the galactic longitude reported by APOGEE in the `GLON` field is in the range [$0^{\\circ}$, $360^{\\circ}$].\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "glon_gt_345 = mainStar['GLON']>345.0  #l > 345 (or > -15)\n",
      "glon_lt_15 = mainStar['GLON']<15.0  #l < 15\n",
      "glat_in_plane = np.abs(mainStar['GLAT'])< 5 # |b| < 5"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Boolean arrays can be combined in many ways (below, we use [`numpy` logical operators]( functionshttp://docs.scipy.org/doc/numpy/reference/routines.logic.html#logical-operations)). In this case, we want stars that satisfy either of the longitude constraints and the latitude constraint."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "inner_galaxy_stars = np.logical_and(np.logical_or(glon_gt_345, glon_lt_15), glat_in_plane)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "inner_galaxy_stars  #Shows that inner_galaxy_mask is just another boolean mask"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "inner_galaxy_stars.shape #with the same number of elements as mainStar"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      " We can check for the number of stars that our criteria by asking\n",
      "  for the sum of the mask (remember: True=1 and False=0)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print('There are {0} main survey stars in our inner Galaxy region'.format(np.sum(inner_galaxy_stars)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Boolean index arrays do not need to be assigned to variables to be combined. A shortcut to making the `inner_galaxy_stars` mask would be"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "inner_galaxy_stars2 = np.logical_and(np.logical_or(mainStar['GLON']>345.0, mainStar['GLON']<15.0), np.abs(mainStar['GLAT'])< 5)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "(inner_galaxy_stars2 == inner_galaxy_stars).all() \n",
      "#check if all the elements of one array equal another"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now let's look at the distribution of radial velocities for main\n",
      "  survey targets towards the inner galaxy. Below, we use the `matplotlib` histogram function [`hist`](http://matplotlib.org/api/pyplot_api.html)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.figure() #make figure\n",
      "plt.hist(mainStar[inner_galaxy_stars]['VHELIO_AVG'], bins = np.arange(-500,500,20))\n",
      "#mainStar[inner_galaxy_stars] returns only mainStar entries with a True value in inner_galaxy_stars\n",
      "plt.xlabel('VHELIO_AVG [km/s]', fontsize=15)\n",
      "plt.ylabel('Number of Stars', fontsize=15)\n",
      "plt.title('APOGEE Radial Velocities for Inner Galaxy sightlines')\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "\n",
      "  Beautiful! We find a sharp peak at VHELIO_AVG~ 0 km/s because many\n",
      "  stellar orbits are perpendicular to our line of sight when we look\n",
      "  towards the Galactic center. The positive skew is likely the result of an asymmetric distribution of Galactic longitude for our `inner_galaxy_stars`. The 4th quadrant ($270^{\\circ} < l < 360^{\\circ}$) of the Galaxy is less accessible from the Northern hemisphere. Our coverage of this area of the Galaxy will greatly improve in APOGEE-2. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.figure() #make figure\n",
      "plt.hist(mainStar[inner_galaxy_stars]['GLON'], bins = np.arange(0,361,10))\n",
      "#mainStar[inner_galaxy_stars] returns only mainStar entries with a True value in inner_galaxy_stars\n",
      "plt.xlabel('Galactic Longitude [GLON, degrees]', fontsize=15)\n",
      "plt.ylabel('Number of Stars', fontsize=15)\n",
      "plt.xlim(0,361)\n",
      "plt.title('Galactic Longitude distribution for Inner Galaxy sightline stars')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Sure enough, the longitude distribution is heavily skewed towards stars at $0^{\\circ} < l < 15^{\\circ}$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "10 Galactic Plane Metallicity Maps\n",
      "=========================\n",
      " \n",
      "\n",
      "  This next plot is loosely based on the kinematics maps from Ness et al., in prep. (2015). Here we will map the metallicity of the Galaxy as a function of sky position, narrowing our analysis to $l<180^{\\circ}$ sightlines in the Galactic plane.\n",
      "\n",
      "We need to divide stars into 2-D position bins (using `GLON` and `GLAT`) and then compute the mean metallicity of all the stars in each bin. Conveniently, `matplotlib` provides a convenience function for this type of exploration: `hexbin` returns\n",
      "  \n",
      "> a hexagonal binning plot of x versus y, where x, y are 1-D sequenc\n",
      "es of the same length, N. If C is None (the default), this is a histogram of\n",
      "the number of occurences of the observations at (x[i],y[i]).  \n",
      "> \\- [`matplotlib` API](http://matplotlib.org/api/pyplot_api.html)\n",
      " \n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "#define our zone of spatial interest and assign subset to variable\n",
      "galactic_plane_stars = np.logical_and(mainStar['GLON']<180.0,np.abs(mainStar['GLAT'])< 15)\n",
      "planeStar = mainStar[galactic_plane_stars] # planeStar now points to all the mainStar entries that are galactic_plane_stars."
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print('{0} stars are \"galactic_plane_stars\"'.format(planeStar.shape[0])) "
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now we can use `hexbin` to plot the mean metallicity of stars as a function of position in the sky."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.figure()\n",
      "plt.hexbin(planeStar['GLON'], planeStar['GLAT'], C=planeStar['FE_H'], gridsize=30, mincnt=2)\n",
      "#Adding the 'C' keyword makes hexbin return the mean of the array for stars in each bin\n",
      "#See http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.hexbin for options\n",
      "plt.xlabel('Galactic Longitude', fontsize=16)\n",
      "plt.ylabel('Galactic Latitude', fontsize=16)\n",
      "cb = plt.colorbar(label=r'$\\overline{[\\mathrm{Fe/H}]}$')\n",
      "#Label Colorbar\n",
      "plt.title('APOGEE Metallicity across the Sky')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The hexagons above are colored by the mean metallicity of all the stars that have the labeled Galactic longitude and latitude. But, **WHAT HAPPENED?** The range of the colorbar suggests that many patches of the sky have a strongly negative iron abundance. How could the mean metallicity of some regions be ~-5000? This is out first plot using the chemical abundance data in the `allStar` catalog. Before we publish our miraculously discovery, we should make sure that the reduction and ASPCAP pipeline does not report any errors for these stars. "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "After this tutorial, you should read [Using APOGEE Stellar Abundances](http://www.sdss.org/dr12/irspec/abundances/) before proceeding with your own science. Heed the warning at the top of page\n",
      ">ASPCAP results should not be used blindly. They are provided with the general caveats that affect abundances determined spectroscopically under the standard assumption. Users are encouraged to thoroughly examine both the detectability and quality of the spectral features employed in the derivation of the desired element abundances. \n",
      "\n",
      "To ensure our chemical abundance results are robust, we should only use stars that:\n",
      "\n",
      " - are not commissioning targets. The commissioning data contain odd spectra that may not produce reliable stellar parameters (we have already done this by selecting only main survey targets).\n",
      " - had no issues reported with their parameter determination. The 'STAR_BAD' bit (bit 23) of the [ASPCAPFLAG bitmask](http://www.sdss.org/dr12/algorithms/bitmasks/#APOGEE_ASPCAPFLAG) is a catch-all bit designed to be set if ASPCAP returns an error during processing.\n",
      " - had no issues reported during the specific individual element determination. Specifically, we need to check if the 'GRIDEDGE_BAD' (bit 0), 'CALRANGE_BAD' (bit 1), and 'CALRANGE_WARN' (bit 9) are set in the [individual element bitmask](http://www.sdss.org/dr12/algorithms/bitmasks/#APOGEE_PARAMFLAG). See [ASPCAP Element Bitmasks](http://www.sdss.org/dr12/irspec/abundances/#ASPCAPElementBitmasks) for a description of these bits or other important caveats."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "#We could build up the abundance masks using planeStar\n",
      "#But let's check the abundance pipeline for all main survey targets via mainStar\n",
      "\n",
      "stargood = mainStar['ASPCAPFLAG'] & 2**23 == 0 #True if STAR_BAD is NOT set\n",
      "gridedge_good = mainStar['FE_H_FLAG'] & 2**0 == 0 #True if GRIDEDGE_BAD is NOT set\n",
      "calrange_good = mainStar['FE_H_FLAG'] & 2**1 == 0 #True if CALRANGE_BAD is NOT set \n",
      "calrange_nowarn = mainStar['FE_H_FLAG'] & 2**9 == 0 #True if CALRANGE_WARN is NOT set \n",
      "\n",
      "#Want a combined index that returns stars that meet all of the above criteria\n",
      "fehflag_good = np.logical_and.reduce((gridedge_good,calrange_good,calrange_nowarn))\n",
      "\n",
      "#reduce lets us apply the logical_and to all arrays at once\n",
      "#See http://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.reduce.html for further reading\n",
      "\n",
      "feh_robust = np.logical_and(stargood, fehflag_good)\n",
      "#feh_robust is True only when all none of the abundance error bits are set"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "By applying these essential quality control cuts on the data, we will lose stars from our sample by defintion. Let's see what the numbers are..."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print('The allStar file contains {0} entries'.format(allStar.shape[0]))\n",
      "print('{0} of which are main survey targets'.format(mainStar.shape[0]))\n",
      "print('{0} of {1} main stars do not have STAR_BAD set'.format(np.sum(stargood),mainStar.shape[0]))\n",
      "print('{0} of {1} main stars report no individual FE abundance errors'.format(np.sum(fehflag_good),mainStar.shape[0]))\n",
      "print('{0} of {1} main stars report no parameter errors and no individual [Fe/H] abundance errors'.format(np.sum(feh_robust),mainStar.shape[0]))\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "#Now define sample is all inner_galaxy stars that ALSO meet the pipeline criteria\n",
      "goodPlaneStar = mainStar[np.logical_and(galactic_plane_stars,feh_robust)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print('{0} stars are \"galactic_plane_stars\" with robust [Fe/H] determination'.format(goodPlaneStar.shape[0]))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's remake our plot with the `goodPlaneStar` sample."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.figure()\n",
      "plt.hexbin(goodPlaneStar['GLON'], goodPlaneStar['GLAT'], C=goodPlaneStar['FE_H'], \n",
      "           gridsize=30, mincnt=2)\n",
      "#hexbin plots mean of 'C' array\n",
      "plt.xlabel('Galactic Longitude', fontsize=16)\n",
      "plt.ylabel('Galactic Latitude', fontsize=16)\n",
      "cb = plt.colorbar(label=r'$\\overline{[\\mathrm{Fe/H}]}$')\n",
      "plt.title('APOGEE Metallicity across the Sky')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Awesome! We now see the metallicity of the central region of the Galaxy as a function of position. The metallicity gradient from low longitude, metal-rich areas to high longitude, metal-poor areas is clearly visible."
     ]
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {
      "collapsed": true
     },
     "source": [
      "11 Plot: Metallicity vs. RV for Quadrant 2 Stars"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Once you get a feel for this workflow, you can quickly make plots using tens of thousands of stars. To make a plot showing the radial velocity and metallicity of all stars that are in quadrant 2:\n",
      "\n",
      "- Define the spatial region of interest\n",
      "- Ensure the metallicity determination is robust as we did for the Galactic plane sample above\n",
      "- Make the plot using stars that meet the above criteria"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "#Quadrant 2 is 90 < GLON < 180\n",
      "quadrant2 = np.logical_and(mainStar['GLON']>90, mainStar['GLON']<180)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can reuse `feh_robust` from Section 10 to quickly focus on the sample we want."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "# Using the feh_robust boolean array from Section 10\n",
      "goodQuadrant2 = mainStar[np.logical_and(quadrant2,feh_robust)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`goodQuadrant2` now refers to stars with good FeH determination and in quadrant 2."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print('{0} stars are \"quadrant2\" with robust [Fe/H] determination'.format(goodQuadrant2.shape[0]))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now let's make a scatter plot of [Fe/H] vs. line-of-sight velocity for stars in `goodQuadrant2`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.figure()\n",
      "plt.scatter(goodQuadrant2['VHELIO_AVG'], goodQuadrant2['FE_H'], edgecolor='none', alpha=0.2)\n",
      "#see http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.scatter for scatter options\n",
      "plt.xlim(-150,90) # limiting the range of the x-axis\n",
      "plt.ylim(-0.8,0.45) # limiting the range of the y-axis\n",
      "plt.xlabel('LOS Velocity [km/s]', fontsize=14)\n",
      "plt.ylabel('[Fe/H]', fontsize=14)\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# 12 The Red Clump Catalog\n",
      "\n",
      "The DR12 APOGEE Red Clump Catalog [Bovy et al. (2014)](http://adsabs.harvard.edu/abs/2014ApJ...790..127B) contains 19,937 red clump stars (vs. 10,341 in the original version of the catalog) with precise distance measurements. The catalog paper's abstract reads\n",
      "\n",
      ">The Sloan Digital Sky Survey III's Apache Point Observatory Galactic Evolution Experiment (APOGEE) is a high-resolution near-infrared spectroscopic survey covering all of the major components of the Galaxy, including the dust-obscured regions of the inner Milky Way disk and bulge. Here we present a sample of 10,341 likely red-clump stars (RC) from the first two years of APOGEE operations, selected based on their position in color-metallicity-surface-gravity-effective-temperature space using a new method calibrated using stellar-evolution models and high-quality asteroseismology data. The narrowness of the RC locus in color-metallicity-luminosity space allows us to assign distances to the stars with an accuracy of 5 to 10%. The sample extends to typical distances of about 3 kpc from the Sun, with some stars out to 8 kpc, and spans a volume of approximately 100 kpc^3 over 5 kpc <~ R <~ 14 kpc, |Z| <~ 2 kpc, and -15 deg <~ Galactocentric azimuth <~ 30 deg. The APOGEE red-clump (APOGEE-RC) catalog contains photometry from 2MASS, reddening estimates, distances, line-of-sight velocities, stellar parameters and elemental abundances determined from the high-resolution APOGEE spectra, and matches to major proper motion catalogs. We determine the survey selection function for this data set and discuss how the RC selection samples the underlying stellar populations. We use this sample to limit any azimuthal variations in the median metallicity within the ~45 degree-wide azimuthal region covered by the current sample to be <= 0.02 dex, which is more than an order of magnitude smaller than the radial metallicity gradient. This result constrains coherent non-axisymmetric flows within a few kpc from the Sun.  \n",
      "`-` [Bovy et al. (2014)](http://adsabs.harvard.edu/abs/2014ApJ...790..127BD)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## 12.1 Getting the Data\n",
      "  **Option 1**: Directly download the RC catalog file [HERE](http://data.sdss3.org/sas/dr12/apogee/vac/apogee-rc/cat/apogee-rc-DR12.fits). Once the catalog is downloaded, move the file  into the path assigned to the `data_directory` variable in Section 6.2.\n",
      "  \n",
      "  **Option 2**: Automatic Download. As you run the code, it will search for the *apogee-rc-DR12.fits* file in the path assigned to `data_directory`. If it is not found, the catalog will be downloaded."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "12.2 Reading the RC Catalog"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "#Make sure the urllib module has been imported and geturl_func has been defined!\n",
      "\n",
      "rc_file = 'apogee-rc-DR12.fits'\n",
      "RCstar_localpath = os.path.join(data_directory,rc_file)\n",
      "\n",
      "RCstar_file_link = 'http://data.sdss3.org/sas/dr12/apogee/vac/apogee-rc/cat/apogee-rc-DR12.fits'\n",
      "#Download allStar file if it does not exist\n",
      "if not os.path.exists(RCstar_localpath):\n",
      "    geturl_func(RCstar_file_link, filename=RCstar_localpath, reporthook=report)\n",
      "\n",
      "allRC_fits = pyfits.open(RCstar_localpath)\n",
      "allRC = allRC_fits[1].data #Takes data is HDU 1 and assigns it to allStar"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## 12.3 Red Clump Data Model\n",
      "\n",
      "All available data for the Red Clump sample is described by the data model at http://data.sdss3.org/datamodel/files/APOGEE_RC/cat/apogee-rc-DR12.html.\n",
      "Some interesting additional data fields available for RC stars:\n",
      "\n",
      "| Field     | dtype | Units     |Description                                                |\n",
      "|-----      |-----  | -----     |------------                                               |\n",
      "|RC_DIST\t|float64|\tkpc \t|Distance                                                   |\n",
      "|RC_DM\t    |float64|\t    \t|Distance modulus                                           |\n",
      "|RC_GALR\t|float64|\tkpc \t|Galactocentric distance assuming R0 = 8 kpc, Z0 = 25 pc    |\n",
      "|RC_GALPHI\t|float64|\tradians\t|Galactocentric azimuth assuming R0 = 8 kpc, Z0 = 25 pc     |\n",
      "|RC_GALZ\t|float64|\tkpc \t|Distance from the mid-plane assuming R0 = 8 kpc, Z0 = 25 pc|\n",
      "\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## 12.4 Plot: [$\\alpha$/Fe] vs. [Fe/H] for annulus of stars\n",
      "We can use the precise distances of RC stars to probe particular regions of the galaxy. For example, let's examine the [$\\alpha$/Fe] vs. [Fe/H] plane for stars with $6 < R_{\\mathrm{GC}} < 8$ kpc and $|z_{\\mathrm{GC}}| < 1$ kpc."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "#define the spatial region of interest\n",
      "annulus_r6_r8 = np.logical_and(allRC['RC_GALR']>6, allRC['RC_GALR']<8)\n",
      "close_to_plane = np.abs(allRC['RC_GALZ'])<1.0\n",
      "\n",
      "inplane_annulus = np.logical_and(annulus_r6_r8, close_to_plane)#annulusRCPlaneStars\n",
      "\n",
      "annulusRC = allRC[inplane_annulus]\n",
      "\n",
      "plt.hexbin(annulusRC['METALS'], annulusRC['ALPHAFE'], gridsize=30, mincnt=1)\n",
      "\n",
      "#'METALS' is technically [Z/H], but for our purposes, it is akin to [Fe/H]\n",
      "plt.xlabel('[Fe/H]', fontsize=16)\n",
      "plt.ylabel(r'[$\\alpha$/Fe]', fontsize=16)\n",
      "cb = plt.colorbar(label='Number of RC stars per bin')\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "13 Introduction to Individual Spectra"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In additional to stellar parameters and abundances, APOGEE releases all the spectra it collects in various stages of the reduction pipele. In general, there are three types of spectra you might be interested in.\n",
      "\n",
      "| Type | Description\n",
      "|-----:|:-----|\n",
      "|Combined spectra |from all visits to a star are available in apStar files.|\n",
      "|Individual visit spectra |of each visit to each star are available in apVisit files.|\n",
      "|Pseudo-continuum normalized spectra| that are used in the derivation of stellar parameters are available in aspcapStar files.|\n",
      "\n",
      "You will find the above table at the [Using APOGEE Spectra](http://www.sdss.org/dr12/irspec/spectra/)  documentation. If you plan on working with individual spectra in the future, make sure to read [Using APOGEE Spectra](http://www.sdss.org/dr12/irspec/spectra/) first to find out how the spectra are produced, important caveats, data quality flags, and directory structure. Refer to Holtzman et al. (2015) for further details.  \n",
      "\n",
      "Below, we will plot both the raw data and best-fitting ASPCAP model to the pseudo-continuum normalized spectra of a single star. The method below is easily generalizable to any APOGEE spectrum of this type. "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## 13.1 Spectra Directory Structure and Filename Conventions\n",
      "\n",
      "APOGEE spectra are organized differently depending on their type. ASPCAP spectra (psuedo-continuum normalized spectra) are stored according to their field. The field associated with an ASPCAP spectrum is encoded in the `LOCATION_ID` field of the `allStar` file. \n",
      "\n",
      "The ASPCAP spectrum directory structure is\n",
      ">APOGEE_REDUX/APRED_VERS/APSTAR_VERS/ASPCAP_VERS/RESULTS_VERS/LOCATION_ID  \n",
      "\n",
      "where from\n",
      "[APOGEE Reduction Version Numbers](http://www.sdss.org/dr12/irspec/spectro_data/#ReductionVersionNumbers)\n",
      "\n",
      "|||\n",
      "|-\n",
      "|APRED_VERS| denotes the version of the basic spectra reduction used to generate the visit spectra.|\n",
      "|APSTAR_VERS| denotes the version of the code used to combine individual visit spectra into higher signal-to-noise, resampled, combined spectra.|\n",
      "|ASPCAP_VERS |denotes the version of the code used to determine the stellar atmospheric parameters for each combined spectrum.|\n",
      "|RESULTS_VERS| denotes the version of the code used to (1) compile the final results, and (2) apply additional empirical corrections and set data quality flags based on evaluation of results for calibration objects. A given RESULTS_VERS implies versions of all of the preceding steps, and thus uniquely identifies the full set of software used.|\n",
      "\n",
      "The DR12 reduction versions:\n",
      "\n",
      "|APRED_VERS | APSTAR_VERS |ASPCAP_VERS | RESULTS_VERS|\n",
      "|---::--|-----:---|----:----|----:----|\n",
      "|r5| stars|  l25_6d |v603|\n",
      "\n",
      "Finally, the ASPCAP specta are stored as FITS files with the following [SAS URL format](www.sdss.org/dr12/irspec/spectro_data/index.html#IntermediateDataProducts): \n",
      "> aspcapStar-APRED_VERS-RESULTS_VERS-STARNAME.fits  \n",
      "\n",
      "STARNAME is the `APOGEE_ID` field found in the `allStar` catalog. Let's see how this works for a \"random\" (the quotes will be explained later) main survey star. Below, we will grab the aspcapStar (ASPCAP spectrum file) for the 17301st star in the `mainStar` sample defined in Section 8.\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "html_prefix = 'http://data.sdss3.org/sas/dr12/'\n",
      "APOGEE_REDUX = 'apogee/spectro/redux/'"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "os.path.join(html_prefix, APOGEE_REDUX)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "indx = 17301  # index to grab the star of interest\n",
      "#The variable names below match the nomenclature introduced in the preceeding cells.\n",
      "LOCATION_ID = mainStar['LOCATION_ID'][indx] \n",
      "APOGEE_ID = mainStar['APOGEE_ID'][indx]\n",
      "html_prefix = 'http://data.sdss3.org/sas/dr12/'\n",
      "APOGEE_REDUX = 'apogee/spectro/redux/'\n",
      "APRED_VERS = 'r5'\n",
      "APSTAR_VERS = 'stars'\n",
      "ASPCAP_VERS = 'l25_6d'\n",
      "RESULTS_VERS = 'v603'\n",
      "\n",
      "##From above\n",
      "url_directory = os.path.join(html_prefix,APOGEE_REDUX,APRED_VERS,APSTAR_VERS,ASPCAP_VERS,str(LOCATION_ID))\n",
      "spectrum_filename = 'aspcapStar-{0}-{1}-{2}.fits'.format(APRED_VERS, RESULTS_VERS, APOGEE_ID)\n",
      "url_location = os.path.join(url_directory,spectrum_filename)\n",
      "\n",
      "#See what we did\n",
      "print('{0} star of mainStar sample:'.format(indx))\n",
      "print('APOGEE_ID: {0}\\t LOCATION_ID: {1}'.format(APOGEE_ID, LOCATION_ID))\n",
      "print('ASPCAP spectrum located in directory:\\n{0}'.format(url_directory))\n",
      "print('Filename: {0}'.format(spectrum_filename))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## 13.2 Download Spectrum\n",
      "\n",
      "Formatting `url_location` properly is the hardest part of obtaining an APOGEE spectrum. Now, just download it!"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "#First check to see if the file already exists, no sense in downloading it again.\n",
      "\n",
      "local_spectrum_path = os.path.join(data_directory,spectrum_filename)\n",
      "if not os.path.exists(local_spectrum_path):\n",
      "    geturl_func(url_location, filename=local_spectrum_path)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## 13.3 Anatomy of ASPCAP Spectrum File\n",
      "\n",
      "The ASPCAP spectrum [data model](http://data.sdss3.org/datamodel/files/APOGEE_REDUX/APRED_VERS/APSTAR_VERS/ASPCAP_VERS/RESULTS_VERS/LOCATION_ID/aspcapStar.html)  describes the structure of the ASPCAP spectum FITS file."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "apogeeSpectrum = pyfits.open(local_spectrum_path) # Open and read spectrum FITS table\n",
      "\n",
      "flux = apogeeSpectrum[1].data # HDU1: Pseudo-continumm normalized flux\n",
      "model = apogeeSpectrum[3].data # HDU3: Best fit model spectrum\n",
      "error = apogeeSpectrum[2].data # HDU1: Pseudo-continumm normalized flux\n",
      "nlambda = apogeeSpectrum[1].data.shape[0] # The number of "
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The arrays `flux`, `model`, and `error` are 1-D arrays representing the normalized flux of each component at `nlambda` different wavelengths ($\\lambda$). Note that $\\lambda$ are stored as $\\log{\\lambda}$ per the [data model](http://data.sdss3.org/datamodel/files/APOGEE_REDUX/APRED_VERS/APSTAR_VERS/ASPCAP_VERS/RESULTS_VERS/LOCATION_ID/aspcapStar.html). Now let's determine the wavelength range for this star's spectrum."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "wavelength_start = apogeeSpectrum[1].header['CRVAL1'] #access starting wavelength [log lambda]\n",
      "disperion = apogeeSpectrum[1].header['CDELT1'] #Dispersion [log lambda]\n",
      "wavelength_end = disperion*(nlambda) + wavelength_start\n",
      "loglambda_range = np.arange(wavelength_start,wavelength_end,disperion)\n",
      "lambda_range = 10**loglambda_range  #make the scale linear \n",
      "SNR = apogeeSpectrum[0].header['SNR'] #note the SNR for plotting"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## 13.4 Plotting Individual Spectra\n",
      "\n",
      "We have computed the wavelength range and read the data and model flux. Below we divide the plot into 3 subplots to highlight the blue, green, and red chips (see [Wavelength Coverage]( http://www.sdss.org/dr12/irspec/spectra/#WavelengthCoverageandDetectorGaps) for details)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "fig,axs = plt.subplots(nrows=3, ncols=1)\n",
      "axs[0].plot(lambda_range, flux, color = 'k', label='Data')\n",
      "axs[0].plot(lambda_range, model, color = 'red',alpha=0.5, label='Model')\n",
      "axs[0].set_xlim(15160,15795)\n",
      "axs[0].legend(loc='lower right')\n",
      "axs[1].plot(lambda_range, flux, color = 'k', label='Data')\n",
      "axs[1].plot(lambda_range, model, color = 'red',alpha=0.5, label='Model')\n",
      "axs[1].set_ylabel('Normalized Flux', fontsize=25)\n",
      "axs[1].set_xlim(15875, 16420)\n",
      "axs[2].plot(lambda_range, flux, color = 'k', label='Data')\n",
      "axs[2].plot(lambda_range, model, color = 'red',alpha=0.5, label='Model')\n",
      "axs[2].set_xlim(16495, 16940)\n",
      "axs[2].set_xlabel('Wavelength [Angstroms]')\n",
      "axs[0].set_title(\"ID = \"+str(spectrum_filename)+\", SNR = \"+str(np.int(SNR)), fontsize = 12) \n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "You've completed the APOGEE \"Zero to Hero\" data interaction tutorial."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Sources/Thanks\n",
      "================\n",
      "\n",
      "  The material herein is a result of the whole APOGEE team. Special thanks to Jon Holtzman, Melissa Ness, Jo Bovy, Carlos Allende Prieto, and many others. Written by Jonathan Bird.\n",
      "\n",
      "\n"
     ]
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Further Reading"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## The APOGEE python module\n",
      "\n",
      "Jo Bovy has written the excellent and well-documented [APOGEE](https://github.com/jobovy/apogee). It has convenient shortcuts for many of the operations in this tutorial and contains powerful tools for analyzing APOGEE spectra. This is *the* module for you if you're a pythonista (or an aspiring one) and want to work with APOGEE data. https://github.com/jobovy/apogee"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}