WML-with XGBoost for cancer detection.ipynb

{
    "nbformat_minor": 2, 
    "cells": [
        {
            "source": "## Watson Machine Learning with XGBoost ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "This notebook contains steps and code to get data from the IBM Watson Studio Community, create a predictive model, and start scoring new data. This notebook introduces commands for getting data and for basic data cleaning and exploration, model training, model persistance to Watson Machine Learning repository, model deployment, and scoring.\n\nSome familiarity with Python is helpful. This notebook uses Python 3.5 on Spark 2.1, XGBoost and scikit-learn 0.17.\n\nYou will use a publicly available data set, the Breast Cancer Wisconsin (Diagnostic) Data Set to train XGBoost Model, which is short for \u201cExtreme Gradient Boosting\u201d, to classify breast cancer tumors (as benign or malignant) from 569 diagnostic images based on measurements such as radius, texture, perimeter and area.\n\n## Learning goals\n\nThe learning goals of this notebook are:\n\n-  Load a CSV file into Pandas DataFrame.\n-  Explore data.\n-  Prepare data for training and evaluation.\n-  Create an XGBoost machine learning model.\n-  Train and evaluate a model.\n-  Use cross-validation to optimize model's hyperparameters.\n-  Persist a model in Watson Machine Learning repository.\n-  Deploy a model for online scoring using Wastson Machine Learning API.\n-  Score sample scoring data using the Watson Machine Learning API.\n-  Explore and visualize prediction result using the plotly package.\n\n\n## Contents\n\nThis notebook contains the following parts:\n\n1.\t[Setup](#setup)\n2.\t[Load and explore data](#load)\n3.\t[Create XGBoost model](#model)\n4.\t[Persist model](#persistence)\n5.\t[Deploy and score in a Cloud](#scoring)\n6.\t[Summary and next steps](#summary)", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"setup\"></a>\n## 1. Setup\n\nBefore you use the sample code in this notebook, you must perform the following setup tasks:\n\n-  Create a [Watson Machine Learning Service](https://console.ng.bluemix.net/catalog/services/ibm-watson-machine-learning/) instance (a free plan is offered). \n-  Upload **Breast Cancer Wisconsin (Diagnostic) Data Set** data as a data asset in Watson Studio.\n-  Make sure that you are using a Spark 2.0 kernel.\n\n### Getting Breast Cancer Wisconsin (Diagnostic) Data Set  \n\nThe Breast Cancer Wisconsin (Diagnostic) Data Set is a freely available data set on the Watson Studio home page.\n\n1.  Go to the [Breast Cancer Wisconsin (Diagnostic) Data Set](https://dataplatform.ibm.com/exchange/public/entry/view/c173693bf48aeb22e41bbe2b41d79c1f) data card on the Watson Studio **Community** page and open the card by double-clicking it.\n2.  Click the link icon.\n4.  Select the link, copy it by pressing Ctrl+C, and then, click **Close**.\n5.  In the following cell, replace the **link_to_data** variable value with the link.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"load\"></a>\n## 2. Load and explore data", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In this section you will load the data as a Pandas DataFrame and perform a basic exploration.\n\nLoad the data to the Pandas DataFrame by using `wget` to upload the data to gpfs and then the `read` method. ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "**Example**: First, you need to install required packages. You can do it by running the following code. Run it only one time.<BR><BR>\n`!pip install wget --user` <BR>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 1, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "BreastCancerWisconsinDataSet (3).csv\n"
                }
            ], 
            "source": "import wget\n\nlink_to_data = 'https://apsportal.ibm.com/exchange-api/v1/entries/c173693bf48aeb22e41bbe2b41d79c1f/data?accessKey=941eec501eadcdceb5abd25cf7c029d5'\nWisconsinDataSet = wget.download(link_to_data)\n\nprint(WisconsinDataSet)"
        }, 
        {
            "source": "The csv file **BreastCancerWisconsinDataSet.csv** is availble on gpfs now. Load the file to Pandas DataFrame using code below.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 2, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 2, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>id</th>\n      <th>diagnosis</th>\n      <th>radius_mean</th>\n      <th>texture_mean</th>\n      <th>perimeter_mean</th>\n      <th>area_mean</th>\n      <th>smoothness_mean</th>\n      <th>compactness_mean</th>\n      <th>concavity_mean</th>\n      <th>concave points_mean</th>\n      <th>...</th>\n      <th>radius_worst</th>\n      <th>texture_worst</th>\n      <th>perimeter_worst</th>\n      <th>area_worst</th>\n      <th>smoothness_worst</th>\n      <th>compactness_worst</th>\n      <th>concavity_worst</th>\n      <th>concave points_worst</th>\n      <th>symmetry_worst</th>\n      <th>fractal_dimension_worst</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>842302</td>\n      <td>M</td>\n      <td>17.99</td>\n      <td>10.38</td>\n      <td>122.80</td>\n      <td>1001.0</td>\n      <td>0.11840</td>\n      <td>0.27760</td>\n      <td>0.3001</td>\n      <td>0.14710</td>\n      <td>...</td>\n      <td>25.38</td>\n      <td>17.33</td>\n      <td>184.60</td>\n      <td>2019.0</td>\n      <td>0.1622</td>\n      <td>0.6656</td>\n      <td>0.7119</td>\n      <td>0.2654</td>\n      <td>0.4601</td>\n      <td>0.11890</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>842517</td>\n      <td>M</td>\n      <td>20.57</td>\n      <td>17.77</td>\n      <td>132.90</td>\n      <td>1326.0</td>\n      <td>0.08474</td>\n      <td>0.07864</td>\n      <td>0.0869</td>\n      <td>0.07017</td>\n      <td>...</td>\n      <td>24.99</td>\n      <td>23.41</td>\n      <td>158.80</td>\n      <td>1956.0</td>\n      <td>0.1238</td>\n      <td>0.1866</td>\n      <td>0.2416</td>\n      <td>0.1860</td>\n      <td>0.2750</td>\n      <td>0.08902</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>84300903</td>\n      <td>M</td>\n      <td>19.69</td>\n      <td>21.25</td>\n      <td>130.00</td>\n      <td>1203.0</td>\n      <td>0.10960</td>\n      <td>0.15990</td>\n      <td>0.1974</td>\n      <td>0.12790</td>\n      <td>...</td>\n      <td>23.57</td>\n      <td>25.53</td>\n      <td>152.50</td>\n      <td>1709.0</td>\n      <td>0.1444</td>\n      <td>0.4245</td>\n      <td>0.4504</td>\n      <td>0.2430</td>\n      <td>0.3613</td>\n      <td>0.08758</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>84348301</td>\n      <td>M</td>\n      <td>11.42</td>\n      <td>20.38</td>\n      <td>77.58</td>\n      <td>386.1</td>\n      <td>0.14250</td>\n      <td>0.28390</td>\n      <td>0.2414</td>\n      <td>0.10520</td>\n      <td>...</td>\n      <td>14.91</td>\n      <td>26.50</td>\n      <td>98.87</td>\n      <td>567.7</td>\n      <td>0.2098</td>\n      <td>0.8663</td>\n      <td>0.6869</td>\n      <td>0.2575</td>\n      <td>0.6638</td>\n      <td>0.17300</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>84358402</td>\n      <td>M</td>\n      <td>20.29</td>\n      <td>14.34</td>\n      <td>135.10</td>\n      <td>1297.0</td>\n      <td>0.10030</td>\n      <td>0.13280</td>\n      <td>0.1980</td>\n      <td>0.10430</td>\n      <td>...</td>\n      <td>22.54</td>\n      <td>16.67</td>\n      <td>152.20</td>\n      <td>1575.0</td>\n      <td>0.1374</td>\n      <td>0.2050</td>\n      <td>0.4000</td>\n      <td>0.1625</td>\n      <td>0.2364</td>\n      <td>0.07678</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows \u00d7 32 columns</p>\n</div>", 
                        "text/plain": "         id diagnosis  radius_mean  texture_mean  perimeter_mean  area_mean  \\\n0    842302         M        17.99         10.38          122.80     1001.0   \n1    842517         M        20.57         17.77          132.90     1326.0   \n2  84300903         M        19.69         21.25          130.00     1203.0   \n3  84348301         M        11.42         20.38           77.58      386.1   \n4  84358402         M        20.29         14.34          135.10     1297.0   \n\n   smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n0          0.11840           0.27760          0.3001              0.14710   \n1          0.08474           0.07864          0.0869              0.07017   \n2          0.10960           0.15990          0.1974              0.12790   \n3          0.14250           0.28390          0.2414              0.10520   \n4          0.10030           0.13280          0.1980              0.10430   \n\n            ...             radius_worst  texture_worst  perimeter_worst  \\\n0           ...                    25.38          17.33           184.60   \n1           ...                    24.99          23.41           158.80   \n2           ...                    23.57          25.53           152.50   \n3           ...                    14.91          26.50            98.87   \n4           ...                    22.54          16.67           152.20   \n\n   area_worst  smoothness_worst  compactness_worst  concavity_worst  \\\n0      2019.0            0.1622             0.6656           0.7119   \n1      1956.0            0.1238             0.1866           0.2416   \n2      1709.0            0.1444             0.4245           0.4504   \n3       567.7            0.2098             0.8663           0.6869   \n4      1575.0            0.1374             0.2050           0.4000   \n\n   concave points_worst  symmetry_worst  fractal_dimension_worst  \n0                0.2654          0.4601                  0.11890  \n1                0.1860          0.2750                  0.08902  \n2                0.2430          0.3613                  0.08758  \n3                0.2575          0.6638                  0.17300  \n4                0.1625          0.2364                  0.07678  \n\n[5 rows x 32 columns]"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "import pandas as pd\n\nWisconsinDataSet = \"BreastCancerWisconsinDataSet.csv\"\ndf_data = pd.read_csv(WisconsinDataSet)\ndf_data.head()"
        }, 
        {
            "source": "We can see how many records and features we have in our data set.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 3, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 3, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "(569, 32)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df_data.shape"
        }, 
        {
            "source": "We can generate descriptive statistics for our dataset, excluding NaN values.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 4, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 4, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>id</th>\n      <th>radius_mean</th>\n      <th>texture_mean</th>\n      <th>perimeter_mean</th>\n      <th>area_mean</th>\n      <th>smoothness_mean</th>\n      <th>compactness_mean</th>\n      <th>concavity_mean</th>\n      <th>concave points_mean</th>\n      <th>symmetry_mean</th>\n      <th>...</th>\n      <th>radius_worst</th>\n      <th>texture_worst</th>\n      <th>perimeter_worst</th>\n      <th>area_worst</th>\n      <th>smoothness_worst</th>\n      <th>compactness_worst</th>\n      <th>concavity_worst</th>\n      <th>concave points_worst</th>\n      <th>symmetry_worst</th>\n      <th>fractal_dimension_worst</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>count</th>\n      <td>5.690000e+02</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>...</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n      <td>569.000000</td>\n    </tr>\n    <tr>\n      <th>mean</th>\n      <td>3.037183e+07</td>\n      <td>14.127292</td>\n      <td>19.289649</td>\n      <td>91.969033</td>\n      <td>654.889104</td>\n      <td>0.096360</td>\n      <td>0.104341</td>\n      <td>0.088799</td>\n      <td>0.048919</td>\n      <td>0.181162</td>\n      <td>...</td>\n      <td>16.269190</td>\n      <td>25.677223</td>\n      <td>107.261213</td>\n      <td>880.583128</td>\n      <td>0.132369</td>\n      <td>0.254265</td>\n      <td>0.272188</td>\n      <td>0.114606</td>\n      <td>0.290076</td>\n      <td>0.083946</td>\n    </tr>\n    <tr>\n      <th>std</th>\n      <td>1.250206e+08</td>\n      <td>3.524049</td>\n      <td>4.301036</td>\n      <td>24.298981</td>\n      <td>351.914129</td>\n      <td>0.014064</td>\n      <td>0.052813</td>\n      <td>0.079720</td>\n      <td>0.038803</td>\n      <td>0.027414</td>\n      <td>...</td>\n      <td>4.833242</td>\n      <td>6.146258</td>\n      <td>33.602542</td>\n      <td>569.356993</td>\n      <td>0.022832</td>\n      <td>0.157336</td>\n      <td>0.208624</td>\n      <td>0.065732</td>\n      <td>0.061867</td>\n      <td>0.018061</td>\n    </tr>\n    <tr>\n      <th>min</th>\n      <td>8.670000e+03</td>\n      <td>6.981000</td>\n      <td>9.710000</td>\n      <td>43.790000</td>\n      <td>143.500000</td>\n      <td>0.052630</td>\n      <td>0.019380</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.106000</td>\n      <td>...</td>\n      <td>7.930000</td>\n      <td>12.020000</td>\n      <td>50.410000</td>\n      <td>185.200000</td>\n      <td>0.071170</td>\n      <td>0.027290</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.156500</td>\n      <td>0.055040</td>\n    </tr>\n    <tr>\n      <th>25%</th>\n      <td>8.692180e+05</td>\n      <td>11.700000</td>\n      <td>16.170000</td>\n      <td>75.170000</td>\n      <td>420.300000</td>\n      <td>0.086370</td>\n      <td>0.064920</td>\n      <td>0.029560</td>\n      <td>0.020310</td>\n      <td>0.161900</td>\n      <td>...</td>\n      <td>13.010000</td>\n      <td>21.080000</td>\n      <td>84.110000</td>\n      <td>515.300000</td>\n      <td>0.116600</td>\n      <td>0.147200</td>\n      <td>0.114500</td>\n      <td>0.064930</td>\n      <td>0.250400</td>\n      <td>0.071460</td>\n    </tr>\n    <tr>\n      <th>50%</th>\n      <td>9.060240e+05</td>\n      <td>13.370000</td>\n      <td>18.840000</td>\n      <td>86.240000</td>\n      <td>551.100000</td>\n      <td>0.095870</td>\n      <td>0.092630</td>\n      <td>0.061540</td>\n      <td>0.033500</td>\n      <td>0.179200</td>\n      <td>...</td>\n      <td>14.970000</td>\n      <td>25.410000</td>\n      <td>97.660000</td>\n      <td>686.500000</td>\n      <td>0.131300</td>\n      <td>0.211900</td>\n      <td>0.226700</td>\n      <td>0.099930</td>\n      <td>0.282200</td>\n      <td>0.080040</td>\n    </tr>\n    <tr>\n      <th>75%</th>\n      <td>8.813129e+06</td>\n      <td>15.780000</td>\n      <td>21.800000</td>\n      <td>104.100000</td>\n      <td>782.700000</td>\n      <td>0.105300</td>\n      <td>0.130400</td>\n      <td>0.130700</td>\n      <td>0.074000</td>\n      <td>0.195700</td>\n      <td>...</td>\n      <td>18.790000</td>\n      <td>29.720000</td>\n      <td>125.400000</td>\n      <td>1084.000000</td>\n      <td>0.146000</td>\n      <td>0.339100</td>\n      <td>0.382900</td>\n      <td>0.161400</td>\n      <td>0.317900</td>\n      <td>0.092080</td>\n    </tr>\n    <tr>\n      <th>max</th>\n      <td>9.113205e+08</td>\n      <td>28.110000</td>\n      <td>39.280000</td>\n      <td>188.500000</td>\n      <td>2501.000000</td>\n      <td>0.163400</td>\n      <td>0.345400</td>\n      <td>0.426800</td>\n      <td>0.201200</td>\n      <td>0.304000</td>\n      <td>...</td>\n      <td>36.040000</td>\n      <td>49.540000</td>\n      <td>251.200000</td>\n      <td>4254.000000</td>\n      <td>0.222600</td>\n      <td>1.058000</td>\n      <td>1.252000</td>\n      <td>0.291000</td>\n      <td>0.663800</td>\n      <td>0.207500</td>\n    </tr>\n  </tbody>\n</table>\n<p>8 rows \u00d7 31 columns</p>\n</div>", 
                        "text/plain": "                 id  radius_mean  texture_mean  perimeter_mean    area_mean  \\\ncount  5.690000e+02   569.000000    569.000000      569.000000   569.000000   \nmean   3.037183e+07    14.127292     19.289649       91.969033   654.889104   \nstd    1.250206e+08     3.524049      4.301036       24.298981   351.914129   \nmin    8.670000e+03     6.981000      9.710000       43.790000   143.500000   \n25%    8.692180e+05    11.700000     16.170000       75.170000   420.300000   \n50%    9.060240e+05    13.370000     18.840000       86.240000   551.100000   \n75%    8.813129e+06    15.780000     21.800000      104.100000   782.700000   \nmax    9.113205e+08    28.110000     39.280000      188.500000  2501.000000   \n\n       smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\ncount       569.000000        569.000000      569.000000           569.000000   \nmean          0.096360          0.104341        0.088799             0.048919   \nstd           0.014064          0.052813        0.079720             0.038803   \nmin           0.052630          0.019380        0.000000             0.000000   \n25%           0.086370          0.064920        0.029560             0.020310   \n50%           0.095870          0.092630        0.061540             0.033500   \n75%           0.105300          0.130400        0.130700             0.074000   \nmax           0.163400          0.345400        0.426800             0.201200   \n\n       symmetry_mean           ...             radius_worst  texture_worst  \\\ncount     569.000000           ...               569.000000     569.000000   \nmean        0.181162           ...                16.269190      25.677223   \nstd         0.027414           ...                 4.833242       6.146258   \nmin         0.106000           ...                 7.930000      12.020000   \n25%         0.161900           ...                13.010000      21.080000   \n50%         0.179200           ...                14.970000      25.410000   \n75%         0.195700           ...                18.790000      29.720000   \nmax         0.304000           ...                36.040000      49.540000   \n\n       perimeter_worst   area_worst  smoothness_worst  compactness_worst  \\\ncount       569.000000   569.000000        569.000000         569.000000   \nmean        107.261213   880.583128          0.132369           0.254265   \nstd          33.602542   569.356993          0.022832           0.157336   \nmin          50.410000   185.200000          0.071170           0.027290   \n25%          84.110000   515.300000          0.116600           0.147200   \n50%          97.660000   686.500000          0.131300           0.211900   \n75%         125.400000  1084.000000          0.146000           0.339100   \nmax         251.200000  4254.000000          0.222600           1.058000   \n\n       concavity_worst  concave points_worst  symmetry_worst  \\\ncount       569.000000            569.000000      569.000000   \nmean          0.272188              0.114606        0.290076   \nstd           0.208624              0.065732        0.061867   \nmin           0.000000              0.000000        0.156500   \n25%           0.114500              0.064930        0.250400   \n50%           0.226700              0.099930        0.282200   \n75%           0.382900              0.161400        0.317900   \nmax           1.252000              0.291000        0.663800   \n\n       fractal_dimension_worst  \ncount               569.000000  \nmean                  0.083946  \nstd                   0.018061  \nmin                   0.055040  \n25%                   0.071460  \n50%                   0.080040  \n75%                   0.092080  \nmax                   0.207500  \n\n[8 rows x 31 columns]"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df_data.describe()"
        }, 
        {
            "source": "<a id=\"model\"></a>\n## 3. Create XGBoost model\n\nIn recent years, ensemble learning models took the lead and became popular among machine learning practitioners.\n\nThe ensemble learning model employs multiple machine learning algorithms to overcome the potential weaknesses of a single model. For example, if you are going to pick a destination for your next vacation, you probably ask your family and friends, read reviews and blog posts. Based on all the information you have gathered, you make your final decision.\n\nThis phenomenon is referred as the Wisdom of Crowds (WOC) in social sciences and it states that averaging the answers (prediction or probability) of a group will often result better than the answer of one of its members. The idea is that the collective knowledge of diverse and independent individuals will exceed the knowledge of any one of those individuals, helping to eliminate the noise.\n\nXGBoost is an open source library for ensemble-based algorithms.\n\nXGBoost classifier makes its predictions based on the majority vote from collection of models which are a set of classification trees. It uses the combination of weak learners to create single strong learner. It\u2019s a sequential training process, new learners focus on the misclassified examples of previous learners.\n\nIn this section you will learn how to train and test a XGBoost model.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 3.1: Prepare the data", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "After looking at descriptive statistics, we can prepare our data set for model building. We will use the \"diagnosis\" columns as our target variable.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 5, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "y = df_data['diagnosis'].map(lambda d: 1 if d == 'M' else 0)\nX = df_data.drop(['diagnosis', 'id'], axis=1)"
        }, 
        {
            "source": "We split the data set into: train and test data sets.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 6, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n  \"This module will be removed in 0.20.\", DeprecationWarning)\n"
                }
            ], 
            "source": "from sklearn.cross_validation import train_test_split \n\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.35, random_state=143)"
        }, 
        {
            "execution_count": 7, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Number of training records: 11070\nNumber of testing records : 6000\n"
                }
            ], 
            "source": "print(\"Number of training records: \" + str(X_train.size))\nprint(\"Number of testing records : \" + str(X_test.size))"
        }, 
        {
            "source": "As you can see our data has been successfully split into two data sets:\n- The train data set, which is the largest group, is used for training.\n- The test data set is used for model evaluation and is used to test the assumptions of the model.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 3.2: Create the XGBoost model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "We start with importing necessary libraries.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 8, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "import xgboost\nfrom xgboost.sklearn import XGBClassifier\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nfrom sklearn.cross_validation import cross_val_score\nfrom sklearn.metrics import accuracy_score\n\nfrom xgboost import plot_importance\nfrom matplotlib import pyplot\nimport pprint\n%matplotlib inline"
        }, 
        {
            "source": "In the cell below, we create our pipeline which contains the XGBoost classifier:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 9, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "pipeline = Pipeline([('scaler', StandardScaler()), ('classifier', XGBClassifier())])"
        }, 
        {
            "source": "We can see the default parameters of the stages in our pipeline:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 10, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 10, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "Pipeline(memory=None,\n     steps=[('scaler', StandardScaler(copy=True, with_mean=True, with_std=True)), ('classifier', XGBClassifier(base_score=0.5, colsample_bylevel=1, colsample_bytree=1,\n       gamma=0, learning_rate=0.1, max_delta_step=0, max_depth=3,\n       min_child_weight=1, missing=None, n_estimators=100, nthread=-1,\n       objective='binary:logistic', reg_alpha=0, reg_lambda=1,\n       scale_pos_weight=1, seed=0, silent=True, subsample=1))])"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "pipeline"
        }, 
        {
            "source": "After we have set up our pipeline with the XGBoost classifier, we can train it by invoking the fit method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 11, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "execution_count": 11, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "Pipeline(memory=None,\n     steps=[('scaler', StandardScaler(copy=True, with_mean=True, with_std=True)), ('classifier', XGBClassifier(base_score=0.5, colsample_bylevel=1, colsample_bytree=1,\n       gamma=0, learning_rate=0.1, max_delta_step=0, max_depth=3,\n       min_child_weight=1, missing=None, n_estimators=100, nthread=-1,\n       objective='binary:logistic', reg_alpha=0, reg_lambda=1,\n       scale_pos_weight=1, seed=0, silent=True, subsample=1))])"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "pipeline.fit(X_train.values, y_train.values)"
        }, 
        {
            "source": "We plot the feature importance based on fitted trees which allows us to see the features that were useful to construct boosted trees.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 12, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 12, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "<matplotlib.axes._subplots.AxesSubplot at 0x7f5633ec1908>"
                    }, 
                    "output_type": "execute_result"
                }, 
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEWCAYAAABi5jCmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmYFOW1x/HvDwYVZVMQHBkFSWRR\nhBEUNHIRNCAIUWO4LkERl7igQWNIojEqmpsbE0XUq0lQNO64o8YQIwqoMRoFgYgCooEIiIAIkWGR\nmeHcP6pm7BmmZ3pgqquX83mefuiurq4+L+K8U1XnnFdmhnPOOVehUdwBOOecyyw+MTjnnKvCJwbn\nnHNV+MTgnHOuCp8YnHPOVeETg3POuSp8YnAuRZL+IOnauONwLmryOgYXNUnLgHZAecLmzmb26S4c\ncwDwsJkV7Vp02UnS/cAKM/tF3LG43ONnDC5dvmNmzRIeOz0pNARJBXF+/66Q1DjuGFxu84nBxUrS\nUZL+LmmDpPnhmUDFe+dKWihpo6R/Sboo3L4X8Bdgf0kl4WN/SfdL+p+Ezw+QtCLh9TJJP5P0T2CT\npILwc09LWitpqaSxtcRaefyKY0v6qaQ1klZJOkXSiZI+lPSFpJ8nfHa8pKckPR6O511JPRPe7yZp\nVvj38L6kk6p97+8lTZO0CTgfGAn8NBz7n8L9rpL0cXj8DyR9N+EYoyX9TdItktaHYx2a8P4+kv4o\n6dPw/WcT3hsuaV4Y298l9Uj5P7DLSj4xuNhIag/8GfgfYB9gHPC0pH3DXdYAw4EWwLnAREm9zGwT\nMBT4dCfOQM4EhgGtgO3An4D5QHvgeOAKSSekeKz9gD3Cz14H3AOcBfQG/gu4TlKnhP1PBp4Mx/oo\n8KykJpKahHG8BLQFfgg8IqlLwme/D/wKaA48CDwC/DYc+3fCfT4Ov7clcAPwsKTChGP0BRYDbYDf\nAvdKUvjeQ8CewKFhDBMBJPUC7gMuAloDk4DnJe2e4t+Ry0I+Mbh0eTb8jXNDwm+jZwHTzGyamW03\ns+nAbOBEADP7s5l9bIFXCX5w/tcuxnGHmS03sy3AkcC+ZnajmW0zs38R/HA/I8VjlQK/MrNS4DGC\nH7i3m9lGM3sfeB9I/O16jpk9Fe5/K8GkclT4aAbcFMYxA3iBYBKr8JyZvRH+PW2tKRgze9LMPg33\neRxYAvRJ2OXfZnaPmZUDDwCFQLtw8hgKXGxm682sNPz7BvgBMMnM/mFm5Wb2APBVGLPLUVl7ndVl\nnVPM7OVq2zoA/y3pOwnbmgAzAcJLHdcDnQl+idkTeG8X41he7fv3l7QhYVtj4PUUj7Uu/CELsCX8\nc3XC+1sIfuDv8N1mtj28zLV/xXtmtj1h338TnInUFHeNJI0CrgQ6hpuaEUxWFT5L+P7N4clCM4Iz\nmC/MbH0Nh+0AnCPphwnbdkuI2+UgnxhcnJYDD5nZD6q/EV6qeBoYRfDbcml4plFx6aOmdLpNBJNH\nhf1q2Cfxc8uBpWZ28M4EvxMOqHgiqRFQBFRcAjtAUqOEyeFA4MOEz1Yfb5XXkjoQnO0cD7xpZuWS\n5vH131dtlgP7SGplZhtqeO9XZvarFI7jcoRfSnJxehj4jqQTJDWWtEd4U7eI4LfS3YG1QFl49jA4\n4bOrgdaSWiZsmwecGN5I3Q+4oo7vfxv4Mrwh3TSMobukIxtshFX1lnRqmBF1BcElmbeAfxBMaj8N\n7zkMAL5DcHkqmdVA4v2LvQgmi7UQ3LgHuqcSlJmtIriZ/ztJe4cx9A/fvge4WFJfBfaSNExS8xTH\n7LKQTwwuNma2nOCG7M8JfqAtB34CNDKzjcBY4AlgPcHN1+cTPrsImAL8K7xvsT/BDdT5wDKC+xGP\n1/H95QQ/gIuBpcDnwGSCm7dReA44nWA8ZwOnhtfztwEnEVzn/xz4HTAqHGMy9wKHVNyzMbMPgAnA\nmwSTxmHAG/WI7WyCeyaLCG76XwFgZrMJ7jPcGcb9ETC6Hsd1WcgL3JxLA0njgW+a2Vlxx+JcXfyM\nwTnnXBU+MTjnnKvCLyU555yrws8YnHPOVZGVdQytWrWyb37zm3GHkTabNm1ir732ijuMtPIx5wcf\nc/rMmTPnczPbt+49s3RiaNeuHbNnz447jLSZNWsWAwYMiDuMtPIx5wcfc/pI+neq+/qlJOecc1X4\nxOCcc66KrLyU5Jxzuapjx440b96cxo0bU1BQwOzZs/niiy84/fTTWbZsGR07duSJJ55g7733jiyG\nSM8YJI1VsNDK05LelPSVpHEJ7+8h6W0FC7S8L+mGKONxzrlsMHPmTObNm1d5L/Wmm27i+OOPZ8mS\nJRx//PHcdNNNkX5/1GcMYwj6v2wiaN97SrX3vwKOM7OScLGSv0n6i5m9FXFczjmXNZ577jlmzZoF\nwDnnnMOAAQP4zW9+E9n3RTYxSPoDQffH54H7zGyipGGJ+1hQXVcSvmwSPuqsuNtSWk7Hq/7cwBFn\nrh8fVsboPBov+JjzRT6O+f4htaeqSmLw4MFI4qKLLuLCCy9k9erVFBYGi/EVFhayZs2aSGOMbGIw\ns4slDQEGmtnnyfZTsLD5HOCbwF1m9o8k+10IXAjQps2+XHdYWQRRZ6Z2TYP/gfKJjzk/5OOYS0pK\nKn/7r8nNN99MmzZtWL9+PePGjWPLli2UlZVV+Uz11w0t9pvPYevjYkmtgKmSupvZghr2uxu4G6BL\nly72w5EnpznS+MyaNYvT8jDX28ec+/J1zKnWMcyfP5/S0lLat29Ply5dKCwsZNWqVey///6R1kJk\nTLpquHLULGBIzKE451wsNm3axMaNGyufv/TSS3Tv3p2TTjqJBx54AIAHHniAk0+O9hfjWCcGSfuG\nZwpIagp8m2ChEOdchtu6dSt9+vShZ8+eHHrooVx//fUAmBnXXHMNnTt3plu3btxxxx0xR5o9Vq9e\nTb9+/ejZsyd9+vRh2LBhDBkyhKuuuorp06dz8MEHM336dK666qpI44jy5vNYggXD10paSHAPoRGw\nWdIVwCEEl4aGSyonWBnqCTN7IaqYnHMNZ/fdd2fGjBk0a9aM0tJS+vXrx9ChQ1m4cCHLly9n0aJF\nNGrUKPIbpbmkU6dOzJ8/f4ftrVu35pVXXklbHFHeYxgDdAEKgYVmtj5ct3e8mfUFkDQR+CXwoJml\ntD6tcy4zSKJZs2YAlJaWUlpaiiR+//vf8+ijj9KoUXBBom3btnGG6XZCJJeSqqWq9jWz9eFbbwFF\nFfuZ2WvAF1HE4JyLXnl5OcXFxbRt25ZBgwbRt29fPv74Yx5//HGOOOIIhg4dypIlS+IO09VTZAv1\nSFoGHJGYqhpWPXc1swsStnUEXqjrjKFaumrv6267J4KoM1O7prB6S9xRpJePOTMc1r5lSvuVlJRw\n7bXXMnbsWMaMGcO5557LaaedxmuvvcZTTz2V9D5DSUlJ5VlHvohrzAMHDpxjZkektLOZRfIAlgFt\nEl4PBBYCravt1xFYUJ9jd+7c2fLJzJkz4w4h7XzM2Wf8+PF28803W5cuXWzp0qVmZrZ9+3Zr0aJF\n0s9k+5h3RlxjBmZbij9j05KVJKkHMBk42czWpeM7nXPRWrt2LRs2bABgy5YtvPzyy3Tt2pVTTjmF\nGTNmAPDqq6/SuXPnOMN0OyHyAjdJBwLPAGeb2YdRf59zLj1WrVrFOeecQ3l5Odu3b+e0005j+PDh\n9OvXj5EjRzJx4kSaNWvG5MmT4w7V1VM6Kp+vA1oDv5MEUGbhdS5JU4ABQBtJK4DrzezeNMTknEvB\n1q1b6d+/P1999RVlZWWMGDGCG264gdGjR/Pqq6/SsmVLGjVqxIMPPkhxcTEArVq14s9/zq/+R7km\nyl5JHcOnF4SPSmGNwyXABwT3IvYBbvNJwbnMkqxWAYKePiNGjIg5QheFuHol1dWO2zmXAZLVKrjc\nlvaJIZV23HXxttu5z8ecXstuSv6/YHl5Ob179+ajjz7i0ksvpW/fvvz+97/nmmuu4cYbb6xcOGb3\n3XdPY8QuSpHVMdT6pdVqHCSNB0rM7JZaPuN1DHnEx5xeqdQrJNYqtGjRgn322YfS0lImTJjA/vvv\nzznnnFPv7/U6hvSpTx1D7G23U2XedjvuMNLKx5yZ5syZw7p16zj33HMrt+22227ccsstO9UGuj4t\nqHNFNow5Y9puO+cyT7JahVWrVgFBgeyzzz5L9+7e6iyXZM0Zg3MudcnSTEeOHMns2bNp0qQJffr0\nYdKkSTRp0iTpcZLVKhx33HGsXbsWM6O4uJg//OEPaRydi1qkE0NCWmpX4L1wcwnB2s5I6gAsARqH\nr68Diszsyyjjci7XJUszHTlyJA8//DAA3//+95k8eTKXXHJJ0uP06NGDuXPn7rC9orLZ5aaoLyWN\nAU4EjgGONbMeBG22V4Q3nj8B9jGzxsAeBL2UDok4JudyXrI00xNPPBFJSKJPnz6sWLEi5khdJops\nYkil9XbY26kk3N4kfKQ/Tcq5HFRTS+wKpaWlPPTQQwwZ4ivpuh1Fmq6aSuttSY2BOQQrvN1lZj9L\ncixPV80jPua67UxL7IMOOgiAW265hT322IPLLrtsZ0JtMJ6umj4Zm64qaSBwPtCvYpuZlQPF4drP\nUyV1N7MF1T/r6aoD4g4jrXzMDSsxzfSGG26goKCAJ554onKVtbhkQ+pmQ8uGMaftX0VdrbfNbAMw\nC/BzW+d2UbI008mTJ/PXv/6VKVOmxD4puMyVljOGZK23Je0LlJrZBklNgW8Dv0lHTM7lsmRppgUF\nBXTo0IGjjz4agFNPPZXrrrsu5mhdpknXrwyJrbfnSZodbi8EZkr6J/AOMN3MXkhTTM7lrM6dO9Ok\nSRMkYWaUl5cDcPrpp1NQUEBZWRm9evXi6quvjjlSl4mizEoaC2wB1gJ9gOXAZuAcMztCUhfgQUDA\nduBAwOsXnGsAFXUM8+fPZ968ebz44ou89dZbjBw5kkWLFvHee++xZcsWX0TH1SjKS0kVrbULgYVm\ntl7SUIIbyH3NbDFQDJWZSSuBqRHG41zeqK2OoYLXMbhkIpkYamit/ffwrcoahmqOBz42s3+ncnxv\nu537fMx1q61VNtTcLrtCRR3D7bffvtPxutwVWR1DKjUMCdvvA941sztrOZ7XMeQRH3PdvI4hO2VD\nHQNmFsmDYMnONgmvBxK0vGhdbb/dgM+Bdqkeu3PnzpZPZs6cGXcIaedjbljjx4+3m2++ufL5ySef\nbOXl5ZF9X6r8v3P6ALMtxZ+xaclKqqOGYSjB2cLqdMTiXD7wOga3KyL/l5GshiHBmcCUqONwLhdt\n3bqVPn360LNnTw499FCuv/56AG699VbatWuHJHr16sWgQYMYPnw4F198MatXr+boo4+muLiYG2+8\nMeYRuEwUdYHbD4CfAq2AV8JFxMuBY8xsvqQ9gUHAJZLmAivNbHjEMTmXM5K11z7ttNO46KKLGDBg\nAK+//jpt2rQBoKysLOaIXTaIbGIws46SFgG9SJ6yuhloLelKgvsPLaKKx7lclCwt9fDDD485MpfN\nYm27He5XBAwjuAfhnKun2tprO7czMqHt9lPAr4HmwLhkl5I8XTXuKNLLx/y1XUlLPeOMM5g0aRIt\nW6Z2jHTzdNX0yZq225KGA2vMbI6kAbV91rztdtxhpJWPeeckttcG2GOPPTjmmGMq7zFkmmxoQd3Q\nsmHMcbfdPgY4KTyzeAw4TtLD6YrJuWyXLC3VuV2RrjqGGlNWzexqMysys47AGcAMMzsrHTE5lwtW\nrVrFwIED6dGjB0ceeWRlWuodd9xBUVERK1asoEePHlxwwQV1H8y5ULouJSW23QYoS/Val3OZZPny\n5YwaNYrPPvuMRo0aceGFF3L55Zczfvx47rnnHvbdd18A/vd//7dKw7qo9OjRg7lz5+6wfezYsYwd\nOzby73e5KdKJITwTALggfABBS25JCwlabS9JiKWbpH3M7Iso43JuZxUUFDBhwgR69erFxo0b6d27\nN4MGDQLgRz/6EePGjYs5Qud2XVpvPicYAww1s6UVGyR9B/iRTwoukxUWFlJYWAhA8+bN6datGytX\nrow5KucaVtonhsT6Bkn3mdnE8K2UW2N42+3cF+eY62pnXbnfsmXMnTuXvn378sYbb3DnnXfy4IMP\ncsQRRzBhwgT23nvviCN1LhqR1jEk/dJq9Q1ha4wVwDeTnTF4HUPcUaRXnGNOpW5gy5YtXH755Zx1\n1ln079+fL774gpYtWyKJ++67j3Xr1vGzn/2sXt/rOf35IRvqGCJru13bgx1bcp8O/CnVz3vb7dyX\nyWPetm2bDR482CZMmFDj+0uXLrVDDz203sfN5DFHxcecPmRa2+0UnIF3WHVZwMw4//zz6datG1de\neWXl9lWrVlU+nzp1Kt27d48jPOcaRFw3nytJagkcC3j9gkurZKmn1157Lc899xyNGjWibdu23H//\n/ey///4AvPHGGzz00EMcdthhFBcXA0Fq6pQpU5g3bx6S6NixI5MmTYpzaM7tkrgmhubA3yW9A0wH\nZgNfSjrdzJ6KKSaXZ5Klnv7kJz/hl7/8JQB33HEHN954I3/4wx8A6NevX8XlzyrSUbPgXLrENTGs\nJUxXldQYGAX8NaZYXJ5Klnp6yCGHVO6zadMmwqJM5/JG7OmqgAFPA0emOxbnKiSmngJcc801PPjg\ng7Rs2ZKZM2fGHJ1z6RVruiqwO/AocBxwL/BCsktJnq4adxTp1ZBjriv9tHrqaaJHHnmEbdu2VXYr\njZKnbuYHT1etI10VeBI4Ktx2PzAilc97umruS9eY60o9XbZs2U6lnu4M/++cH7IhXTXurKQjgMfC\na7htgBMllZnZs/GG5fKBJUk9XbJkCQcffDAAzz//vLexdnkn1onBzA6qeC7pfoJLST4puLRIlnp6\n7733snjxYho1akSHDh0qM5KcyxdxnzE412CS1SU8+eSTjB8/noULF/L2229zxBHBZVZPPXWuZpFN\nDJLGApcAXYH3ws0lwCVm1lHSAZJmAvsB24G7zWsY3C5IVpfQvXt3nnnmGS666KK4Q3QuK0R5xjAG\nGAoUAgvNbL2koQTrNvcFyoAfm9m7kpoDcyRNN7MPIozJ5bBkdQkV6yU451ITycSQWKsA3Gdmfw/f\negsoAjCzVcCq8PnGcOGe9kCdE4O33c59yca8My2xnXP1E8nEYGYXSxoCDLSwtXbofOAv1feX1BE4\nHPhHsmNWq2PgusPKGjLkjNauafCDMp8kG/OsWbPq/GxFXcIFF1zAu+++W7l9w4YNzJkzh5KSkoYM\ntcGUlJSkNL5c4mPOUKnmtdb3wY6ttQcCC4HW1fZrBswBTk312F7HkPt2dsy11SUce+yx9s477+xi\nZNHx/875wesYQpJ6AJMJ+iOtS9jehKAdxiNm9kw6YnG5y5LUJTjn6ify9RgkHQg8A5xtZh8mbBdB\nG4yFZnZr1HG4+J133nm0bdu2yloFp59+OsXFxRQXF9OxY8fKeoKdUVGXMGPGjMpjTps2jalTp1JU\nVMSbb77JsGHDOOGEExpiOM7lrCjPGJoDfweaAAcAf5O0ClhlQb+OY4Czga2SLiJopjfCzKZFGJOL\n0ejRo7nssssYNWpU5bbHH3+88vmPf/xjWrase1nNZJLVJQB897vf3enjOpdvopwY1hKkq24COgCn\nAOvN7BYAM/ubpH+TsPazy239+/dn2bJlNb5nZjzxxBPMmDEjvUE553YQyaWkaumqI83sHaA0iu9y\nueH111+nXbt2lT2KnHPxSXe66g67Ai9JMmCSmd2dbMfq6ar/98hzDRpzJmvXlKwZb10trj/77DM2\nbdq0Q7rexIkT6dOnT+X2rEjpa2A+5vyQFWNONX2pvg92TFcdD4yrts/+4Z9tgflA/1SO7emq2Wvp\n0qU7tLEuLS21tm3b2vLlyyu35dKYU+Vjzg/ZkK4aeVZSbczs0/DPNcBUoE+c8bh4vPzyy3Tt2pWi\noqK4Q3HOkYZ01WQk7RX2SELSXsBgYEFc8bjonXnmmRx99NEsXryYoqIi7r33XgAee+wxzjzzzJij\nc85ViLzATdJ+wGygBbBd0hXAIQQL80wNF+kpAB41sxejjiefnHfeebzwwgu0bduWBQuCOTdZC+p0\nmDJlSo3b77///rTF4JyrW5RnDLcCrxM0yvuC4J7DB8AwM/vSzP4FHAssARoDZ0k6OsJ48s7o0aN5\n8cWqc21FC+rqaxs751yFONtuA9wOvGhmIyTtBuwZYTx5p6a6gW7dusUTjHMua8TWdltSC6A/MBrA\nzLYB21I5vrfd/lqqbaidcy5Vcbbd7kRQHf1HST0JOqxebmabajqmt92uebx15UMnqxvwFtSZx8ec\nH7JizKnmtdb3QR1tt4EjCFZx6xu+vh34ZSrH9jqG1NVUN2DmLagzkY85P3gdQyih7fbJ9nXb7RXA\nCjOrWJznKaBXOuJxzjmXXGxtt83sM2C5pC7hpuNJYVnPfHb77bfTvXt3Dj30UG677bY696+pbsBb\nUDvn6hJn222AN4F54doMSwhuRrsaLFiwgHvuuYe3336b3XbbjSFDhjBs2LBam84lqxvwFtTOudpE\necawFjiBIDX1aODXwMSKSUFSd+BEoDXB8p6rCYreXA0WLlzIUUcdxZ577klBQQHHHnssU6dOjTss\n51wOirPtdjfgLTPbbGZlwKuA/yqbRPfu3XnttddYt24dmzdvZtq0aSxfvjzusJxzOajel5Ik7Q0c\nYGb/TLaPpdZ2ewHwK0mtgS0EZw+za/nenG+7naxldUVK6cknn8zRRx9N06ZN6dChA5999lnmp73t\npKxI6WtgPub8kA1jliVZCrHKTtIs4CSCiWQewWWiV80s6YrrkpaRsDqbpPFAiYUruIXbzgcuBUoI\nbjxvMbMf1RVPly5dbPHixXXGnStmzZrFgAEDqmz7+c9/TlFREWPGjIknqIjVNOZc52POD3GNWdKc\nhPu7tUr1UlJLM/sSOBX4o5n1Br69swFWMLN7zayXmfUn6Ke0ZFePmcvWrFkDwCeffMIzzzzjHUmd\nc5FI9VJSgaRC4DTgmob6ckltzWxNmNJ6KsFNapfE9773PdatW0eTJk2466672HvvveMOyTmXg1I9\nY7gR+CvwsZm9I6kTKf52L2k/SSuAK4FfSFoR9kkCeFrSB8CfgEvNbH09488rI0aMoFGjRpSVlfHe\ne+/FHY5zLkelNDGY2ZNm1sPMLglf/8vMvlfHZzqa2edm9pmZFZlZCzNrZWZFwGhJC4FPCLqwGnCH\npFd3cTw5K7GOYf78+bzwwgssWeJX3pxzDS+liUFSZ0mvSFoQvu4h6Re78L1jCLKQLgV+B5xkZocC\n/70Lx8xpXsfgnEuXVO8x3AP8BJgEYGb/lPQo8D/1/cJqNQ6PAc+Y2SfhcdekcoxcbbtdWwvt7t27\nc80117Bu3TqaNm3KtGnT0rr6mnMuf6Q6MexpZm+Hy3BW2Km+14k1DsAvgCZhOmxz4HYze7Cmz+VD\n2+1kuc1ex5AffMz5ISvGnEoLVoI1FL4BvBu+HgH8JdUWrjUcbxlB+4s7CRbv2St8vQToXNfnve22\n2dVXX2133XVX+oNJE2/HnB98zOlDPdpup3rGcCnBkpxdJa0ElgIjG2BeWgF8bsHiPJskvQb0BD6s\n/WP5ac2aNbRt27ayjuHNN9+MOyTnXA6qc2KQ1IiggvnbkvYCGpnZxgb6/ueAOyUVALsRNNyb2EDH\nzloTJ05k8uTJSOKwww7j3HPPBbyOwTmXHnVODGa2XdJlwBOWZNnNmkgaC1wCdAUqku5Lwm0VBgOH\nhtvXA78xswWpfkcuWrlyJXfccQcffPABTZs25bTTTmPGjBkMHjyY119/Pe7wnHN5INVLSdMljQMe\nByonBzP7opbPjAGGAoXAQjNbL2kocLeZdQzbbv+AIENpG/AikHupRjuhrKyMLVu20KRJEzZv3kzr\n1q3jDsk5l0dSrXw+j+A+w2vAnPBRWyfUxJTUvvZ1RfNbQFH43Ntu16B9+/aMGzeOAw88kMLCQlq2\nbMmRRx4Zd1jOuTyS0hmDmR1Un4Na8rbb5xNkOEEet91O1l4bYOPGjTzwwAM8/PDDNGvWjPHjx/On\nP/0pjdFlhqxI6WtgPub8kA1jTmlikDSqpu2WpOYgyTEGEkwM/cLPLpT0G2A6wT2G+dRSG2FmdxNk\nRtGlSxf74ciTU/3qrPLkk09y+OGHc8oppwDw6aef8vTTT3tr4jzgY84P2TDmVO8xJF7L2AM4HngX\nSGlikNQDmAwMNbN1FdvN7F7g3nCf/yVIX81rBx54IG+99RabN2+madOmvPLKK3To0CHusJxzeSTV\nS0k/THwtqSXwUCqfDVtqPwOcbWYfVnvP225X07dvX0aMGEGvXr0oKCjg8MMPZ/jw4XGH5ZzLI/Ve\n2jO0GTg4xX2vA1oDvwtbapTZ16sIPR3eYyjF225XatWqFY0bNwaCDCXnnEunVLur/knS8+HjBWAx\nQcZRbW4FXgf2BhYRZCE9XDEpSDqA4J6CCCao7js3hNxSUccwe/ZsFixYQHl5OTNmzIg7LOdcHkn1\njOGWhOdlwL/NrK77ARV1DJuADsAp1d4vA35sZu9Kag7MkTTdzD5IMaac5XUMzrk4pToxnGhmP0vc\nIOk31bclvJdYx3CfmU2UVKWntJmtAlaFzzeGC/e0B+qcGLK97XZt7bUT6xiaNm3K4MGDvY7BOZdW\nqU4Mg4Dqk8DQGrYBtdYx1EhSR+Bw4B+17JMzbbdry2H2OoZANuR6NzQfc37IhjHXOjFIuoTgklAn\nSf9MeKs58EZDBCCpGfA0cIWZfZlsP69jGBBvYGmWDbneDc3HnB+yYcx1nTE8SlCp/GvgqoTtG+vo\nk5QSSU0IJoVHzOyZXT1eLvA6Budc3GqdGMzsP8B/gDMhqDsgKHBrJqmZhUty7gwFuav3EjTYu3Vn\nj5Nr+vbtS1FREXvvvTeS2G+//bjzzjvjDss5l0dSTVf9jqQlBAv0vEqwAttfav1QcLnp75JekPQV\ncD3wS0krJLUAjgHOBi6TtEXSPEkn7uxAcsXKlSv58MMP2bBhA1u3bqVPnz787W9/izss51weSfXm\n8/8ARwEvm9nhYd+jM+v4zFpG6NCLAAAV+klEQVR2TFddb2YVqa9/k3QsQZ+kB82suN7R5yhPV3XO\nxSnVttulYY+jRpIamdlMIOkP8mrpqiPN7B2C6uYqzOw1YJfvVeQSb7vtnItbqmcMG8LsodeBRySt\nofZOqPVKV02Ft93OL9mQ0tfQfMz5IRvGnOrEcDLBmglXACOBlsCNUQVVE09XHRBvYGmWDSl9Dc3H\nnB+yYcypdlfdJKkDcLCZPSBpT6BxtKHlJ09Xdc7FLdWspB8ATwGTwk3tgWejCiqfJbbdPuyww9i+\nfbu33XbOpVWqN58vJUgv/RLAzJYAbVP5oKT9JK0ArgR+kZCuiqQpwJtAl3D7+fUdQLZavHgxxcXF\nlY8WLVpw2223AXDDDTewaNEiFixYwEMPPcRuu+0Wc7TOuXyS6j2Gr8xsW7ieApIKAKvtA2bWMeFl\nUeJ7ksaG7TYWEazzfGAYy/YU48l6Xbp0Yd68eQCUl5fTvn17vvvd78YclXPOpX7G8KqknwNNJQ0C\nngR2JVVmDHAi8A7wgZn1BAYAEyTl3a/Hr7zyCt/4xjf8XoJzLiOkesZwFXA+8B5wETCNYA3neqtW\n4/Ao0Dxsj9GMoKahzrap2dJ2u7b22okee+wxzjyzrnpB55xLD5klvyIk6cBd6YdUy3GXAUcAXxFM\nEF0JWmicbmY1/sSvVsfQ+7rb7mnosBpcbfUKFUpLSxkxYgR//OMf2WeffWrcp6SkhGbNmjV0eBnN\nx5wffMzpM3DgwDkJyyrXzsySPoB3E54/Xdu+9XkQ9FpqA4wAJhIs7/lNgl5MLer6fOfOnS1XPPvs\nszZo0KBa95k5c2Z6gskgPub84GNOH2C2pfgzuq57DEp43in1uSll5wLPhHF/FE4MXSP4now1ZcoU\nv4zknMsodU0MluR5Q/kEOB5AUjugC/CvCL4n4yxevJgePXrw1FNPceutt1ZJV3XOuTjVdfO5p6Qv\nCc4cmobPCV+bmbWo7cOSxgKXEJwFvBduLgGahM9/CfxJ0tXh66nWQL2VMl2XLl345z+DRfE8XdU5\nl0nqWqhnV9tejCFovV1IsCDPeklDgfFm9rmkxkAroBuwAnhH0iFm9sEufm9W8XRV51wmSbWOod6q\npaX2NbP14Vtv8XXBWx/gIzP7l5ltAx4jaNiXVzxd1TmXSWpNV93lg4dpqYmXhySNA7qa2QWSRgBD\nzOyC8L2zCSaRy2o4lqer5hEfc37wMadPfdJVUy1waxDhym/nA/0qNtWwW40zleVo2+3nnnuOvn37\ncuqppybdJxva9DY0H3N+8DFnprRNDJJ6EFRLD7VgNTgI7isckLBbEfBpumLKBJ6u6pzLNJHdY0gk\n6UDgGeBsM/sw4a13gIMlHRT2SDqD4J5EXti8eTPTp0+v9WzBOefSLS0TA3Ad0Br4naR5kmYDmFkZ\ncBnwV2Ah8ISZvZ+mmGK1ePFivvWtb3HAAQdw7LHHeh2Dcy5jRH0p6VaCdaI/IGix3Qu4xsxuSdhn\nO8F9BQPKI44nY3jbbedcpop6YqioY9gEdABOSXwzrGO4CxjE13UMz3sdg3POxSeyiaFaHcN9ZjZR\nUvU+1JV1DOFnKuoYap0YvO22c85FJ7KJwcwuljQEGFhLm4v2wPKE1yuAvjXtWK2OgesOq3PZhtjN\nmjWrzn1KS0t5+umnGT58eNL9S0pKUjpWLvEx5wcfc2ZKax1DDbyOwesYauRjzg8+5syUrqykZLyO\nwesYnHMZJu4zhso6BmAlQR3D9+MNaed17NiR5s2b07hxYwoKCpg9e3at+1fUMUyaNClNETrnXN3S\nMjFI2g+YDbQAtkv6BbAaaAtsIKhhAHgw2+sYZs6cSZs2bVLad88992TdunV17+icc2kU6cRgZh0T\nXlZ0VEXSIoI01rXAJjOzsGXGE1HG45xzrm5pv8dQLY31B/Z1e9e9iGaVuLSRxODBg+nduzd33313\n3OE459xOibTtdtIvTWjHLem7wK8JLisNM7M3k3wm9rbbdbXR/vzzz2nTpg3r169n3LhxjB07lp49\ne+7y93pr4vzgY84P3nY7BWY2FZgqqT/BUp/fTrJfVqWrzp8/n9LS0gZJS8uG9LaG5mPODz7mzBR3\numolM3sN+Iak1O7cZphNmzaxcePGyucvvfQS3bt3jzkq55yrv1jPGCR9E/g4vPncC9gNyMo0ndWr\nV1c2wSsrK+P73/8+Q4YMiTkq55yrv7jPGL4HLJA0j6CZ3ukWx02PBtCpUyf+85//sH37dpo0acLU\nqVPjDsk553ZKXGcMFe243wVeAU4kmKQ2xxRPg6lPHYNzzmWiuM4YxhBMBo8AB4ePC4HfxxSPc865\nUNrPGKrVMXQGRoeXj96S1EpSoZmtqu0YcbXdrquNdkUdgyQuuugiLrzwwjRF5pxzDSfWOgbgfuAm\nM/tbuP0V4GdmtkOTIa9j8FzvXOdjzg9ex1C3nG277XUMu8bHnB98zJkp7qyknGm77XUMzrlcEffE\n8DwwSoGjgP/UdX8hE5SXl3P44YczfPjwym2rV6+mX79+9OzZkz59+jBs2DCvY3DOZaVILyVJGgtc\nAnQF3gs3lwBNwufTgKuBr4DtwKuS9jCzrVHGtatuv/12unXrxpdfflm5rVOnTsyfPz/GqJxzrmFE\nfcZQkZZ6DHCsmfUg6Ie0IlwHen+CdZ9bmtkewH8IFuvJWCtWrODPf/4zF1xwQdyhOOdcJCKbGKql\npfY1s/XhW2+RsDYDwVlLU0kFwJ5k+D2GK664gt/+9rc0ahT3VTjnnItGZJeSzOxiSUOAgeHZQYXz\ngb+E+6yUdAvwCbAFeMnMXqrpeNXSVfm/R56LKvSkaalvvvkmpaWlbNy4kXnz5rFu3TpmzZoVWRwV\nSkpK0vI9mcTHnB98zBnKzCJ7AMuANgmvBxIs49k6fL03MAPYl+C+w7PAWXUdt3PnzhaHq666ytq3\nb28dOnSwdu3aWdOmTW3kyJGRf+/MmTMj/45M42PODz7m9AFmW4o/u9N2PSRcunMycLKZVXRQ/Taw\n1MzWmlkp8AzwrXTFVF+//vWvWbFiBcuWLeOxxx7juOOO4+GHH447LOeca1BpmRgkHUjwQ/9sM/sw\n4a1PgKMk7SlJwPEEZxTOOedikq7K5+uA1sDvgp//lJnZEWb2D0lPEXRZLQPmElY3x23r1q3079+f\nr776irKyMkaMGMENN9xQ+f6AAQMyvnrROed2RmQTQ1jDsAVYS1DDsJygrfYlZjY/3GcZsBHYSjBZ\nnB1VPPW1++67M2PGDJo1a0ZpaSn9+vVj6NChHHXUUXGH5pxzkYryjGEMMBQoBBaa2XpJQwnOCPom\n7Fc9aykjSKpsdFVaWkppaSnh2Y5zzuW0SCaGajUM95nZ38O3qtcw7JSGartdVxvt8vJyevfuzUcf\nfcSll15K3759a93fOedyQWRttytaayeeDUgaB3Q1swvC10uB9QQdVSdZ0EE12fEavO12XW20K5SU\nlHDttdcyduxYDjrooF3+3vry1sT5wcecH7ztdgJJAwmK2/olbD7GzD6V1BaYLmmRmb1W0+ct5rbb\nc+bMYd26dZx77rlp/V7Ijja9Dc3HnB98zJkpXemqNdUwYGafhn+uAaYCfdIRTyrWrl3Lhg0bANiy\nZQsvv/wyXbt2jTkq55yLXuQTQ7IaBkl7SWpe8RwYDCyIOp7qli9fzsCBA+nWrRuHHnoot99+OwCr\nVq1i4MCB9OjRgyOPPJJBgwZVabPtnHO5Kh2XknaoYQAeBMYCHSQ1ArYBGwjOGF5MQ0yVCgoKmDBh\nAr169WLjxo307t2bQYMG0aNHD+bOnZvOUJxzLiNE2USvY/j0gvBRSdIiYBDQARhnZrH9Kl5YWEhh\nYSEAzZs3p1u3bqxcuZJDDjkkrpCccy5Wae8dXS2V9fB0f39tli1bxty5cz0t1TmX1yJLV631S8NU\nVqA78DTB2s+fEpw9vJ/kMzudrppKWuqWLVu4/PLLOeuss+jfv3/Kx04HT+nLDz7m/JAN6aqRtt1O\n9iBsxw20AJqF204ElqTy+YZuu71t2zYbPHiwTZgwoUGP21C8NXF+8DHnB2+7XQcz+9LMSsLn04Am\nktqkOQbOP/98unXrxpVXXpnOr3bOuYwU68Qgab+w3TaS+oTxrKv9Uw3rjTfe4KGHHmLGjBkUFxdT\nXFzMtGnT0hmCc85llLRVPicxArhEUhlBJ9YzwlOetOnQoQMDBgzgs88+o1GjRlx44YWceOKJ6QzB\nOecySlwTw63A68C/gA+BbxCcLXyZ7kCS1TF4uqpzLl/FdSlpDMHN5g+AeWbWAxgF3J7uQAoLC+nV\nqxdQtY7BOefyVdrPGKrVMXQCTgAws0WSOkpqZ2araztGfdtu19Veu3I/r2NwzrnY6xiuBPYwsyvD\nm89/B/qa2ZwaPuN1DHnEx5wffMzpU586hrgnhm0El48OJ1j+sytwgYVLfybTpUsXW7x4cYPFU1pa\nyvDhwznhhBMyMmU1G9r0NjQfc37wMaePpMxbj6EmZvYlcC5AmLa6NHykMwavY3DOuQRx1zG0krRb\n+PIC4LVwsmhw5513Hm3btqV79+5Vtnsdg3POVRXpGYOkscAlBNlH+wO9gGsSdukGPC+pBbCVYBW3\nPcxsa0PHMnr0aC677DJGjRpVZXu/fv2I43Kac85lqqgvJY0BhgKbCFpsnwJft+SWtDtQAhxoZlsk\nPQGcAdzf0IH079+fZcuWNfRhnXMu50R2KalaWupIM3sHKK1h1wKgqaQCYE+CLqvOOediEuVCPRdL\nGgIMNLPPk+yzUtItwCcELTFeMrOXatq3Wroq//fIczvsU1da6meffcamTZuYNWtWfYYSu5KSkqyL\neVf5mPODjzkzxZqVJGlv4GTgIIKlPZ+UdJaZPVx9XzO7G7gbgnTVH448ud7ft2zZMvbaa6+sS4/z\nlL784GPOD9kw5lizkoBvA0vNbK2ZlQLPAN+KOSbnnMtrcU8MnwBHSdozrGM4HlgYxRedeeaZHH30\n0SxevJiioiLuvffeKL7GOeeyXlouJUnaD5hNsGLbdklXAIeY2T8kPQW8C5QBcwkvFzW0pk2bUl5e\nTpcuXViwYEEUX+GcczkhyqyksQQ3lCcBU4F9gRvNrJWZFQGFkuYR3GPYChwIzDGzr6KIZ/To0bz4\n4otRHNo553JKlGcMNdYwVDCzxUAxgKTGwEqCCSQSXsfgnHOpiWRiqFbDcJ+ZTZRUW+/r44GPzezf\nqRw/WdvtVNtrO+ecSy6SiSGVGoZqzgCm1LZD9TqG6w4r22GfunKDvY4he/iY84OPOTPFveYzYRO9\nk4Cra9vP6xgGxB1GWvmY84OPOTPFna4KwX2Id+tatc0551x6ZMLEcCZ1XEZqCJ06deIb3/gG77//\nvtcxOOdcLaK+lPQDSecCRcBuQGNJvyDopvqlpD2BQcAlkuYCK81seBSB3H///TRr1oxRo0Z5HYNz\nztUiyiZ6HSUtAk6gasrq+orFeMxsM9Ba0pUEFc8toorH01Wdcy41sbfdllQEDAMmRxWLc8651MXa\ndjt0G/BToHltx/O227PiDiOtfMz5wcecmeJuuz0cWGNmcyQNqG1fT1cdEHcYaeVjzg8+5swUd1bS\nMcBJkpYBjwHHSdphLQbnnHPpE+vEYGZXm1lRuAb0GcAMMzsriu/yttvOOZeauNtuf5mO7weYMiXy\nUgnnnMsJkU4M4ZlAhaI69p0FzIowHOeccymI+x6Dc865DOMTg3POuSpkZnHHUG+SNgKL444jjdoA\nqbQvzyU+5vzgY06fDma2byo7xt52eyctNrMj4g4iXSTNzqfxgo85X/iYM5NfSnLOOVeFTwzOOeeq\nyNaJ4e64A0izfBsv+JjzhY85A2XlzWfnnHPRydYzBueccxHxicE551wVWTUxSBoiabGkjyRdFXc8\nUZB0n6Q1khYkbNtH0nRJS8I/944zxoYm6QBJMyUtlPS+pMvD7Tk7bkl7SHpb0vxwzDeE2w+S9I9w\nzI9L2i3uWBuSpMaS5kp6IXyd6+NdJuk9SfMkzQ63Zfy/66yZGCQ1Bu4ChgKHAGdKOiTeqCJxPzCk\n2rargFfM7GDglfB1LikDfmxm3YCjgEvD/7a5PO6vgOPMrCdQDAyRdBTwG2BiOOb1wPkxxhiFywmW\n8a2Q6+OFYLGy4oTahYz/d501EwPQB/jIzP5lZtsI1m+o/2o9Gc7MXgO+qLb5ZOCB8PkDBGtn5wwz\nW2Vm74bPNxL84GhPDo/bAiXhyybhw4DjgKfC7Tk15urL+EoSOTzeWmT8v+tsmhjaA8sTXq8It+WD\ndma2CoIfokDbmOOJjKSOwOHAP8jxcYeXVeYBa4DpwMfABjMrC3fJtX/jFcv4bg9ftya3xwvBZP+S\npDnh8sSQBf+us6klhmrY5rm2OURSM+Bp4Aoz+zL4hTJ3mVk5UCypFTAV6FbTbumNKhpJlvHNh/+n\njzGzTyW1BaZLWhR3QKnIpjOGFcABCa+LgE9jiiXdVksqBAj/XBNzPA1OUhOCSeERM3sm3Jzz4wYw\nsw0Ea5EcBbSSVPELWy79G99hGV+CM4hcHS8AZvZp+Ocagsm/D1nw7zqbJoZ3gIPDLIbdCJYCfT7m\nmNLleeCc8Pk5wHMxxtLgwmvN9wILzezWhLdydtyS9g3PFJDUFPg2wb2VmcCIcLecGXOSZXxHkqPj\nBZC0l6TmFc+BwcACsuDfdVZVPks6keC3jMbAfWb2q5hDanCSpgADCFrzrgauB54FngAOBD4B/tvM\nqt+gzlqS+gGvA+/x9fXnnxPcZ8jJcUvqQXDjsTHBL2hPmNmNkjoR/Ea9DzAXOMvMvoov0oYXXkoa\nZ2bDc3m84dimhi8LgEfN7FeSWpPh/66zamJwzjkXvWy6lOSccy4NfGJwzjlXhU8MzjnnqvCJwTnn\nXBU+MTjnnKsimyqfnYuUpHKClNkKp5jZspjCcS42nq7qXEhSiZk1S+P3FST0CXIuY/ilJOdSJKlQ\n0mthb/0Fkv4r3D5E0rvh2gqvhNv2kfSspH9KeissaEPSeEl3S3oJeDBspHezpHfCfS+KcYjOAX4p\nyblETcNupwBLzey71d7/PvDXsHq1MbCnpH2Be4D+ZrZU0j7hvjcAc83sFEnHAQ8SrLsA0BvoZ2Zb\nwo6b/zGzIyXtDrwh6SUzWxrlQJ2rjU8Mzn1ti5kV1/L+O8B9YcO/Z81sXtje4bWKH+QJrQ36Ad8L\nt82Q1FpSy/C9581sS/h8MNBDUkW/oJbAwYBPDC42PjE4lyIze01Sf4LFZh6SdDOwgZpbRdfWUnpT\ntf1+aGZ/bdBgndsFfo/BuRRJ6kCwpsA9BN1gewFvAsdKOijcp+JS0mvAyHDbAOBzM/uyhsP+Fbgk\nPAtBUuewE6dzsfEzBudSNwD4iaRSoAQYZWZrw/sEz0hqRNBbfxAwHvijpH8Cm/m6zXJ1k4GOwLth\n+/G1ZOBSjy6/eLqqc865KvxSknPOuSp8YnDOOVeFTwzOOeeq8InBOedcFT4xOOecq8InBuecc1X4\nxOCcc66K/wccwGzQzDBLtQAAAABJRU5ErkJggg==\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x7f5656f64240>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "xgboost.plot_importance(pipeline.steps[1][1])"
        }, 
        {
            "source": "We can now make predictions on test data and evaluate the model.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 13, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n"
                }
            ], 
            "source": "y_pred = pipeline.predict(X_test.values)"
        }, 
        {
            "execution_count": 14, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Accuracy: 94.50%\n"
                }
            ], 
            "source": "accuracy = accuracy_score(y_test, y_pred)\n\nprint(\"Accuracy: %.2f%%\" % (accuracy * 100.0))"
        }, 
        {
            "source": "We can tune our model now to achieve better accuracy by using grid search and cross validation.\n\nXGBoost has an extensive catalog of hyperparameters which provides great flexibility to shape the algorithm\u2019s desired behavior. Let\u2019s have a look at the most important ones:\n- learning_rate (default=0.1): Boosting learning rate (xgb\u2019s \u201ceta\u201d)\n- n_estimators (default=100): Number of boosted trees to fit\n- max_depth (default=3): Maximum tree depth for base learners\n- objective (default='binary:logistic'): Specify the learning task and the corresponding learning objective or a custom objective function to be used", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In below cell, we create our XGBoost pipeline and set up the parameter space.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 15, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "pipeline_gs = Pipeline([('scaler', StandardScaler()), ('classifier', XGBClassifier())])\nparameters = {'classifier__learning_rate': [0.01, 0.03], 'classifier__n_estimators': [50, 200]}"
        }, 
        {
            "source": "We can search for the best parameters over the specified parameters with GridSearchCV. You can use estimator.get_params().keys() to see the available hyperparameters for search.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 16, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/grid_search.py:42: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.\n  DeprecationWarning)\n"
                }
            ], 
            "source": "from sklearn.grid_search import GridSearchCV\n\nclf = GridSearchCV(pipeline_gs, parameters)"
        }, 
        {
            "execution_count": 17, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n"
                }, 
                {
                    "execution_count": 17, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "GridSearchCV(cv=None, error_score='raise',\n       estimator=Pipeline(memory=None,\n     steps=[('scaler', StandardScaler(copy=True, with_mean=True, with_std=True)), ('classifier', XGBClassifier(base_score=0.5, colsample_bylevel=1, colsample_bytree=1,\n       gamma=0, learning_rate=0.1, max_delta_step=0, max_depth=3,\n       min_child_weight=1, missing=None, n_estimators=100, nthread=-1,\n       objective='binary:logistic', reg_alpha=0, reg_lambda=1,\n       scale_pos_weight=1, seed=0, silent=True, subsample=1))]),\n       fit_params={}, iid=True, n_jobs=1,\n       param_grid={'classifier__n_estimators': [50, 200], 'classifier__learning_rate': [0.01, 0.03]},\n       pre_dispatch='2*n_jobs', refit=True, scoring=None, verbose=0)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "clf.fit(X_train.values, y_train.values)"
        }, 
        {
            "source": "We can see the performance result of all parameter combinations including the best parameter combination based on the model performance from the grid search scores.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 18, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 18, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "[mean: 0.93767, std: 0.01382, params: {'classifier__n_estimators': 50, 'classifier__learning_rate': 0.01},\n mean: 0.93225, std: 0.01382, params: {'classifier__n_estimators': 200, 'classifier__learning_rate': 0.01},\n mean: 0.92954, std: 0.01014, params: {'classifier__n_estimators': 50, 'classifier__learning_rate': 0.03},\n mean: 0.94309, std: 0.01150, params: {'classifier__n_estimators': 200, 'classifier__learning_rate': 0.03}]"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "clf.grid_scores_"
        }, 
        {
            "source": "Best score and parameter set", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 19, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Best score: 0.943089430894309\nBest parameter set: {'classifier__n_estimators': 200, 'classifier__learning_rate': 0.03}\n"
                }
            ], 
            "source": "print(\"Best score: %s\" % (clf.best_score_))\nprint(\"Best parameter set: %s\" % (clf.best_params_))"
        }, 
        {
            "source": "We can see the accuracy of the best parameter combination on test set.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 20, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Accuracy: 95.50%\n"
                }, 
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n"
                }
            ], 
            "source": "y_pred = clf.predict(X_test.values)\n\naccuracy = accuracy_score(y_test, y_pred)\nprint(\"Accuracy: %.2f%%\" % (accuracy * 100.0))"
        }, 
        {
            "execution_count": 21, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "[0]\n"
                }, 
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/usr/local/src/conda3_runtime/home/envs/DSX-Python35-Spark/lib/python3.5/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n  if diff:\n"
                }
            ], 
            "source": "y_pred = clf.predict([X_test.values[55]])\nprint(y_pred)"
        }, 
        {
            "source": "Let's see how we can save our XGBoost pipeline using the Watson Machine Learning instance and deploy it for online scoring.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"persistence\"></a>\n## 4. Persist model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In this section you will learn how to use Python client libraries to store your XGBoost model in the Watson Machine Learning repository.\n\nFirst, you must import the client libraries.\n\n**Note**: Apache\u00ae Spark 2.1 or higher is required.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 22, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from repository.mlrepository import MetaNames\nfrom repository.mlrepository import MetaProps\nfrom repository.mlrepositoryclient import MLRepositoryClient\nfrom repository.mlrepositoryartifact import MLRepositoryArtifact"
        }, 
        {
            "source": "Authenticate to Watson Machine Learning service on IBM Cloud.\n\n**Action**: Enter authentication information from your Watson Machine Learning service instance here.</div>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 23, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# Enter your credentials here\nwml_credentials = {\n  \"url\": \"https://ibm-watson-ml.mybluemix.net\",\n  \"access_key\": \"***\",\n  \"username\": \"***\",\n  \"password\": \"***\",\n  \"instance_id\": \"***\"\n}"
        }, 
        {
            "execution_count": 24, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "service_path = wml_credentials[\"url\"]\nusername = wml_credentials[\"username\"]\npassword = wml_credentials[\"password\"]\ninstance_id = wml_credentials[\"instance_id\"]"
        }, 
        {
            "source": "**Tip**: service_path, user and password can be found on **Service Credentials** tab of service instance created in IBM Cloud. If you cannot see **instance_id** field in **Serice Credentials** generate new credentials by pressing **New credential (+)** button. ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 25, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "ml_repository_client = MLRepositoryClient(service_path)\nml_repository_client.authorize(username, password)"
        }, 
        {
            "source": "### 4.1: Save a XGBoost model in the ML Repository\n\nIn this subsection you will learn how to save model artifact to your Watson Machine Learning instance.\n\nCreate an artifact and save to ML Repository", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In this subsection you will learn how to save model artifact to your Watson Machine Learning instance.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Create an artifact and save to ML Repository", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 26, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# Check if props is mandatory\n# AUTHOR_NAME and AUTHOR_EMAIL have been removed for Watson Studio sharing\nprops1 = MetaProps({MetaNames.AUTHOR_NAME:\"***\", MetaNames.AUTHOR_EMAIL:\"***\"})\n\nmodel_artifact = MLRepositoryArtifact(clf, name='Tumor_Type_Detection_v1_pipe', meta_props=props1)\n\nsaved_model = ml_repository_client.models.save(model_artifact)"
        }, 
        {
            "source": "Get saved model metadata from Watson Machine Learning.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 27, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "{'authorEmail': 'umit.cakmak@pl.ibm.com',\n 'authorName': 'Umit',\n 'creationTime': datetime.datetime(2018, 3, 18, 23, 37, 40, 678000, tzinfo=tzutc()),\n 'lastUpdated': datetime.datetime(2018, 3, 18, 23, 37, 40, 850000, tzinfo=tzutc()),\n 'modelType': 'scikit-model-0.19',\n 'modelVersionHref': 'https://ibm-watson-ml.mybluemix.net/v2/artifacts/models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/versions/2d5908e6-513a-42a1-b227-f306dcdfb8ce',\n 'pipelineVersionHref': None,\n 'runtime': 'python-3.5',\n 'trainingDataRef': {},\n 'version': '2d5908e6-513a-42a1-b227-f306dcdfb8ce'}\n"
                }
            ], 
            "source": "dict_meta = saved_model.meta.get()\npprint.pprint(dict_meta)"
        }, 
        {
            "source": "# 5. Deploy and score in a Cloud", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In this section you will learn how to create online scoring and to score a new data record by using the Watson Machine Learning REST API. \nFor more information about REST APIs, see the [Swagger Documentation](http://watson-ml-api.mybluemix.net/).\n", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "To work with the Watson Machine Leraning REST API you must generate an access token. To do that you can use the following sample code:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 28, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "import urllib3, requests, json\n\nheaders = urllib3.util.make_headers(basic_auth='{}:{}'.format(username, password))\nurl = '{}/v3/identity/token'.format(service_path)\nresponse = requests.get(url, headers=headers)\nmltoken = json.loads(response.text).get('token')\nheader = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + mltoken}"
        }, 
        {
            "source": "### 5.1: Create online scoring endpoint", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Now you can create an online scoring endpoint.\n\nGet published_models url from instance details", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 29, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "<Response [200]>\n{\"metadata\":{\"guid\":\"37c88e6f-2a51-43f8-be5a-17a928580075\",\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075\",\"created_at\":\"2018-01-04T14:17:28.939Z\",\"modified_at\":\"2018-03-18T23:37:41.004Z\"},\"entity\":{\"source\":\"Bluemix\",\"published_models\":{\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models\"},\"usage\":{\"expiration_date\":\"2018-04-01T00:00:00.000Z\",\"computation_time\":{\"limit\":180000,\"current\":0},\"model_count\":{\"limit\":200,\"current\":125},\"prediction_count\":{\"limit\":5000,\"current\":0},\"gpu_count\":{\"limit\":8,\"current\":0},\"capacity_units\":{\"limit\":180000000,\"current\":0},\"deployment_count\":{\"limit\":5,\"current\":1}},\"plan_id\":\"3f6acf43-ede8-413a-ac69-f8af3bb0cbfe\",\"status\":\"Active\",\"organization_guid\":\"b0e61605-a82e-4f03-9e9f-2767973c084d\",\"region\":\"us-south\",\"account\":{\"id\":\"f52968f3dbbe7b0b53e15743d45e5e90\",\"name\":\"Umit Cakmak's Account\",\"type\":\"TRIAL\"},\"owner\":{\"ibm_id\":\"31000292EV\",\"email\":\"umit.cakmak@pl.ibm.com\",\"user_id\":\"43e0ee0e-6bfb-48fc-bcd8-d61e40d19253\",\"country_code\":\"POL\",\"beta_user\":true},\"deployments\":{\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/deployments\"},\"space_guid\":\"4c55eb1c-d6fe-4f0a-9390-35c9a7ecf27a\",\"plan\":\"lite\"}}\n"
                }
            ], 
            "source": "#endpoint_instance = service_path + \"/v3/wml_instances/\" + instance_id\nendpoint_instance = service_path + \"/v3/wml_instances/\" + instance_id\nheader = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + mltoken} \n\nresponse_get_instance = requests.get(endpoint_instance, headers=header)\nprint(response_get_instance)\nprint(response_get_instance.text)"
        }, 
        {
            "execution_count": 30, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models\n"
                }
            ], 
            "source": "endpoint_published_models = json.loads(response_get_instance.text).get('entity').get('published_models').get('url')\nprint(endpoint_published_models)"
        }, 
        {
            "source": "Execute the following sample code that uses the published_models endpoint to get deployments url.\n\nGet the list of published models", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 31, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "<Response [200]>\n{\"limit\":1000,\"resources\":[{\"metadata\":{\"guid\":\"6b76f1d7-aff2-445d-b816-0c4c573fbf7d\",\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/6b76f1d7-aff2-445d-b816-0c4c573fbf7d\",\"created_at\":\"2018-02-18T19:51:57.776Z\",\"modified_at\":\"2018-02-18T19:52:24.785Z\"},\"entity\":{\"runtime_environment\":\"spark-2.0\",\"learning_configuration_url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/6b76f1d7-aff2-445d-b816-0c4c573fbf7d/learning_configuration\",\"author\":{\"name\":\"IBM\",\"email\":\"\"},\"name\":\"Product Line Prediction\",\"description\":\"Predicts clients' interests in terms of sport product lines for chain stores in Europe.\",\"label_col\":\"PRODUCT_LINE\",\"learning_iterations_url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/6b76f1d7-aff2-445d-b816-0c4c573fbf7d/learning_iterations\",\"training_data_schema\":{\"type\":\"struct\",\"fields\":[{\"name\":\"PRODUCT_LINE\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"scale\":0,\"name\":\"PRODUCT_LINE\"}},{\"name\":\"GENDER\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"scale\":0,\"name\":\"GENDER\"}},{\"type\":\"integer\",\"name\":\"AGE\",\"nullable\":true,\"metadata\":{\"scale\":0,\"name\":\"AGE\"}},{\"name\":\"MARITAL_STATUS\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"scale\":0,\"name\":\"MARITAL_STATUS\"}},{\"name\":\"PROFESSION\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"scale\":0,\"name\":\"PROFESSION\"}}]},\"feedback_url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/6b76f1d7-aff2-445d-b816-0c4c573fbf7d/feedback\",\"latest_version\":{\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/6b76f1d7-aff2-445d-b816-0c4c573fbf7d/versions/5f6008c9-e8d2-4512-ad19-bff34b15ac8c\",\"guid\":\"5f6008c9-e8d2-4512-ad19-bff34b15ac8c\",\"created_at\":\"2018-02-18T19:51:57.835Z\"},\"model_type\":\"mllib-2.0\",\"deployments\":{\"count\":1,\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/6b76f1d7-aff2-445d-b816-0c4c573fbf7d/deployments\"},\"deployed_version\":{\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/6b76f1d7-aff2-445d-b816-0c4c573fbf7d/versions/5f6008c9-e8d2-4512-ad19-bff34b15ac8c\",\"guid\":\"5f6008c9-e8d2-4512-ad19-bff34b15ac8c\"},\"evaluation_metrics_url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/6b76f1d7-aff2-445d-b816-0c4c573fbf7d/evaluation_metrics\",\"input_data_schema\":{\"type\":\"struct\",\"fields\":[{\"name\":\"GENDER\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"scale\":0,\"name\":\"GENDER\"}},{\"type\":\"integer\",\"name\":\"AGE\",\"nullable\":true,\"metadata\":{\"scale\":0,\"name\":\"AGE\"}},{\"name\":\"MARITAL_STATUS\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"scale\":0,\"name\":\"MARITAL_STATUS\"}},{\"name\":\"PROFESSION\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"scale\":0,\"name\":\"PROFESSION\"}}]}}},{\"metadata\":{\"guid\":\"4b7c9d27-6ca3-4b89-b726-9dd32ccb3958\",\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958\",\"created_at\":\"2018-03-18T23:37:40.678Z\",\"modified_at\":\"2018-03-18T23:37:40.850Z\"},\"entity\":{\"runtime_environment\":\"python-3.5\",\"learning_configuration_url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/learning_configuration\",\"author\":{\"name\":\"Umit\",\"email\":\"umit.cakmak@pl.ibm.com\"},\"name\":\"Tumor_Type_Detection_v1_pipe\",\"learning_iterations_url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/learning_iterations\",\"feedback_url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/feedback\",\"latest_version\":{\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/versions/2d5908e6-513a-42a1-b227-f306dcdfb8ce\",\"guid\":\"2d5908e6-513a-42a1-b227-f306dcdfb8ce\",\"created_at\":\"2018-03-18T23:37:40.850Z\"},\"model_type\":\"scikit-learn-0.19\",\"deployments\":{\"count\":0,\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/deployments\"},\"evaluation_metrics_url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/evaluation_metrics\"}}],\"first\":{\"url\":\"https://deployment/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models?limit=1000\"}}\n"
                }
            ], 
            "source": "header = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + mltoken}\n\nresponse_get = requests.get(endpoint_published_models, headers=header)\nprint(response_get)\nprint(response_get.text)"
        }, 
        {
            "source": "Get published model deployment url", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 32, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/deployments\n"
                }
            ], 
            "source": "[endpoint_deployments] = [x.get('entity').get('deployments').get('url') for x in json.loads(response_get.text).get('resources') if x.get('metadata').get('guid') == saved_model.uid]\n\nprint(endpoint_deployments)"
        }, 
        {
            "source": "#### 5.1.2 Deploy Model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Create online deployment for published model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 33, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "<Response [201]>\n{\"metadata\":{\"guid\":\"65e272bc-ff7e-4136-b210-c7a1a831a7f8\",\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/deployments/65e272bc-ff7e-4136-b210-c7a1a831a7f8\",\"created_at\":\"2018-03-18T23:38:07.407Z\",\"modified_at\":\"2018-03-18T23:38:11.463Z\"},\"entity\":{\"runtime_environment\":\"python-3.5\",\"name\":\"skl_xgb_cancer1_pipe\",\"scoring_url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/deployments/65e272bc-ff7e-4136-b210-c7a1a831a7f8/online\",\"description\":\"skl_xgb_Cancer\",\"published_model\":{\"author\":{\"name\":\"Umit\",\"email\":\"umit.cakmak@pl.ibm.com\"},\"name\":\"Tumor_Type_Detection_v1_pipe\",\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958\",\"guid\":\"4b7c9d27-6ca3-4b89-b726-9dd32ccb3958\",\"created_at\":\"2018-03-18T23:38:07.382Z\"},\"status_details\":{\"status\":\"DEPLOY_SUCCESS\"},\"model_type\":\"scikit-learn-0.19\",\"status\":\"DEPLOY_SUCCESS\",\"type\":\"online\",\"deployed_version\":{\"url\":\"https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/versions/2d5908e6-513a-42a1-b227-f306dcdfb8ce\",\"guid\":\"2d5908e6-513a-42a1-b227-f306dcdfb8ce\"}}}\n"
                }
            ], 
            "source": "payload_online = {\"name\": \"skl_xgb_cancer1_pipe\", \"description\": \"skl_xgb_Cancer\", \"type\": \"online\"}\nresponse_online = requests.post(endpoint_deployments, json=payload_online, headers=header)\n\nprint(response_online)\nprint(response_online.text)"
        }, 
        {
            "execution_count": 34, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/wml_instances/37c88e6f-2a51-43f8-be5a-17a928580075/published_models/4b7c9d27-6ca3-4b89-b726-9dd32ccb3958/deployments/65e272bc-ff7e-4136-b210-c7a1a831a7f8/online\n"
                }
            ], 
            "source": "scoring_url = json.loads(response_online.text).get('entity').get('scoring_url')\nprint(scoring_url)"
        }, 
        {
            "source": "### 5. 3 Perform Prediction", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Now, you can send (POST) new scoring records (new data) for which you would like to get predictions. To do that, execute the following sample code:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 37, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "('{\"fields\": [\"prediction\", \"probability\"], \"values\": [[0, '\n '[0.8568325638771057, 0.1431674212217331]]]}\\n')\n"
                }
            ], 
            "source": "payload_scoring = {\n   \"values\": [[0.16, 0.20, 0.010, 0.081, 11.69, 13.19, 14.78, 848.8, 19.48, 627.7, 43.75, 1.23, 3.27, 0.44, 6697.0, 208.3, 324.8, 139.2, 153.6, 2789.0, 0.2, 0.30, 0.013, 0.001, 15.90, 29.47, 35.97, 15.83, 31.03, 0.08]]        \n}\n\nresponse_scoring = requests.post(scoring_url, json=payload_scoring, headers=header)\npprint.pprint(response_scoring.text)"
        }, 
        {
            "source": "As we can see, given patient record is classified as benign.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"summary\"></a>\n## 6. Summary and next steps     ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "You successfully completed this notebook! You learned how to use XGBoost machine learning as well as Watson Machine Learning for model creation and deployment. Check out our _[Online Documentation](https://console.ng.bluemix.net/docs/services/PredictiveModeling/pm_service_api_spark.html)_ for more samples, tutorials, documentation, how-tos, and blog posts. ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### Data citations\n\nLichman, M. (2013). [UCI Machine Learning Repository](http://archive.ics.uci.edu/ml). Irvine, CA: University of California, School of Information and Computer Science.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### Author\n\n**Umit Mert Cakmak** is Data Scientist at IBM with a track record of developing enterprise-level applications that substantially increases clients' ability to turn data into actionable insights.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Copyright \u00a9 2017, 2018 IBM. This notebook and its source code are released under the terms of the MIT License", 
            "cell_type": "markdown", 
            "metadata": {}
        }
    ], 
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.5", 
            "name": "python3", 
            "language": "python"
        }, 
        "language_info": {
            "mimetype": "text/x-python", 
            "nbconvert_exporter": "python", 
            "version": "3.5.4", 
            "name": "python", 
            "file_extension": ".py", 
            "pygments_lexer": "ipython3", 
            "codemirror_mode": {
                "version": 3, 
                "name": "ipython"
            }
        }
    }, 
    "nbformat": 4
}