1.apply-classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Apply Machine Learning Classifiers to PDX RNAseq data\n",
    "\n",
    "**Gregory Way, 2018**\n",
    "\n",
    "In the following notebook, I apply three distinct classifiers to patient derived xenograft (PDX) RNAseq data (FPKM).\n",
    "The first classifier detects Ras activation. For more details about the algorithm and results, refer to [Way et al. 2018](https://doi.org/10.1016/j.celrep.2018.03.046 \"Machine Learning Detects Pan-cancer Ras Pathway Activation in The Cancer Genome Atlas\"). I also include _TP53_ inactivation predictions. This classifier was previously applied in [Knijnenburg et al. 2018](https://doi.org/10.1016/j.celrep.2018.03.076 \"Genomic and Molecular Landscape of DNA Damage Repair Deficiency across The Cancer Genome Atlas\"). The third is a classifier that predicts _NF1_ inactivation. We previously applied this in [Way et al. 2017](https://doi.org/10.1186/s12864-017-3519-7 \"A machine learning classifier trained on cancer transcriptomes detects NF1 inactivation signal in glioblastoma\").\n",
    "\n",
    "To apply other classifiers (targeting other genes of interest) refer to https://github.com/greenelab/pancancer.\n",
    "\n",
    "Also note that we have implemented classifiers on a larger scale and made the models accessible to non-computational biologists (see [Project Cognoma](http://cognoma.org)).\n",
    "\n",
    "## Procedure\n",
    "\n",
    "1. Load RNAseq matrix (`.RDS`)\n",
    "  * The matrix is in `sample` x `gene symbol` format (250 x 51,968)\n",
    "2. Process matrix\n",
    "  * Take the z-score by gene\n",
    "3. Load classifier coefficients\n",
    "  * For both _Ras_, _TP53_, and _NF1_ classifiers\n",
    "4. Apply each classifier to the input data\n",
    "  * This also requires additional processing steps to the input data (subsetting to the respective classifier genes)\n",
    "  * Also note that not all genes are present in the input RNAseq genes.\n",
    "5. Shuffle the gene expression data (by gene) and apply each classifier to random data\n",
    "\n",
    "### Important Caveat\n",
    "\n",
    "Because not all of the classifier genes are found in the input dataset, the classifier probability is not calibrated correctly. The scores should be interpreted as continuous values representing relative gene alterations, and not as a pure probability estimate.\n",
    "\n",
    "## Output\n",
    "\n",
    "The output of this notebook are the predicted scores for both classifiers across all samples for real data and shuffled data. This is in the form of a single text file with three columns (`sample_id`, `ras_score`, `tp53_score`, `nf1_score`,  `ras_shuffle`, `tp53_shuffle`, `nf1_shuffle`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import rpy2.robjects as robjects\n",
    "from rpy2.robjects import pandas2ri\n",
    "\n",
    "from utils import apply_classifier, shuffle_columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "pandas2ri.activate()\n",
    "loadRDA = robjects.r['load']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make results directory\n",
    "os.makedirs('./results', exist_ok=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(244, 51968)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>gene_short_name</th>\n",
       "      <th>5S_rRNA</th>\n",
       "      <th>7SK</th>\n",
       "      <th>A-575C2.4</th>\n",
       "      <th>A1BG</th>\n",
       "      <th>A1BG-AS1</th>\n",
       "      <th>A1CF</th>\n",
       "      <th>A2M</th>\n",
       "      <th>A2M-AS1</th>\n",
       "      <th>A2ML1</th>\n",
       "      <th>A2ML1-AS1</th>\n",
       "      <th>...</th>\n",
       "      <th>ZWINT</th>\n",
       "      <th>ZXDA</th>\n",
       "      <th>ZXDB</th>\n",
       "      <th>ZXDC</th>\n",
       "      <th>ZYG11A</th>\n",
       "      <th>ZYG11AP1</th>\n",
       "      <th>ZYG11B</th>\n",
       "      <th>ZYX</th>\n",
       "      <th>ZZEF1</th>\n",
       "      <th>ZZZ3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ALL-102</th>\n",
       "      <td>0.0</td>\n",
       "      <td>3.58839</td>\n",
       "      <td>0.537883</td>\n",
       "      <td>4.27196</td>\n",
       "      <td>3.76506</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.89165</td>\n",
       "      <td>0.905337</td>\n",
       "      <td>0.074723</td>\n",
       "      <td>3.804390e-281</td>\n",
       "      <td>...</td>\n",
       "      <td>42.6210</td>\n",
       "      <td>2.54349</td>\n",
       "      <td>4.53236</td>\n",
       "      <td>6.76070</td>\n",
       "      <td>0.010528</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.98401</td>\n",
       "      <td>54.4958</td>\n",
       "      <td>12.08540</td>\n",
       "      <td>5.84061</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ALL-105</th>\n",
       "      <td>0.0</td>\n",
       "      <td>5.30719</td>\n",
       "      <td>1.062420</td>\n",
       "      <td>1.27740</td>\n",
       "      <td>2.48042</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.61281</td>\n",
       "      <td>0.979303</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.456910e-285</td>\n",
       "      <td>...</td>\n",
       "      <td>44.5727</td>\n",
       "      <td>1.99089</td>\n",
       "      <td>3.62848</td>\n",
       "      <td>6.12268</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.65392</td>\n",
       "      <td>13.3985</td>\n",
       "      <td>8.11452</td>\n",
       "      <td>7.62316</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ALL-106</th>\n",
       "      <td>0.0</td>\n",
       "      <td>9.07089</td>\n",
       "      <td>0.885766</td>\n",
       "      <td>1.01986</td>\n",
       "      <td>5.37849</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.64102</td>\n",
       "      <td>0.604450</td>\n",
       "      <td>0.091972</td>\n",
       "      <td>4.924820e-02</td>\n",
       "      <td>...</td>\n",
       "      <td>20.8279</td>\n",
       "      <td>1.80050</td>\n",
       "      <td>2.22317</td>\n",
       "      <td>10.31110</td>\n",
       "      <td>0.005779</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.07277</td>\n",
       "      <td>19.0703</td>\n",
       "      <td>14.91710</td>\n",
       "      <td>12.20690</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3 rows × 51968 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "gene_short_name  5S_rRNA      7SK  A-575C2.4     A1BG  A1BG-AS1  A1CF  \\\n",
       "ALL-102              0.0  3.58839   0.537883  4.27196   3.76506   0.0   \n",
       "ALL-105              0.0  5.30719   1.062420  1.27740   2.48042   0.0   \n",
       "ALL-106              0.0  9.07089   0.885766  1.01986   5.37849   0.0   \n",
       "\n",
       "gene_short_name      A2M   A2M-AS1     A2ML1      A2ML1-AS1    ...     \\\n",
       "ALL-102          5.89165  0.905337  0.074723  3.804390e-281    ...      \n",
       "ALL-105          7.61281  0.979303  0.000000  3.456910e-285    ...      \n",
       "ALL-106          7.64102  0.604450  0.091972   4.924820e-02    ...      \n",
       "\n",
       "gene_short_name    ZWINT     ZXDA     ZXDB      ZXDC    ZYG11A  ZYG11AP1  \\\n",
       "ALL-102          42.6210  2.54349  4.53236   6.76070  0.010528       0.0   \n",
       "ALL-105          44.5727  1.99089  3.62848   6.12268  0.000000       0.0   \n",
       "ALL-106          20.8279  1.80050  2.22317  10.31110  0.005779       0.0   \n",
       "\n",
       "gene_short_name   ZYG11B      ZYX     ZZEF1      ZZZ3  \n",
       "ALL-102          3.98401  54.4958  12.08540   5.84061  \n",
       "ALL-105          4.65392  13.3985   8.11452   7.62316  \n",
       "ALL-106          5.07277  19.0703  14.91710  12.20690  \n",
       "\n",
       "[3 rows x 51968 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load PDX gene expression data in RDA format\n",
    "file = os.path.join('data', 'raw', '2019-02-14-PPTC_FPKM_matrix_withModelID-244.rda')\n",
    "\n",
    "loadRDA(file)\n",
    "exprs_df = robjects.r['rna.mat'].set_index('gene_short_name').transpose()\n",
    "\n",
    "print(exprs_df.shape)\n",
    "exprs_df.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>gene_short_name</th>\n",
       "      <th>5S_rRNA</th>\n",
       "      <th>7SK</th>\n",
       "      <th>A-575C2.4</th>\n",
       "      <th>A1BG</th>\n",
       "      <th>A1BG-AS1</th>\n",
       "      <th>A1CF</th>\n",
       "      <th>A2M</th>\n",
       "      <th>A2M-AS1</th>\n",
       "      <th>A2ML1</th>\n",
       "      <th>A2ML1-AS1</th>\n",
       "      <th>...</th>\n",
       "      <th>ZWINT</th>\n",
       "      <th>ZXDA</th>\n",
       "      <th>ZXDB</th>\n",
       "      <th>ZXDC</th>\n",
       "      <th>ZYG11A</th>\n",
       "      <th>ZYG11AP1</th>\n",
       "      <th>ZYG11B</th>\n",
       "      <th>ZYX</th>\n",
       "      <th>ZZEF1</th>\n",
       "      <th>ZZZ3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ALL-102</th>\n",
       "      <td>-0.106346</td>\n",
       "      <td>-0.358926</td>\n",
       "      <td>0.162776</td>\n",
       "      <td>-0.095863</td>\n",
       "      <td>0.560052</td>\n",
       "      <td>-0.076358</td>\n",
       "      <td>-0.157051</td>\n",
       "      <td>0.377248</td>\n",
       "      <td>-0.167790</td>\n",
       "      <td>-0.283153</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.102367</td>\n",
       "      <td>0.979007</td>\n",
       "      <td>0.409628</td>\n",
       "      <td>-0.230326</td>\n",
       "      <td>-0.511862</td>\n",
       "      <td>-0.271235</td>\n",
       "      <td>-0.808946</td>\n",
       "      <td>0.849673</td>\n",
       "      <td>1.233418</td>\n",
       "      <td>-1.360337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ALL-105</th>\n",
       "      <td>-0.106346</td>\n",
       "      <td>-0.209068</td>\n",
       "      <td>1.380222</td>\n",
       "      <td>-0.885029</td>\n",
       "      <td>-0.230339</td>\n",
       "      <td>-0.076358</td>\n",
       "      <td>-0.133974</td>\n",
       "      <td>0.456380</td>\n",
       "      <td>-0.188862</td>\n",
       "      <td>-0.283153</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.022131</td>\n",
       "      <td>0.514752</td>\n",
       "      <td>0.056417</td>\n",
       "      <td>-0.458639</td>\n",
       "      <td>-0.526122</td>\n",
       "      <td>-0.271235</td>\n",
       "      <td>-0.693156</td>\n",
       "      <td>-0.599859</td>\n",
       "      <td>0.271131</td>\n",
       "      <td>-1.066703</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ALL-106</th>\n",
       "      <td>-0.106346</td>\n",
       "      <td>0.119079</td>\n",
       "      <td>0.970210</td>\n",
       "      <td>-0.952899</td>\n",
       "      <td>1.552735</td>\n",
       "      <td>-0.076358</td>\n",
       "      <td>-0.133595</td>\n",
       "      <td>0.055347</td>\n",
       "      <td>-0.162926</td>\n",
       "      <td>1.210141</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.998308</td>\n",
       "      <td>0.354800</td>\n",
       "      <td>-0.492739</td>\n",
       "      <td>1.040170</td>\n",
       "      <td>-0.518294</td>\n",
       "      <td>-0.271235</td>\n",
       "      <td>-0.620760</td>\n",
       "      <td>-0.399811</td>\n",
       "      <td>1.919641</td>\n",
       "      <td>-0.311638</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ALL-107</th>\n",
       "      <td>-0.106346</td>\n",
       "      <td>-0.309800</td>\n",
       "      <td>0.514608</td>\n",
       "      <td>-0.384559</td>\n",
       "      <td>0.821157</td>\n",
       "      <td>-0.076358</td>\n",
       "      <td>-0.209385</td>\n",
       "      <td>-0.545406</td>\n",
       "      <td>-0.155709</td>\n",
       "      <td>-0.283153</td>\n",
       "      <td>...</td>\n",
       "      <td>0.622859</td>\n",
       "      <td>0.612896</td>\n",
       "      <td>0.540251</td>\n",
       "      <td>-0.542007</td>\n",
       "      <td>-0.518498</td>\n",
       "      <td>-0.271235</td>\n",
       "      <td>-0.501401</td>\n",
       "      <td>-0.092437</td>\n",
       "      <td>1.377851</td>\n",
       "      <td>-0.262022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ALL-108</th>\n",
       "      <td>-0.106346</td>\n",
       "      <td>-0.013369</td>\n",
       "      <td>4.960214</td>\n",
       "      <td>-0.688568</td>\n",
       "      <td>2.886945</td>\n",
       "      <td>-0.076358</td>\n",
       "      <td>-0.217480</td>\n",
       "      <td>-0.465763</td>\n",
       "      <td>-0.182332</td>\n",
       "      <td>-0.283153</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.071698</td>\n",
       "      <td>0.503192</td>\n",
       "      <td>0.046245</td>\n",
       "      <td>0.643061</td>\n",
       "      <td>-0.520042</td>\n",
       "      <td>-0.271235</td>\n",
       "      <td>-0.479504</td>\n",
       "      <td>0.058586</td>\n",
       "      <td>1.986380</td>\n",
       "      <td>-0.086291</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 51968 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "gene_short_name   5S_rRNA       7SK  A-575C2.4      A1BG  A1BG-AS1      A1CF  \\\n",
       "ALL-102         -0.106346 -0.358926   0.162776 -0.095863  0.560052 -0.076358   \n",
       "ALL-105         -0.106346 -0.209068   1.380222 -0.885029 -0.230339 -0.076358   \n",
       "ALL-106         -0.106346  0.119079   0.970210 -0.952899  1.552735 -0.076358   \n",
       "ALL-107         -0.106346 -0.309800   0.514608 -0.384559  0.821157 -0.076358   \n",
       "ALL-108         -0.106346 -0.013369   4.960214 -0.688568  2.886945 -0.076358   \n",
       "\n",
       "gene_short_name       A2M   A2M-AS1     A2ML1  A2ML1-AS1    ...        ZWINT  \\\n",
       "ALL-102         -0.157051  0.377248 -0.167790  -0.283153    ...    -0.102367   \n",
       "ALL-105         -0.133974  0.456380 -0.188862  -0.283153    ...    -0.022131   \n",
       "ALL-106         -0.133595  0.055347 -0.162926   1.210141    ...    -0.998308   \n",
       "ALL-107         -0.209385 -0.545406 -0.155709  -0.283153    ...     0.622859   \n",
       "ALL-108         -0.217480 -0.465763 -0.182332  -0.283153    ...    -0.071698   \n",
       "\n",
       "gene_short_name      ZXDA      ZXDB      ZXDC    ZYG11A  ZYG11AP1    ZYG11B  \\\n",
       "ALL-102          0.979007  0.409628 -0.230326 -0.511862 -0.271235 -0.808946   \n",
       "ALL-105          0.514752  0.056417 -0.458639 -0.526122 -0.271235 -0.693156   \n",
       "ALL-106          0.354800 -0.492739  1.040170 -0.518294 -0.271235 -0.620760   \n",
       "ALL-107          0.612896  0.540251 -0.542007 -0.518498 -0.271235 -0.501401   \n",
       "ALL-108          0.503192  0.046245  0.643061 -0.520042 -0.271235 -0.479504   \n",
       "\n",
       "gene_short_name       ZYX     ZZEF1      ZZZ3  \n",
       "ALL-102          0.849673  1.233418 -1.360337  \n",
       "ALL-105         -0.599859  0.271131 -1.066703  \n",
       "ALL-106         -0.399811  1.919641 -0.311638  \n",
       "ALL-107         -0.092437  1.377851 -0.262022  \n",
       "ALL-108          0.058586  1.986380 -0.086291  \n",
       "\n",
       "[5 rows x 51968 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Transform the gene expression data (z-score by gene)\n",
    "scaled_fit = StandardScaler().fit(exprs_df)\n",
    "exprs_scaled_df = pd.DataFrame(scaled_fit.transform(exprs_df),\n",
    "                               index=exprs_df.index,\n",
    "                               columns=exprs_df.columns)\n",
    "exprs_scaled_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Shuffle input RNAseq matrix and apply classifiers\n",
    "exprs_shuffled_df = exprs_scaled_df.apply(shuffle_columns, axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Apply Ras Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(185, 3)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>weight</th>\n",
       "      <th>abs_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6702</th>\n",
       "      <td>PBX3</td>\n",
       "      <td>0.139761</td>\n",
       "      <td>0.139761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4592</th>\n",
       "      <td>SPRY2</td>\n",
       "      <td>0.123713</td>\n",
       "      <td>0.123713</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6708</th>\n",
       "      <td>C15orf52</td>\n",
       "      <td>0.106100</td>\n",
       "      <td>0.106100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2882</th>\n",
       "      <td>PPP1R3B</td>\n",
       "      <td>0.104700</td>\n",
       "      <td>0.104700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1520</th>\n",
       "      <td>MLPH</td>\n",
       "      <td>0.102069</td>\n",
       "      <td>0.102069</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       feature    weight   abs_val\n",
       "6702      PBX3  0.139761  0.139761\n",
       "4592     SPRY2  0.123713  0.123713\n",
       "6708  C15orf52  0.106100  0.106100\n",
       "2882   PPP1R3B  0.104700  0.104700\n",
       "1520      MLPH  0.102069  0.102069"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load RAS Classifier\n",
    "file = os.path.join('data', 'ras_classifier_coefficients.tsv')\n",
    "ras_coef_df = pd.read_table(file, index_col=0)\n",
    "ras_coef_df = ras_coef_df.query('abs_val > 0')\n",
    "\n",
    "print(ras_coef_df.shape)\n",
    "ras_coef_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Apply the Ras classifier to the input RNAseq matrix\n",
    "ras_scores_df, ras_common_genes_df, ras_missing_genes_df = (\n",
    "    apply_classifier(coef_df=ras_coef_df, rnaseq_df=exprs_scaled_df)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are a total of 173 out of 185 genes in common (93.51%) between the datasets\n"
     ]
    }
   ],
   "source": [
    "# Determine the extent of coefficient overlap\n",
    "print('There are a total of {} out of {} genes in common ({}%) between the datasets'\n",
    "      .format(ras_common_genes_df.shape[0],\n",
    "              ras_coef_df.shape[0],\n",
    "              round(ras_common_genes_df.shape[0] / ras_coef_df.shape[0] * 100, 2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>weight</th>\n",
       "      <th>abs_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6994</th>\n",
       "      <td>KIAA1539</td>\n",
       "      <td>0.069202</td>\n",
       "      <td>0.069202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5954</th>\n",
       "      <td>C12orf11</td>\n",
       "      <td>0.063514</td>\n",
       "      <td>0.063514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7023</th>\n",
       "      <td>ZNF322A</td>\n",
       "      <td>0.048071</td>\n",
       "      <td>0.048071</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8033</th>\n",
       "      <td>log10_mut</td>\n",
       "      <td>0.048028</td>\n",
       "      <td>0.048028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6042</th>\n",
       "      <td>C17orf63</td>\n",
       "      <td>0.035493</td>\n",
       "      <td>0.035493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4795</th>\n",
       "      <td>C17orf37</td>\n",
       "      <td>-0.034015</td>\n",
       "      <td>0.034015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1672</th>\n",
       "      <td>C9orf86</td>\n",
       "      <td>0.028749</td>\n",
       "      <td>0.028749</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4927</th>\n",
       "      <td>BRP44</td>\n",
       "      <td>-0.026352</td>\n",
       "      <td>0.026352</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>LOC96610</td>\n",
       "      <td>-0.020048</td>\n",
       "      <td>0.020048</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7281</th>\n",
       "      <td>FAM125B</td>\n",
       "      <td>0.018148</td>\n",
       "      <td>0.018148</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1617</th>\n",
       "      <td>C17orf28</td>\n",
       "      <td>0.010409</td>\n",
       "      <td>0.010409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7851</th>\n",
       "      <td>LOC25845</td>\n",
       "      <td>-0.008914</td>\n",
       "      <td>0.008914</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        feature    weight   abs_val\n",
       "6994   KIAA1539  0.069202  0.069202\n",
       "5954   C12orf11  0.063514  0.063514\n",
       "7023    ZNF322A  0.048071  0.048071\n",
       "8033  log10_mut  0.048028  0.048028\n",
       "6042   C17orf63  0.035493  0.035493\n",
       "4795   C17orf37 -0.034015  0.034015\n",
       "1672    C9orf86  0.028749  0.028749\n",
       "4927      BRP44 -0.026352  0.026352\n",
       "52     LOC96610 -0.020048  0.020048\n",
       "7281    FAM125B  0.018148  0.018148\n",
       "1617   C17orf28  0.010409  0.010409\n",
       "7851   LOC25845 -0.008914  0.008914"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Which Genes are Missing?\n",
    "ras_missing_genes_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAEltJREFUeJzt3X2MZXddx/H3B7ZAZbAtFibLUhmQ8lC7sdKxYIg6Iw+prfKQINJgbbG6SIQY2WgajFJFTH1AEiIKJTxUAgwVipQWJE3T6QoBdLatbEuDlLJCH2yttKUDCG75+se9zc6ss3vPzNw7d+bX9yuZzD3nnvs73/nuvZ89c+7vnklVIUna+h427gIkScNhoEtSIwx0SWqEgS5JjTDQJakRBrokNcJA10Naknck+cOO274vyZ+OuiZprbaNuwBpnKrqt4Y1VpICTqyqm4c1prQaHqFLUiMMdG1ZSV6V5BNLlm9OcsmS5W8kOSXJM5JcmeSbSb6c5OVLtll2GiXJ7ye5I8ntSX4jSSV56pLdHpfkiiT3J/lCkh/rP25P//5/S7KY5FdG95NLKzPQtZVdA/xMkocl2Q4cBTwXIMlTgAngK8CVwAeBxwNnAX+b5McPHSzJ6cDrgecDTwV+boV9ngX8MXAccDPwZoCq+tn+/T9RVRNV9eFh/ZBSVwa6tqyqugW4HziFXvh+GrgtyTP6y/8M/CKwv6reW1UHqupa4KPAy1YY8uXAe6vqxqr6Dr3gPtSlVfUvVXUA+EB/39Km4Jui2uquAWboHVFfA9xLL8x/ur/8JODZSe5d8phtwPtXGOsJwMKS5W+ssM1/Lrn9HXq/BUibgoGure4a4JeAJwN/Ri/QX0kv0P8GOBG4pqpe0GGsO4AnLlk+YbilSqPlKRdtddcAs8DRVXUrvdMspwM/AlwHXA48LcnZSY7qf/1UkmeuMNYlwKuSPDPJDwF/tMpa7gSesuafRFonA11bWlX9O7BIL8ipqm8BtwCfraoHqup+4IXAK4Db6Z0y+XPgkSuM9SngbcDV9N7w/Fz/ru91LOcC4OIk9y6dSSNtlPgHLqSV9Y/ibwAe2X8TVNrUPEKXlkjy0iSPSHIcvSP5Txjm2ioMdGm5VwP/BXwVeAB4zXjLkbrzlIskNcIjdElqxMB56EkeBeyhNytgG/CRqnpjkicDc8BjgWuBs6vq+0ca6/jjj6+pqal1F70Zffvb3+bRj370uMvYNOzHcvbjIHuxXJd+7N279+6qetygsbp8sOh7wM9X1WKSo4DPJPkUvWtevLWq5pK8AzgP+LsjDTQ1NcXCwsKRNtmy5ufnmZmZGXcZm4b9WM5+HGQvluvSjyT/0WWsgadcqmexv3hU/6uAnwc+0l9/MfCSLjuUJI1Gp3PoSR6e5HrgLnpXrvsqcO+S6Vy3AjtGU6IkqYtVzXJJcizwMXofiX5vVT21v/4E4JNVtXOFx+wCdgFMTk6eOjc3N4y6N53FxUUmJrxO04Psx3L24yB7sVyXfszOzu6tqulBY63q4lxVdW+SeeA5wLFJtvWP0p9I72PVKz3mIuAigOnp6Wr13JnnBZezH8vZj4PsxXLD7MfAUy5JHtc/MifJ0fQu/n8TvetdPHhN6XOAjw+lIknSmnQ5Qt9O74JDD6f3H8AlVXV5ki8Bc/0/33Ud8O4R1ilJGmBgoFfVF4GfXGH9LcBpoyhKkrR6flJUkhphoEtSI/wTdGra1PlXdNpu/4VnjrgSafQ8QpekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDViYKAnOSHJ1UluSnJjkt/pr78gyW1Jru9/nTH6ciVJh7OtwzYHgN1VdW2SxwB7k1zZv++tVfVXoytPktTVwECvqjuAO/q3709yE7Bj1IVJklYnVdV942QK2AOcDLweOBf4FrBA7yj+nhUeswvYBTA5OXnq3NzcemvelBYXF5mYmBh3GZvGof3Yd9t9nR63c8cxQ61jXPs9lM+Pg+zFcl36MTs7u7eqpgeN1TnQk0wA1wBvrqpLk0wCdwMFvAnYXlW/fqQxpqena2FhodP+tpr5+XlmZmbGXcamcWg/ps6/otPj9l945lDrGNd+D+Xz4yB7sVyXfiTpFOidZrkkOQr4KPCBqroUoKrurKoHquoHwLuA07qMJUkajS6zXAK8G7ipqv56yfrtSzZ7KXDD8MuTJHXVZZbLc4GzgX1Jru+vewNwVpJT6J1y2Q+8eiQVSpI66TLL5TNAVrjrk8MvR5K0Vn5SVJIa0eWUi9S8zTIbRloPj9AlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqE89C1qTgfXFo7j9AlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQMDPckJSa5OclOSG5P8Tn/9Y5NcmeQr/e/Hjb5cSdLhdDlCPwDsrqpnAs8BfjvJScD5wFVVdSJwVX9ZkjQmAwO9qu6oqmv7t+8HbgJ2AC8GLu5vdjHwklEVKUkaLFXVfeNkCtgDnAx8vaqOXXLfPVX1/067JNkF7AKYnJw8dW5ubp0lb06Li4tMTEyMu4xN49B+7LvtvqGOv3PHMZ22G9d+D+Xz4yB7sVyXfszOzu6tqulBY3UO9CQTwDXAm6vq0iT3dgn0paanp2thYaHT/raa+fl5ZmZmxl3GpnFoP6bOv2Ko4++/8MxO241rv4fy+XGQvViuSz+SdAr0TrNckhwFfBT4QFVd2l99Z5Lt/fu3A3d1GUuSNBpdZrkEeDdwU1X99ZK7LgPO6d8+B/j48MuTJHW1rcM2zwXOBvYlub6/7g3AhcAlSc4Dvg788mhKlCR1MTDQq+ozQA5z9/OGW44kaa38pKgkNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEV0+KSod1uEufrV75wHOHfKFsSQdmUfoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaMTDQk7wnyV1Jbliy7oIktyW5vv91xmjLlCQN0uUI/X3A6Susf2tVndL/+uRwy5IkrdbAQK+qPcA3N6AWSdI6pKoGb5RMAZdX1cn95QuAc4FvAQvA7qq65zCP3QXsApicnDx1bm5uCGVvPouLi0xMTIy7jKHYd9t96x5j8mi487tDKOYwdu44ptN2w/hZ1rLfQ7X0/Fgve7Fcl37Mzs7urarpQWOtNdAngbuBAt4EbK+qXx80zvT0dC0sLAzc31Y0Pz/PzMzMuMsYiqnzr1j3GLt3HuAt+7YNoZqV7b/wzE7bDeNnWct+D9XS82O97MVyXfqRpFOgr2mWS1XdWVUPVNUPgHcBp61lHEnS8Kwp0JNsX7L4UuCGw20rSdoYA38nTvIhYAY4PsmtwBuBmSSn0Dvlsh949QhrlCR1MDDQq+qsFVa/ewS1SJLWwU+KSlIjRjcNQWrQambNDHsmzlpn2OihwyN0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AinLWpLGvZFt6QWeIQuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGuG0RWlElk6t3L3zAOc61VIj5hG6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYMDPQk70lyV5Iblqx7bJIrk3yl//240ZYpSRqkyxH6+4DTD1l3PnBVVZ0IXNVfliSN0cBAr6o9wDcPWf1i4OL+7YuBlwy5LknSKqWqBm+UTAGXV9XJ/eV7q+rYJfffU1UrnnZJsgvYBTA5OXnq3NzcEMrefBYXF5mYmBh3GUe077b7Nmxfk0fDnd/dsN1tesPox84dxwynmDHbCq+VjdSlH7Ozs3uranrQWCO/fG5VXQRcBDA9PV0zMzOj3uVYzM/Ps9l/to28fOvunQd4yz6vzvygYfRj/ytnhlPMmG2F18pGGmY/1jrL5c4k2wH63+8aSjWSpDVba6BfBpzTv30O8PHhlCNJWqsu0xY/BHwOeHqSW5OcB1wIvCDJV4AX9JclSWM08KReVZ11mLueN+RaJEnr4CdFJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWrEtnEXIGm4ps6/otN2+y88c8SVaKN5hC5JjTDQJakRBrokNWJd59CT7AfuBx4ADlTV9DCKkiSt3jDeFJ2tqruHMI4kaR085SJJjUhVrf3BydeAe4AC3llVF62wzS5gF8Dk5OSpc3Nza97fZra4uMjExESnbffddt9Q971zxzFj2e+RTB4Nd353w3a36Q2jH8P+dx7F86bLmKt5rTwUdOnH7Ozs3i6ntNcb6E+oqtuTPB64EnhdVe053PbT09O1sLCw5v1tZvPz88zMzHTatus84a66zice9n6PZPfOA7xlnx9zeNAw+jHsf+dRPG+6jLma18pDQZd+JOkU6Os65VJVt/e/3wV8DDhtPeNJktZuzYGe5NFJHvPgbeCFwA3DKkyStDrr+R1wEvhYkgfH+WBV/dNQqpIkrdqaA72qbgF+Yoi1SJLWwWmLktQIpyE0YCNnr2h8hv3v7POmPR6hS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEY4bbGDLtO7du88wMzoS5G0iQz7wmXr5RG6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmN2DLz0If9h28lPfS0niMeoUtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGbJlpi121Pi1J2sy6Xmr63FVcdrYLX889HqFLUiMMdElqhIEuSY1YV6AnOT3Jl5PcnOT8YRUlSVq9NQd6kocDbwd+ATgJOCvJScMqTJK0Ous5Qj8NuLmqbqmq7wNzwIuHU5YkabVSVWt7YPIy4PSq+o3+8tnAs6vqtYdstwvY1V98OvDltZe7qR0P3D3uIjYR+7Gc/TjIXizXpR9PqqrHDRpoPfPQs8K6//e/Q1VdBFy0jv1sCUkWqmp63HVsFvZjOftxkL1Ybpj9WM8pl1uBE5YsPxG4fX3lSJLWaj2B/q/AiUmenOQRwCuAy4ZTliRptdZ8yqWqDiR5LfBp4OHAe6rqxqFVtvU0f1pplezHcvbjIHux3ND6seY3RSVJm4ufFJWkRhjoktQIA32VBl3uIMnrk3wpyReTXJXkSeOoc6N0vfxDkpclqSTNTlfr0oskL+8/P25M8sGNrnEjdXit/GiSq5Nc13+9nDGOOjdCkvckuSvJDYe5P0ne1u/VF5M8a007qiq/On7Re/P3q8BTgEcA/wacdMg2s8AP9W+/BvjwuOseZz/62z0G2AN8Hpged91jfG6cCFwHHNdffvy46x5zPy4CXtO/fRKwf9x1j7AfPws8C7jhMPefAXyK3ud7ngN8YS378Qh9dQZe7qCqrq6q7/QXP09vfn6rul7+4U3AXwD/s5HFbbAuvfhN4O1VdQ9AVd21wTVupC79KOCH+7ePoeHPsVTVHuCbR9jkxcDfV8/ngWOTbF/tfgz01dkBfGPJ8q39dYdzHr3/dVs1sB9JfhI4oaou38jCxqDLc+NpwNOSfDbJ55OcvmHVbbwu/bgA+NUktwKfBF63MaVtSqvNlhU19yfoRqzT5Q4AkvwqMA383EgrGq8j9iPJw4C3AuduVEFj1OW5sY3eaZcZer+5/XOSk6vq3hHXNg5d+nEW8L6qekuSnwbe3+/HD0Zf3qbTOVuOxCP01el0uYMkzwf+AHhRVX1vg2obh0H9eAxwMjCfZD+9c4OXNfrGaJfnxq3Ax6vqf6vqa/QuVHfiBtW30br04zzgEoCq+hzwKHoXqnooGsqlVAz01Rl4uYP+KYZ30gvzls+RwoB+VNV9VXV8VU1V1RS99xReVFUL4yl3pLpcCuMf6b1pTpLj6Z2CuWVDq9w4XfrxdeB5AEmeSS/Q/2tDq9w8LgN+rT/b5TnAfVV1x2oH8ZTLKtRhLneQ5E+Ahaq6DPhLYAL4hyQAX6+qF42t6BHq2I+HhI69+DTwwiRfAh4Afq+q/nt8VY9Ox37sBt6V5HfpnV44t/pTPlqT5EP0TrUd33/P4I3AUQBV9Q567yGcAdwMfAd41Zr202j/JOkhx1MuktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ14v8A4I1hxuD8f7cAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Distribution of predictions of the Ras Classifier applied to input data\n",
    "ras_scores_df.T.hist(bins=30);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Apply Ras Classifier to Shuffled Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Apply the Ras classifier to the input RNAseq matrix \n",
    "ras_shuffle_scores_df, ras_shuffle_common_genes_df, ras_shuffle_missing_genes_df = (\n",
    "    apply_classifier(coef_df=ras_coef_df, rnaseq_df=exprs_shuffled_df)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Apply TP53 Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(319, 3)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>weight</th>\n",
       "      <th>abs_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6922</th>\n",
       "      <td>DDB2</td>\n",
       "      <td>-0.212359</td>\n",
       "      <td>0.212359</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6472</th>\n",
       "      <td>AEN</td>\n",
       "      <td>-0.195592</td>\n",
       "      <td>0.195592</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3464</th>\n",
       "      <td>RPS27L</td>\n",
       "      <td>-0.181545</td>\n",
       "      <td>0.181545</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2341</th>\n",
       "      <td>MDM2</td>\n",
       "      <td>-0.145332</td>\n",
       "      <td>0.145332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4346</th>\n",
       "      <td>BAX</td>\n",
       "      <td>-0.125992</td>\n",
       "      <td>0.125992</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     feature    weight   abs_val\n",
       "6922    DDB2 -0.212359  0.212359\n",
       "6472     AEN -0.195592  0.195592\n",
       "3464  RPS27L -0.181545  0.181545\n",
       "2341    MDM2 -0.145332  0.145332\n",
       "4346     BAX -0.125992  0.125992"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load RAS Classifier\n",
    "file = os.path.join('data', 'tp53_classifier_coefficients.tsv')\n",
    "tp53_coef_df = pd.read_table(file, index_col=0)\n",
    "tp53_coef_df = tp53_coef_df.query('abs_val > 0')\n",
    "\n",
    "print(tp53_coef_df.shape)\n",
    "tp53_coef_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Apply the TP53 classifier to the input RNAseq matrix\n",
    "tp53_scores_df, tp53_common_genes_df, tp53_missing_genes_df = (\n",
    "    apply_classifier(coef_df=tp53_coef_df, rnaseq_df=exprs_scaled_df)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are a total of 301 out of 319 genes in common (94.36%) between the datasets\n"
     ]
    }
   ],
   "source": [
    "# Determine the extent of coefficient overlap\n",
    "print('There are a total of {} out of {} genes in common ({}%) between the datasets'\n",
    "      .format(tp53_common_genes_df.shape[0],\n",
    "              tp53_coef_df.shape[0],\n",
    "              round(tp53_common_genes_df.shape[0] / tp53_coef_df.shape[0] * 100, 2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>weight</th>\n",
       "      <th>abs_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4274</th>\n",
       "      <td>C19orf2</td>\n",
       "      <td>0.044708</td>\n",
       "      <td>0.044708</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7355</th>\n",
       "      <td>C6orf192</td>\n",
       "      <td>-0.040536</td>\n",
       "      <td>0.040536</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7856</th>\n",
       "      <td>C17orf68</td>\n",
       "      <td>-0.033953</td>\n",
       "      <td>0.033953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7603</th>\n",
       "      <td>KIAA1804</td>\n",
       "      <td>0.031246</td>\n",
       "      <td>0.031246</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6210</th>\n",
       "      <td>KIAA0415</td>\n",
       "      <td>0.026226</td>\n",
       "      <td>0.026226</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4984</th>\n",
       "      <td>C19orf50</td>\n",
       "      <td>0.026029</td>\n",
       "      <td>0.026029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7669</th>\n",
       "      <td>FTHL3</td>\n",
       "      <td>0.022275</td>\n",
       "      <td>0.022275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2958</th>\n",
       "      <td>C20orf43</td>\n",
       "      <td>0.022058</td>\n",
       "      <td>0.022058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3682</th>\n",
       "      <td>KIAA0090</td>\n",
       "      <td>0.019398</td>\n",
       "      <td>0.019398</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6322</th>\n",
       "      <td>KIAA1632</td>\n",
       "      <td>-0.017291</td>\n",
       "      <td>0.017291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7602</th>\n",
       "      <td>C1orf57</td>\n",
       "      <td>-0.015932</td>\n",
       "      <td>0.015932</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3252</th>\n",
       "      <td>C1orf93</td>\n",
       "      <td>0.013086</td>\n",
       "      <td>0.013086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5260</th>\n",
       "      <td>CHCHD8</td>\n",
       "      <td>0.011840</td>\n",
       "      <td>0.011840</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4531</th>\n",
       "      <td>LGTN</td>\n",
       "      <td>-0.010107</td>\n",
       "      <td>0.010107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7110</th>\n",
       "      <td>LOC147727</td>\n",
       "      <td>0.007801</td>\n",
       "      <td>0.007801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8003</th>\n",
       "      <td>CESC</td>\n",
       "      <td>-0.006096</td>\n",
       "      <td>0.006096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1564</th>\n",
       "      <td>PTPRK</td>\n",
       "      <td>0.004631</td>\n",
       "      <td>0.004631</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3870</th>\n",
       "      <td>LPPR2</td>\n",
       "      <td>-0.002444</td>\n",
       "      <td>0.002444</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        feature    weight   abs_val\n",
       "4274    C19orf2  0.044708  0.044708\n",
       "7355   C6orf192 -0.040536  0.040536\n",
       "7856   C17orf68 -0.033953  0.033953\n",
       "7603   KIAA1804  0.031246  0.031246\n",
       "6210   KIAA0415  0.026226  0.026226\n",
       "4984   C19orf50  0.026029  0.026029\n",
       "7669      FTHL3  0.022275  0.022275\n",
       "2958   C20orf43  0.022058  0.022058\n",
       "3682   KIAA0090  0.019398  0.019398\n",
       "6322   KIAA1632 -0.017291  0.017291\n",
       "7602    C1orf57 -0.015932  0.015932\n",
       "3252    C1orf93  0.013086  0.013086\n",
       "5260     CHCHD8  0.011840  0.011840\n",
       "4531       LGTN -0.010107  0.010107\n",
       "7110  LOC147727  0.007801  0.007801\n",
       "8003       CESC -0.006096  0.006096\n",
       "1564      PTPRK  0.004631  0.004631\n",
       "3870      LPPR2 -0.002444  0.002444"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Which Genes are Missing?\n",
    "tp53_missing_genes_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAE8tJREFUeJzt3XuQZGV5x/HvoysKDq5EdMosxBFFFHfUhPYWK9ojYm2EQJKiFAossNBJSGmsuMZoWUZzMSExG8uoKbMVEUqRAZEIYoxSSoNagM5ycbl4QdzorsqqK6uDKK4++aNbZ5zM0Ke7T0/PvPv9VE3Rp/vtc55+9syPM6f7vB2ZiSRp7XvAqAuQJNXDQJekQhjoklQIA12SCmGgS1IhDHRJKoSBrv1aRLwnIt5Ucex5EfH3w65J6te6URcgjVJm/mld64qIBI7MzDvqWqfUC4/QJakQBrrWrIh4WUR8dMHyHRFx8YLlb0bE0yLiiRFxZUTsiYgvR8SLF4z5tdMoEfG6iPh2RHwrIl4eERkRj1+w2UMi4mMR8aOIuD4iHtd53jWdx2+OiLmIeMnwXrm0NANda9nVwO9FxAMi4tHAg4DnAETEEcAY8FXgSuCDwKOAU4F/j4gnL15ZRGwCXgO8AHg88Lwltnkq8DfAIcAdwFsBMvO5ncefmpljmXlRXS9SqspA15qVmXcCPwKeRjt8PwHsiogndpY/A5wA7MjM92Xmvsy8AfgwcPISq3wx8L7MvDUzf0w7uBe7NDM/n5n7gAs625ZWBd8U1Vp3NdCkfUR9NXA37TB/dmf5McAzI+LuBc9ZB7x/iXX9JjC7YPmbS4z5zoLbP6b9V4C0KhjoWuuuBv4AeCzwD7QD/TTagf4u4Ejg6sw8rsK6vg0ctmD58HpLlYbLUy5a664GpoADM3Mn7dMsm4BHADcCVwBPiIiXRsSDOj9Pj4gnLbGui4GXRcSTIuIg4K97rOUu4Ii+X4k0IANda1pmfgWYox3kZOYPgTuBz2XmzzPzR8ALgVOAb9E+ZfJPwIOXWNfHgX8DrqL9hue1nYd+WrGctwDnR8TdCz9JI62U8AsupKV1juJvAR7ceRNUWtU8QpcWiIg/iogDIuIQ2kfyHzXMtVYY6NKv+xPgu8DXgJ8DZ4+2HKk6T7lIUiE8QpekQqzo59APPfTQnJiY4J577uGhD33oSm561bIX8+xFm32YZy/atm3b9r3MfGS3cSsa6BMTE8zOztJqtWg2myu56VXLXsyzF232YZ69aIuI/60yzlMuklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRBdAz0izo2I3RFxy6L7X9X5fsZbI+Kfh1eiJKmKKkfo59GeX/pXImIKOAl4SmY+GfiX+kuTJPWia6Bn5jXAnkV3nw2ck5k/7YzZPYTaJEk9qDQ5V0RMAFdk5sbO8k3AZbSP3H8CvDYzv7DMc6eBaYDx8fFjZmZmmJubY2zMr2IE7MUCc3NzfH3vzyuNndywfsjVjI77xDx70TY1NbUtMxvdxvV76f864BDgWcDTgYsj4ohc4v8OmbkV2ArQaDSy2Wx6Oe8C9mJeq9Viy2fvqTR2x2nN4RYzQu4T8+xFb/r9lMtO4NJs+zzwC+DQ+sqSJPWq30D/CPB8gIh4AnAA8L26ipIk9a7rKZeIuBBoAodGxE7gzcC5wLmdjzLeB5yx1OkWSdLK6RromXnqMg+dXnMtkqQBeKWoJBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQXQM9Is6NiN2dbyda/NhrIyIjwu8TlaQRq3KEfh6wafGdEXE4cBzwjZprkiT1oWugZ+Y1wJ4lHno78DrA7xKVpFWgr3PoEXEisCszb665HklSnyKz+wF2REwAV2Tmxog4CLgKeGFm7o2IHUAjM7+3zHOngWmA8fHxY2ZmZpibm2NsbKyml7C22Yt5c3NzfH3vzyuNndywfsjVjI77xDx70TY1NbUtMxvdxq3rY92PAx4L3BwRAIcBN0TEMzLzO4sHZ+ZWYCtAo9HIZrNJq9Wi2Wz2seny2It5rVaLLZ+9p9LYHac1h1vMCLlPzLMXvek50DNzO/CoXy53O0KXJK2MKh9bvBC4FjgqInZGxFnDL0uS1KuuR+iZeWqXxydqq0aS1DevFJWkQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVop+5XFS4idd/rPLYHeccP8RKVk4vr7mqUnqjtcMjdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhqnwF3bkRsTsibllw39si4ksR8cWI+K+IePhwy5QkdVPlCP08YNOi+64ENmbmU4CvAG+ouS5JUo+6BnpmXgPsWXTfJzNzX2fxOuCwIdQmSepBZGb3QRETwBWZuXGJxz4KXJSZH1jmudPANMD4+PgxMzMzzM3NMTY2NkjdxViNvdi+a2/lsZMb1te2zvED4a57V367w1K1xsVW4z4xKvaibWpqaltmNrqNGyjQI+KNQAP446ywokajkbOzs7RaLZrNZtft7g9WYy+GMTlXlXVuntzHlu3V5ourc7vD0u/kXKtxnxgVe9EWEZUCve/ZFiPiDOAE4NgqYS5JGq6+Aj0iNgF/BTwvM39cb0mSpH5U+djihcC1wFERsTMizgLeBRwMXBkRN0XEe4ZcpySpi65H6Jl56hJ3v3cItUiSBuCVopJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFaLvS/+1elSdr6TfuUU0XIv//TZP7uPMJf5N6/73c78pj0foklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUiCpfQXduROyOiFsW3PcbEXFlRHy1899DhlumJKmbKkfo5wGbFt33euBTmXkk8KnOsiRphLoGemZeA+xZdPdJwPmd2+cDf1hzXZKkHkVmdh8UMQFckZkbO8t3Z+bDFzz+g8xc8rRLREwD0wDj4+PHzMzMMDc3x9jYWA3lry7bd+2tNG5yw/pf3a6jF/1st4711b3O8QPhrnsrb7pYy/Whaq+rqnu/GYZSs6JXU1NT2zKz0W3c0AN9oUajkbOzs7RaLZrNZtftrjX9zF5XRy/qnjWv6vrqXufmyX1s2e4EoMv1YX+cbbHUrOhVRFQK9H4/5XJXRDy6s6FHA7v7XI8kqSb9BvrlwBmd22cAl9VTjiSpX1U+tnghcC1wVETsjIizgHOA4yLiq8BxnWVJ0gh1PWGZmacu89CxNdciSRqAV4pKUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIJ87Yj/QyR8so1yn90vZdezmzwj42yvlmVhOP0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRADBXpE/EVE3BoRt0TEhRHxkLoKkyT1pu9Aj4gNwJ8DjczcCDwQOKWuwiRJvRn0lMs64MCIWAccBHxr8JIkSf2IzOz/yRGvBt4K3At8MjNPW2LMNDANMD4+fszMzAxzc3OMjY31vd3VavuuvZXGTW5Y/6vbdfSi6nZXu/ED4a57R13F6C3Xh4X7zf2pe3+out1h2L1n70j2iVG+5qVMTU1ty8xGt3F9B3pEHAJ8GHgJcDfwIeCSzPzAcs9pNBo5OztLq9Wi2Wz2td3VrOrMgwtnhqujF6XMeLh5ch9btjsB6HJ9qDqjYN37wyhnMnznBZeNZJ9YbbM3RkSlQB/klMsLgK9n5ncz82fApcDvDrA+SdIABgn0bwDPioiDIiKAY4Hb6ylLktSrvgM9M68HLgFuALZ31rW1prokST0a6ORUZr4ZeHNNtUiSBuCVopJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklSIgQI9Ih4eEZdExJci4vaIeHZdhUmSejPQV9AB7wD+JzNPjogDgINqqEmS1Ie+Az0iHgY8FzgTIDPvA+6rpyxJUq8iM/t7YsTTgK3AbcBTgW3AqzPznkXjpoFpgPHx8WNmZmaYm5tjbGxsoMIHtX3X3spjJzesr3WdC9dXRy96eS2r2fiBcNe9o65i9JbrQ937YVVVtzsMu/fsHck+McrXvJSpqaltmdnoNm6QQG8A1wHPyczrI+IdwA8z803LPafRaOTs7CytVotms9nXdusy8fqPVR6745zja13nwvXV0YteXstqtnlyH1u2D3oWcO1brg9174dVVd3uMLzzgstGsk+M8jUvJSIqBfogb4ruBHZm5vWd5UuA3xlgfZKkAfQd6Jn5HeCbEXFU565jaZ9+kSSNwKB/y7wKuKDzCZc7gZcNXpIkqR8DBXpm3gR0Pa8jSRo+rxSVpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQxU2csRbmNVlY4+bJfZy5TM2rbT4Jjdao9u1hzHu02g2j1yvRG4/QJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAMHekQ8MCJujIgr6ihIktSfOo7QXw3cXsN6JEkDGCjQI+Iw4HjgP+spR5LUr8jM/p8ccQnwj8DBwGsz84QlxkwD0wDj4+PHzMzMMDc3x9jYWN/bvT/bd+2tfZ2TG9YPbdvjB8Jd9678dlej++vF/mR/6EPVfXv3nr3F9KLqa17K1NTUtsxsdBvX92yLEXECsDszt0VEc7lxmbkV2ArQaDSy2WzSarVoNpd9ykCWm7lwEDtOaw5t25sn97Fl+9L/DMPc7mp0f73Yn+wPfai6b7/zgsuK6UXV1zyIQU65PAc4MSJ2ADPA8yPiA7VUJUnqWd+BnplvyMzDMnMCOAX4dGaeXltlkqSe+Dl0SSpELSenMrMFtOpYlySpPx6hS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUiDUzScJEIfOV9GJ/fM3aP1TdtzdPDrmQwniELkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5Jheg70CPi8Ii4KiJuj4hbI+LVdRYmSerNIJf+7wM2Z+YNEXEwsC0irszM22qqTZLUg76P0DPz25l5Q+f2j4DbgQ11FSZJ6k1k5uAriZgArgE2ZuYPFz02DUwDjI+PHzMzM8Pc3BxjY2M9bWP7rr0D17kajR8Id9076ipWB3vRZh/mldSLyQ3r+37u1NTUtsxsdBs3cKBHxBhwNfDWzLz0/sY2Go2cnZ2l1WrRbDZ72k6pMw9untzHlu1rZtLLobIXbfZhXkm92HHO8X0/NyIqBfpAn3KJiAcBHwYu6BbmkqThGuRTLgG8F7g9M/+1vpIkSf0Y5Aj9OcBLgedHxE2dnxfVVJckqUd9n5zKzM8CUWMtkqQBeKWoJBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFWLQL4neFBFfjog7IuL1dRUlSerdIF8S/UDg3cDvA0cDp0bE0XUVJknqzSBH6M8A7sjMOzPzPmAGOKmesiRJvYrM7O+JEScDmzLz5Z3llwLPzMxXLho3DUx3Fo8CvgwcCnyv36ILYy/m2Ys2+zDPXrQ9JjMf2W3QugE2EEvc9//+75CZW4Gtv/bEiNnMbAyw7WLYi3n2os0+zLMXvRnklMtO4PAFy4cB3xqsHElSvwYJ9C8AR0bEYyPiAOAU4PJ6ypIk9arvUy6ZuS8iXgl8AnggcG5m3lrx6Vu7D9lv2It59qLNPsyzFz3o+01RSdLq4pWiklQIA12SCjHUQO82NUBEPDgiLuo8fn1ETAyznlGq0IvXRMRtEfHFiPhURDxmFHUOW9XpIiLi5IjIiCj2I2tVehERL+7sF7dGxAdXusaVUuH347ci4qqIuLHzO/KiUdS56mXmUH5ov1H6NeAI4ADgZuDoRWP+DHhP5/YpwEXDqmeUPxV7MQUc1Ll9dom9qNKHzriDgWuA64DGqOse4T5xJHAjcEhn+VGjrnuEvdgKnN25fTSwY9R1r8afYR6hV5ka4CTg/M7tS4BjI2KpC5bWuq69yMyrMvPHncXraH+uvzRVp4v4O+CfgZ+sZHErrEovXgG8OzN/AJCZu1e4xpVSpRcJPKxzez1e87KkYQb6BuCbC5Z3du5bckxm7gP2Ao8YYk2jUqUXC50FfHyoFY1G1z5ExG8Dh2fmFStZ2AhU2SeeADwhIj4XEddFxKYVq25lVenFW4DTI2In8N/Aq1amtLVlkEv/u6kyNUCl6QMKUPl1RsTpQAN43lArGo377UNEPAB4O3DmShU0QlX2iXW0T7s0af/F9pmI2JiZdw+5tpVWpRenAudl5paIeDbw/k4vfjH88taOYR6hV5ka4FdjImId7T+l9gyxplGpNE1CRLwAeCNwYmb+dIVqW0nd+nAwsBFoRcQO4FnA5YW+MVr19+OyzPxZZn6d9sR2R65QfSupSi/OAi4GyMxrgYfQnrhLCwwz0KtMDXA5cEbn9snAp7Pzrkdhuvaic6rhP2iHeannSu+3D5m5NzMPzcyJzJyg/V7CiZk5O5pyh6rK78dHaL9ZTkQcSvsUzJ0rWuXKqNKLbwDHAkTEk2gH+ndXtMo1YGiB3jkn/supAW4HLs7MWyPibyPixM6w9wKPiIg7gNcARX7rUcVevA0YAz4UETdFRHHz4lTsw36hYi8+AXw/Im4DrgL+MjO/P5qKh6diLzYDr4iIm4ELgTMLPfgbiJf+S1IhvFJUkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RC/B/WuhAGvK0GdgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Distribution of predictions of the TP53 Classifier applied to input data\n",
    "tp53_scores_df.T.hist(bins=30);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Apply TP53 Classifier to Shuffled Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Apply the Ras classifier to the input RNAseq matrix \n",
    "tp53_shuffle_scores_df, tp53_shuffle_common_genes_df, tp53_shuffle_missing_genes_df = (\n",
    "    apply_classifier(coef_df=tp53_coef_df, rnaseq_df=exprs_shuffled_df)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Apply NF1 Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(55, 3)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>weight</th>\n",
       "      <th>abs_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3815</th>\n",
       "      <td>SPRED2</td>\n",
       "      <td>0.165174</td>\n",
       "      <td>0.165174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7151</th>\n",
       "      <td>GOSR1</td>\n",
       "      <td>-0.155423</td>\n",
       "      <td>0.155423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7894</th>\n",
       "      <td>KCNS3</td>\n",
       "      <td>0.067975</td>\n",
       "      <td>0.067975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1483</th>\n",
       "      <td>WDR45L</td>\n",
       "      <td>0.067480</td>\n",
       "      <td>0.067480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>856</th>\n",
       "      <td>FCGRT</td>\n",
       "      <td>-0.062596</td>\n",
       "      <td>0.062596</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     feature    weight   abs_val\n",
       "3815  SPRED2  0.165174  0.165174\n",
       "7151   GOSR1 -0.155423  0.155423\n",
       "7894   KCNS3  0.067975  0.067975\n",
       "1483  WDR45L  0.067480  0.067480\n",
       "856    FCGRT -0.062596  0.062596"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load NF1 Classifier\n",
    "file = os.path.join('data', 'nf1_classifier_coefficients.tsv')\n",
    "nf1_coef_df = pd.read_table(file, index_col=0)\n",
    "nf1_coef_df = nf1_coef_df.query('abs_val > 0')\n",
    "\n",
    "print(nf1_coef_df.shape)\n",
    "nf1_coef_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Apply the NF1 classifier to the input RNAseq matrix\n",
    "nf1_scores_df, nf1_common_genes_df, nf1_missing_genes_df = (\n",
    "    apply_classifier(coef_df=nf1_coef_df, rnaseq_df=exprs_scaled_df)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are a total of 53 out of 55 genes in common (96.36%) between the datasets\n"
     ]
    }
   ],
   "source": [
    "# Determine the extent of coefficient overlap\n",
    "print('There are a total of {} out of {} genes in common ({}%) between the datasets'\n",
    "      .format(nf1_common_genes_df.shape[0],\n",
    "              nf1_coef_df.shape[0],\n",
    "              round(nf1_common_genes_df.shape[0] / nf1_coef_df.shape[0] * 100, 2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>weight</th>\n",
       "      <th>abs_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1483</th>\n",
       "      <td>WDR45L</td>\n",
       "      <td>0.067480</td>\n",
       "      <td>0.067480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1819</th>\n",
       "      <td>SFRS9</td>\n",
       "      <td>0.003948</td>\n",
       "      <td>0.003948</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     feature    weight   abs_val\n",
       "1483  WDR45L  0.067480  0.067480\n",
       "1819   SFRS9  0.003948  0.003948"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Which Genes are Missing?\n",
    "nf1_missing_genes_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEICAYAAABVv+9nAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAExhJREFUeJzt3X2MZXV9x/H3lyelDAK6OlkX6qKuCLIVwxRpTOuMT9lCVajUsrGEVeyqUWviNi3R1uIDLTZF0kabdhVlY9SRKgYE1FDKLNX40FlZXHCjPLjVXegilQVGqe3gt3/MMRmHu9xz7z333rm/fb+SyZ6H3z33++Xc++HMueeeicxEklSeg4ZdgCSpPwx4SSqUAS9JhTLgJalQBrwkFcqAl6RCGfA6oEXEP0XEX9Yce0VEfKDfNUlNOWTYBUjDlJlvbmpbEZHAmsy8s6ltSr3wCF6SCmXAa2RFxOsj4ouL5u+MiCsXzf8oIk6JiOdGxA0R8ZOI+F5EvHbRmF857RIRfxYR90bEPRHxxojIiHj2oqc9JiKui4iHI+KbEfGs6nE3V+tvjYi5iPjD/nUu1WPAa5RtBX47Ig6KiJXAocCLACLimcAYcAdwA/Bp4GnAeuAfI+J5SzcWEeuAdwIvA54NvLjFc64H3gscA9wJXAyQmb9TrX9+Zo5l5mebalLqlgGvkZWZdwMPA6ewEMZfAfZExHOr+X8Hfg/YlZmfyMz5zPw28HngnBabfC3wicy8PTN/xkKQL3VVZn4rM+eBT1XPLS1LfsiqUbcVmGThiHsrsI+FcP+tav4ZwAsjYt+ixxwCfLLFtp4OzC6a/1GLMf+1aPpnLPyWIC1LBrxG3VbglcDxwF+zEPCvYyHgPwysAbZm5strbOte4NhF88c1W6o0WJ6i0ajbCkwBh2fmbhZOy6wDngLcAlwLPCcizouIQ6uf34yIE1ts60rg9RFxYkT8GvCeDmvZCzyz606khhnwGmmZ+X1gjoVgJzMfAu4GvpaZj2bmw8ArgHOBe1g4xfJB4AkttvUl4B+Am1j4APXr1aqf1yznImBLROxbfKWONCzhH/yQWquO8m8DnlB9qCqNFI/gpUUi4uyIOCwijmHhSP+LhrtGlQEv/ao3AT8G7gIeBd4y3HKk7nmKRpIK5RG8JBVqoNfBr1ixIlevXt1y3U9/+lOOOOKIQZYzMKX2VmpfUG5vpfYFZfe2bdu2+zPzqZ0+bqABv3r1amZnZ1uum5mZYXJycpDlDEypvZXaF5TbW6l9Qdm9RcR/dvM4T9FIUqEMeEkqlAEvSYUy4CWpUAa8JBXKgJekQhnwklQoA16SCmXAS1Kh/JN9BVh94XW1xu265Mw+VyJpOfEIXpIKZcBLUqEMeEkqlAEvSYUy4CWpUAa8JBXKgJekQhnwklSotgEfEU+MiG9FxK0RcXtEvLdafnxEfDMi7oiIz0bEYf0vV5JUV50j+J8DL8nM5wOnAOsi4nTgg8BlmbkGeAC4oH9lSpI61Tbgc8FcNXto9ZPAS4DPVcu3AGf1pUJJUldqnYOPiIMjYjtwH3ADcBewLzPnqyG7gVX9KVGS1I3IzPqDI44GvgC8B/hEZj67Wn4ccH1mrm3xmI3ARoDx8fFTp6enW257bm6OsbGxjhsYBf3ubceeB2uNW7vqqEaf1302ekrtC8rubWpqaltmTnT6uI7uJpmZ+yJiBjgdODoiDqmO4o8F7tnPYzYDmwEmJiZycnKy5bZnZmbY37pR1+/eNtS9m+Trmq3BfTZ6Su0Lyu6tW3WuonlqdeRORBwOvAzYCdwEnFMNOx+4ul9FSpI6V+cIfiWwJSIOZuF/CFdm5rUR8V1gOiI+ANwCXN7HOiVJHWob8Jn5HeAFLZbfDZzWj6IkSb3zm6ySVCj/ZN8BxD/tJx1YPIKXpEIZ8JJUKANekgplwEtSoQx4SSqUAS9JhTLgJalQBrwkFcqAl6RCGfCSVCgDXpIKZcBLUqEMeEkqlAEvSYUy4CWpUAa8JBXKgJekQhnwklQoA16SCmXAS1KhDHhJKpQBL0mFahvwEXFcRNwUETsj4vaIeEe1/KKI2BMR26ufM/pfriSprkNqjJkHNmXmtyPiSGBbRNxQrbssM/+uf+VJkrrVNuAz817g3mr64YjYCazqd2GSpN5EZtYfHLEauBk4GXgnsAF4CJhl4Sj/gRaP2QhsBBgfHz91enq65bbn5uYYGxvrqPhR0e/edux5sNHtrV11VK1x7rPRU2pfUHZvU1NT2zJzotPH1Q74iBgDtgIXZ+ZVETEO3A8k8H5gZWa+4fG2MTExkbOzsy3XzczMMDk52UHpo6Pfva2+8LpGt7frkjNrjXOfjZ5S+4Kye4uIrgK+1lU0EXEo8HngU5l5FUBm7s3MRzPzF8BHgdM6fXJJUv/UuYomgMuBnZn5oUXLVy4adjZwW/PlSZK6VecqmhcB5wE7ImJ7texdwPqIOIWFUzS7gDf1pUJJUlfqXEXzVSBarLq++XIkSU3xm6ySVCgDXpIKZcBLUqEMeEkqlAEvSYUy4CWpUAa8JBXKgJekQhnwklQoA16SCmXAS1KhDHhJKpQBL0mFqnO7YA1J03+pqenn3bR2nsn+liKpBx7BS1KhDHhJKpQBL0mFMuAlqVAGvCQVyoCXpEIZ8JJUKANekgplwEtSodoGfEQcFxE3RcTOiLg9It5RLX9yRNwQEXdU/x7T/3IlSXXVOYKfBzZl5onA6cBbI+Ik4ELgxsxcA9xYzUuSlom2AZ+Z92bmt6vph4GdwCrg1cCWatgW4Kx+FSlJ6lxkZv3BEauBm4GTgR9m5tGL1j2QmY85TRMRG4GNAOPj46dOT0+33Pbc3BxjY2Od1D4yuu1tx54H+1BNc8YPh72P1Bu7dtVR/S2mYaW+HkvtC8rubWpqaltmTnT6uNoBHxFjwFbg4sy8KiL21Qn4xSYmJnJ2drblupmZGSYnJ2sXPkq67W1Yd5Osa9PaeS7dUe+GpLsuObPP1TSr1NdjqX1B2b1FRFcBX+sqmog4FPg88KnMvKpavDciVlbrVwL3dfrkkqT+qXMVTQCXAzsz80OLVl0DnF9Nnw9c3Xx5kqRu1fn9+kXAecCOiNheLXsXcAlwZURcAPwQ+IP+lChJ6kbbgM/MrwKxn9UvbbYcSVJT/CarJBXKgJekQhnwklQoA16SCmXAS1KhDHhJKpQBL0mFqncjEWlA6t5/Z9TubSMNg0fwklQoA16SCmXAS1KhDHhJKpQBL0mFMuAlqVAGvCQVyoCXpEIZ8JJUKANekgplwEtSoQx4SSqUAS9JhTLgJalQBrwkFcqAl6RCtQ34iPh4RNwXEbctWnZRROyJiO3Vzxn9LVOS1Kk6R/BXAOtaLL8sM0+pfq5vtixJUq/aBnxm3gz8ZAC1SJIaFJnZflDEauDazDy5mr8I2AA8BMwCmzLzgf08diOwEWB8fPzU6enpls8xNzfH2NhYp/WPhKW97djz4BCrac744bD3keE899pVR/V1+6W+HkvtC8rubWpqaltmTnT6uG4Dfhy4H0jg/cDKzHxDu+1MTEzk7Oxsy3UzMzNMTk7WrXukLO2t7h+WXu42rZ3n0h3D+bvt/f6j26W+HkvtC8ruLSK6CviurqLJzL2Z+Whm/gL4KHBaN9uRJPVPVwEfESsXzZ4N3La/sZKk4Wj7+3VEfAaYBFZExG7gr4DJiDiFhVM0u4A39bFGSVIX2gZ8Zq5vsfjyPtQiSWqQ32SVpEIZ8JJUKANekgplwEtSoQx4SSqUAS9JhTLgJalQBrwkFcqAl6RCGfCSVCgDXpIKZcBLUqEMeEkqlAEvSYUy4CWpUAa8JBXKgJekQhnwklQoA16SCmXAS1KhDHhJKpQBL0mFMuAlqVBtAz4iPh4R90XEbYuWPTkiboiIO6p/j+lvmZKkTtU5gr8CWLdk2YXAjZm5BrixmpckLSNtAz4zbwZ+smTxq4Et1fQW4KyG65Ik9Sgys/2giNXAtZl5cjW/LzOPXrT+gcxseZomIjYCGwHGx8dPnZ6ebvkcc3NzjI2NdVr/SFja2449Dw6xmuaMHw57HxnOc69ddVStcXX/Wy/d3v5ej53su7o1DtKB9D4rydTU1LbMnOj0cYf0o5jFMnMzsBlgYmIiJycnW46bmZlhf+tG3dLeNlx43fCKadCmtfNcuqPvL6GWdr1usta4uv+tl25vf6/HTvZd3RoH6UB6n6n7q2j2RsRKgOrf+5orSZLUhG4D/hrg/Gr6fODqZsqRJDWlzmWSnwG+DpwQEbsj4gLgEuDlEXEH8PJqXpK0jLQ9gZqZ6/ez6qUN1yJJapDfZJWkQhnwklQoA16SCmXAS1KhDHhJKpQBL0mFMuAlqVDDuZFIoVbv5z4lm9bOF3P/GUmjwyN4SSqUAS9JhTLgJalQBrwkFcqAl6RCGfCSVKgD9jLJ/V3S2MquS87sYyWS1B8ewUtSoQx4SSqUAS9JhTLgJalQBrwkFeqAvYqmE51ccSNJy4VH8JJUKANekgrV0ymaiNgFPAw8Csxn5kQTRUmSetfEOfipzLy/ge1IkhrkKRpJKlRkZvcPjvgB8ACQwD9n5uYWYzYCGwHGx8dPnZ6ebrmtubk5xsbGuq6lUzv2PDiw5xo/HPY+MrCnG5hh9rV21VG1xnW7n5vorW6NgzTo99kgldzb1NTUtm5Ogfca8E/PzHsi4mnADcDbM/Pm/Y2fmJjI2dnZlutmZmaYnJzsupZODfLSx01r57l0R3lXpA6zr7o3gOt2PzfR23K8Sd2g32eDVHJvEdFVwPd0iiYz76n+vQ/4AnBaL9uTJDWn64CPiCMi4shfTgOvAG5rqjBJUm96+R10HPhCRPxyO5/OzC83UpUkqWddB3xm3g08v8FaJEkN8jJJSSpUeZd26IDgDeCk9jyCl6RCGfCSVCgDXpIKZcBLUqEMeEkqlFfRSENW94qgYd3bppMrlpbj/XcOZB7BS1KhDHhJKpQBL0mFMuAlqVAGvCQVyoCXpEKNzGWS3lxKo6bp12zTl1P6niqfR/CSVCgDXpIKZcBLUqEMeEkqlAEvSYUamatoJNXzeFfHbFo7z4ZlcPXMsG6wNswrh4ZxIzaP4CWpUAa8JBXKgJekQvUU8BGxLiK+FxF3RsSFTRUlSepd1wEfEQcDHwF+FzgJWB8RJzVVmCSpN70cwZ8G3JmZd2fm/wLTwKubKUuS1KvIzO4eGHEOsC4z31jNnwe8MDPftmTcRmBjNXsC8L39bHIFcH9XxSx/pfZWal9Qbm+l9gVl93ZCZh7Z6YN6uQ4+Wix7zP8tMnMzsLntxiJmM3Oih3qWrVJ7K7UvKLe3UvuC8nvr5nG9nKLZDRy3aP5Y4J4etidJalAvAf8fwJqIOD4iDgPOBa5ppixJUq+6PkWTmfMR8TbgK8DBwMcz8/Yeaml7GmeEldpbqX1Bub2V2hfY22N0/SGrJGl585usklQoA16SCjXwgG93e4OIeHNE7IiI7RHx1VH5dmzd2zZExDkRkRExMpdz1dhnGyLix9U+2x4RbxxGnd2os98i4rUR8d2IuD0iPj3oGrtRY59dtmh/fT8i9g2jzm7U6O3XI+KmiLglIr4TEWcMo85O1ejrGRFxY9XTTEQc23ajmTmwHxY+jL0LeCZwGHArcNKSMU9aNP0q4MuDrLFffVXjjgRuBr4BTAy77gb32Qbgw8OutU+9rQFuAY6p5p827Lqb6GvJ+LezcJHE0GtvaJ9tBt5STZ8E7Bp23Q319S/A+dX0S4BPttvuoI/g297eIDMfWjR7BC2+PLUM1b1tw/uBvwX+Z5DF9ajkW1LU6e2PgY9k5gMAmXnfgGvsRqf7bD3wmYFU1rs6vSXwpGr6KEbj+zl1+joJuLGavqnF+scYdMCvAn60aH53texXRMRbI+IuFsLwTwZUWy/a9hURLwCOy8xrB1lYA2rtM+A11a+On4uI41qsX47q9PYc4DkR8bWI+EZErBtYdd2ru8+IiGcAxwP/NoC6mlCnt4uAP4qI3cD1LPyGstzV6etW4DXV9NnAkRHxlMfb6KADvu7tDT6Smc8C/hz4i75X1bvH7SsiDgIuAzYNrKLm1NlnXwRWZ+ZvAP8KbOl7Vc2o09shLJymmWThSPdjEXF0n+vqVa33WeVc4HOZ+Wgf62lSnd7WA1dk5rHAGcAnq/fgclanrz8FXhwRtwAvBvYA84+30UE33entDaaBs/paUTPa9XUkcDIwExG7gNOBa0bkg9a2+ywz/zszf17NfhQ4dUC19arO63E3cHVm/l9m/oCFm+WtGVB93erkfXYuo3N6Bur1dgFwJUBmfh14Igs3IlvO6rzP7snM38/MFwDvrpY9+LhbHfAHCYcAd7PwK+EvP0h43pIxaxZNvxKYHfYHIE30tWT8DKPzIWudfbZy0fTZwDeGXXeDva0DtlTTK1j4Nfopw669176qcScAu6i+8DgKPzX32ZeADdX0iVVQLusea/a1Ajiomr4YeF/b7Q6hkTOA77PwifG7q2XvA15VTf89cDuwnYUPEvYblMvpp11fS8aOTMDX3Gd/U+2zW6t99txh19xgbwF8CPgusAM4d9g1N9FXNX8RcMmwa+3DPjsJ+Fr1etwOvGLYNTfU1znAHdWYjwFPaLdNb1UgSYVa7h88SJK6ZMBLUqEMeEkqlAEvSYUy4CWpUAa8JBXKgJekQv0/l7OMyzR1HKYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Distribution of predictions of the NF1 Classifier applied to input data\n",
    "nf1_scores_df.T.hist(bins=30);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Apply NF1 Classifier to Shuffled Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Apply the NF1 classifier to the input RNAseq matrix \n",
    "nf1_shuffle_scores_df, nf1_shuffle_common_genes_df, nf1_shuffle_missing_genes_df = (\n",
    "    apply_classifier(coef_df=nf1_coef_df, rnaseq_df=exprs_shuffled_df)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combine Ras and TP53 predictions and output to file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sample_id</th>\n",
       "      <th>ras_score</th>\n",
       "      <th>tp53_score</th>\n",
       "      <th>nf1_score</th>\n",
       "      <th>ras_shuffle</th>\n",
       "      <th>tp53_shuffle</th>\n",
       "      <th>nf1_shuffle</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ALL-102</td>\n",
       "      <td>0.524098</td>\n",
       "      <td>0.430674</td>\n",
       "      <td>0.504012</td>\n",
       "      <td>0.578858</td>\n",
       "      <td>0.371647</td>\n",
       "      <td>0.557121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ALL-105</td>\n",
       "      <td>0.417246</td>\n",
       "      <td>0.505113</td>\n",
       "      <td>0.442152</td>\n",
       "      <td>0.315399</td>\n",
       "      <td>0.302545</td>\n",
       "      <td>0.425273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ALL-106</td>\n",
       "      <td>0.446739</td>\n",
       "      <td>0.342593</td>\n",
       "      <td>0.409099</td>\n",
       "      <td>0.529298</td>\n",
       "      <td>0.653105</td>\n",
       "      <td>0.419371</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ALL-107</td>\n",
       "      <td>0.368172</td>\n",
       "      <td>0.418647</td>\n",
       "      <td>0.449422</td>\n",
       "      <td>0.503032</td>\n",
       "      <td>0.497139</td>\n",
       "      <td>0.467002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ALL-108</td>\n",
       "      <td>0.438261</td>\n",
       "      <td>0.450736</td>\n",
       "      <td>0.446484</td>\n",
       "      <td>0.528116</td>\n",
       "      <td>0.501936</td>\n",
       "      <td>0.515667</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  sample_id  ras_score  tp53_score  nf1_score  ras_shuffle  tp53_shuffle  \\\n",
       "0   ALL-102   0.524098    0.430674   0.504012     0.578858      0.371647   \n",
       "1   ALL-105   0.417246    0.505113   0.442152     0.315399      0.302545   \n",
       "2   ALL-106   0.446739    0.342593   0.409099     0.529298      0.653105   \n",
       "3   ALL-107   0.368172    0.418647   0.449422     0.503032      0.497139   \n",
       "4   ALL-108   0.438261    0.450736   0.446484     0.528116      0.501936   \n",
       "\n",
       "   nf1_shuffle  \n",
       "0     0.557121  \n",
       "1     0.425273  \n",
       "2     0.419371  \n",
       "3     0.467002  \n",
       "4     0.515667  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results_list = [ras_scores_df.T, tp53_scores_df.T, nf1_scores_df.T,\n",
    "                ras_shuffle_scores_df.T, tp53_shuffle_scores_df.T,\n",
    "                nf1_shuffle_scores_df.T]\n",
    "\n",
    "all_results = pd.concat(results_list, axis='columns').reset_index()\n",
    "all_results.columns = ['sample_id', 'ras_score', 'tp53_score', 'nf1_score',\n",
    "                       'ras_shuffle', 'tp53_shuffle', 'nf1_shuffle']\n",
    "\n",
    "file = os.path.join('results', 'classifier_scores.tsv')\n",
    "all_results.to_csv(file, sep='\\t', index=False)\n",
    "\n",
    "all_results.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEKCAYAAAAW8vJGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmcXGWZ6PHfU6e6O/tiEpasoCFqJ5MEbVkMgxr0TgQM47DIJuIC1xlRR1miF9mS8Y4so3eUOE5EB1EUIYwmIczFO4CDicClkSSmW8GeCKSTuQJNEkhIqrurnvvHqerUqTpVdaq7Tp1T3c/384FPV/Xpqjenut/nXZ9XVBVjjDEmJxF1AYwxxsSLBQZjjDEeFhiMMcZ4WGAwxhjjYYHBGGOMhwUGY4wxHhYYjDHGeFhgMMYY42GBwRhjjEcy6gIMxtSpU/WYY46JuhjGGNNQnn766VdUdVql60IPDCKyDPhHwAHuUNWvFXx/DvB9YBrwKnCxqnaXe81jjjmG9vb2kEpsjDHDk4i8EOS6UIeSRMQBVgMfBFqBC0SkteCy24C7VHUhsBL4+zDLZIwxpryw5xhOALpUdYeq9gL3AGcVXNMKPJz9+lGf7xtjjKmjsAPDDGBn3uPu7HP5tgJnZ7/+MDBeRKYUvpCIXC4i7SLS/vLLL4dSWGOMMeEHBvF5rjDP91XAe0TkGeA9wC6gv+iHVNeoapuqtk2bVnHuxBhjzCCFPfncDczKezwT2J1/garuBv4KQETGAWer6r6Qy2WMMaaEsHsMTwHHicixItIMnA+sz79ARKaKSK4cX8ZdoWSMMSYioQYGVe0HrgAeAn4H3KuqHSKyUkSWZy97L/CsiDwHHAl8NcwyGWOMKU8a8WjPtrY2tX0MxhhTHRF5WlXbKl1nKTGMMcZ4WGAwZpB69qfYunMvPftTURfFmJpqyFxJxkRt3ZZdrLh/G02JBH2ZDLecvZDliwu36BjTmKzHYEyVevanWHH/Ng71ZXg91c+hvgzX3L/Neg5m2LDAYEyVuvccpCnh/dNpSiTo3nMwohIZU1sWGIyp0szJo+nLZDzP9WUyzJw8OqISGVNbFhiMqdKUcS3ccvZCRjUlGN+SZFRTglvOXsiUcS1RF82YmrDJZ2MGYfniGSyZO5XuPQeZOXm0BQUzrFhgMGaQpoxrsYBghiUbSjLGGONhgcEYY4yHBQZjjDEeFhiMMcZ4WGAwxpiIxS3vlq1KMsaYCN39xAvc9EAnzY7Qn9FY5N2yHoMxxkTk7ide4Nqfb6e3P8P+VDo2ebcsMBhjTAR69qe4aUNH0fNOQiLPu2WBwRhjItC95yBNTnEV3JfWyPNuWWAwxpgIzJw8mrTP0co3fKg18h31FhiMMSYC+ckYxzY7NDvCV/9yARedOCfqotmqJGOMiUpckzFaYDDGmAjFMRmjDSUZY4zxsMBgjDHGwwKDMcYYj9ADg4gsE5FnRaRLRL7k8/3ZIvKoiDwjIttE5PSwy2SMMaa0UAODiDjAauCDQCtwgYi0Flz2FeBeVT0eOB/4dphlMsYYU17YPYYTgC5V3aGqvcA9wFkF1ygwIfv1RGB3yGUyxhhTRtjLVWcAO/MedwMnFlxzI/ALEfksMBZ4f8hlMsYYU0bYPQbxea5wD/gFwJ2qOhM4HfihiBSVS0QuF5F2EWl/+eWXQyiqMcYYCD8wdAOz8h7PpHio6JPAvQCq+jgwCpha+EKqukZV21S1bdq0aSEV1xhjTNiB4SngOBE5VkSacSeX1xdc8yJwGoCIvB03MFiXwBhjIhJqYFDVfuAK4CHgd7irjzpEZKWILM9ediVwmYhsBX4CXKrqk3LQGGNMXYSeK0lVHwQeLHju+ryvO4ElYZfDGGNMMLbz2RhjjIcFBmOMMR4WGIwxxnhYYDDGmIB69qfYunMvPftTURclVHZQjzHGBLBuyy5W3L+NpkSCvkyGW85eyPLFM6IuViisx2CMMRX07E+x4v5tHOrL8Hqqn0N9Ga65f9uw7TlYYDDGmAq69xykKeGtLpsSCbr3HIyoROGywGCMMRXMnDyavkzG81xfJsPMyaMjKlG4LDAYY0wFU8a1cMvZCxnVlGB8S5JRTQluOXshU8a1RF20UNjkszHGBLB88QyWzJ1K956DzJw8etgGBbDAYIwxgU0Z1zKsA0KODSUZY4zxsMBgKhopm3qMMS4bSjJljaRNPcYYl/UYTEkjbVOPMcZlgcGUNNI29Zh4syHN+rGhJFPSSNvUU2s9+1MjYmljPdiQZn1Zj8GUNNI29dTSui27WHLzI1x8x5MsufkR1m/ZFXWRGpYNadaf9RhMWSNpU0+t5Fdkh3B7XNfcv40lc6fa/RuE3JBm7l7C4SFNu5/hsMBgKhopm3pqxSqy2rIhzfqzoSRjaswqstqyIc36sx6DMTWWq8iuKZgsDVKR2YS1PxvSrC8LDMaEYDAVWaOsvIkqeNmQZv1YYDAmJNVUZI0yYd0owcsMjc0xGBMDjbCZ0JaNjhyhBwYRWSYiz4pIl4h8yef73xCRLdn/nhORvWGXyZi4aYQJ60YIXqY2Qg0MIuIAq4EPAq3ABSLSmn+Nqn5BVRer6mLgW8C/hlkmY+KoEVbeNELwMrUR9hzDCUCXqu4AEJF7gLOAzhLXXwDcEHKZTAMbzqt24r7yZiirrUxjCTswzAB25j3uBk70u1BE5gDHAo+EXCbToEbCxGfcV97EPXiZ2gh7jkF8ntMS154PrFXVtO8LiVwuIu0i0v7yyy/XrICmMdjEZ3xMGdfColmTLCgMY2EHhm5gVt7jmcDuEteeD/yk1Aup6hpVbVPVtmnTptWwiKYR2MSnMfUTdmB4CjhORI4VkWbcyn994UUi8lZgMvB4yOUxDcomPuvDzjwwEHJgUNV+4ArgIeB3wL2q2iEiK0Vked6lFwD3qGqpYSYzwjXCqp1GZ6nCTY40Yl3c1tam7e3tURfDRGA4r0qKUs/+FEtufoRDfYd7ZaOaEmxesdTu8zAiIk+ralul6ywlhmkocV+106jikCrcgn58WGAwxkQ+hzMSliI3EsuVZIyJdA7HliLHj/UYjDFAdJvX4jCMZbwsMBhjBkQxhxP1MJYpZkNJxphINdpS5JGw18N6DMaYyDVKDqaRMkluPQZjRpA4t3bjnoNpJE2SW4/BmJDEbV3+SGnthmUkTZJbYDAmBHGrhBvlTOk4G0mT5IGHkkTkSBH5noj8W/Zxq4h8MryiGdOY4jjk0LH7NRIFWfAtO211Gm2SfCiq6THcCfwLcG328XPAT4Hv1bhMxjS0uA05rNuyi2vWbiPVPzJau2FqlEnyoapm8nmqqt4L7m97NnOq76E6xoxkcRpyyPVeCoNCS1KGbWs3bHGfJK+FagLDARGZQvYENhE5CdgXSqmMaWBxGnLwO+BoTLPDdy9ps4lnU1I1Q0lfxD1k5y0ishmYBpwTSqmMaXBxGXLw671kVJk/fWIk5TGNIVBgEJEEMAp4D/BW3LOcn1XVvhDLZkwo6rWMNA4pwnO9l2sKVkhFXS4Tb4ECg6pmROQfVPVkoCPkMpkqxG2tfNzFbRlpTpifY1x6L6ZxVDOU9AsRORv4VzuCMx7iWMnFOVDFdS1/PT7HOPReTOOodo5hLJAWkYO4w0mqqhNCKZkpK46VXBwDVb7uPQdxxH8tf1T3LI6fozGBVyWp6nhVTahqk6pOyD62oBARv9UmUW5YiuOmrkLbd+3jQK93hXXUa/k7du8jUSJYGROVqpLoichyEbkt+9+ZYRXKVBantfIQbqAKkvit0jU9+1Os2thZ9Px1Z7RG2sO67K523ohZsKpWqXtfi8/NRCPwUJKIfA14F3B39qnPi8gpqvqlUEpmyorbapOwAlWQ4akg1/jtRh7b4rBgRjTLNg9vPPNO17UkGyvNQql7X6vPzUSjmjmG04HFqpoBEJEfAM8AFhgiEqfVJmEEqiDj70HH6P0CVzqjkfew8gPVmCaH73z0nZw6b1okZapWqXvfevSEmn1uJhrVZledBLya/dp2yMRAnFab1DpQ+VWejgiP/v4l3ve2I5gyriVwXqJ697Aqrc7y3XiGMn9640zblbr3W3burfiZxC2flPGqJjD8PfCMiDyKuyLpVODLoZTKNKxaBiq/yvNAb5obN3TwlXXbueXshSyZOzXwEFa9elhBhkjiNhQ4GKWGDxfPmlTxM4nbHJnxqmZV0k+Ak4B/zf53sqreU+nnRGSZiDwrIl0i4jvsJCLniUiniHSIyI+DlskMb/k5h8Y2OwPP70+lB1Y9AVXlJQo7AVo1q7OWL57B5hVL+dGnTmTziqUNN75eKifU3CPHV/xM4pRPyhSToHvVROTDwCOqui/7eBLwXlX9eZmfcXDTc38A6AaeAi5Q1c68a44D7gWWquoeETlCVV8qV5a2tjZtb28PVG7T+Hr2p3j09y9x44YO9qcOr+AZ35LkR586kUWzJkW+sS73/vsO9vGZu3/D66l+33IOR6XufZDPJOrPbaQRkadVta3SddUMJd2gqj/LPVDVvSJyA1AyMAAnAF2quiNbqHuAs4D8dYOXAatVdU/2dcsGBTPyTBnXwvvedgRfWbfd83z+0EOUcy35Q0e96QzpgiGS3nSGfQf76NmfqmsZo84JFeQzidMcmTmsmn0MftdWCiwzgJ15j7uzz+WbB8wTkc0i8oSILKuiTGaEiOvQQ+HQUao/g4jQkhTGtyRpcoR0JsNn7v4NS25+hPVbdtWlXOu27GLJzY9w8R1P1vV9zfBQTY+hXUS+DqzGPZPhs8DTFX5GfJ4rHLtKAscB7wVmAr8SkQWqutfzQiKXA5cDzJ49u4pim+EiTstzc/xW14xKOqy+6HgALrurnVSagaGleizJjPtSUBs+ir9qegyfBXpxj/O8DzgEfKbCz3QDs/IezwR2+1yzTlX7VPWPwLO4gcJDVdeoapuqtk2b1hjrvE3txe30rFKra+ZPn8jE0c00O47ne9XuBh/MzuC4pUvJZz2ZxhC4x6CqB8huZstOKo/NPlfOU8BxInIssAs4H7iw4JqfAxcAd4rIVNyhpR1By2VMlHJDXFev3YaTENIZ9QxxDWVJ5mB3Bsd1KWicejLWaykvcI9BRH4sIhNEZCzumQzPisjV5X4mey70FcBDwO+Ae1W1Q0RWisjy7GUPAT0i0gk8Clytqj2D+ccYEwXN/V8PP4KhzYsMJilhz/4Ujz33Mh2793HdGa2xm4+JS0/Gei2VVTPH0Kqqr4nIRcCDwArcOYZby/2Qqj6YvT7/uevzvlbclN5frKIsxsSCN+eRu5Q2vxU82HmRancGr9uyiyvv3UJ/9vImR7hx+XwWTJ8Ym1ZxHHoyceq1xFk1cwxNItIE/CXZOQGKJ5LNMJc/5h2XzJhRliNIK3gw8yLVVKI9+1Ncs3brQFAA6EsrKzd0MrbZoXvPwcg/I4jHyrK49Frirpoewz8DzwNbgcdEZA7wWhiFMvGUP+Z9sK8fEWFU0ok0M2bUGTrDagVXkzLDPYAoQa7HkqMKp3/zV7RE/Bnli3plWRx6LY0g8M7noh8UEcDJziMgIh9T1R/UsnCl2M7n+uvZn2LJzY9wqC/j+/1RTQk2r1ha9w1chWWKohzrt+wqqsBrVQEH3T387q89XJTCu1AU9yaOwvy84i6Mnc8e2bmB/rynPg/UJTCY+vMb884XRWbMMDN0VrNqJcxWcNDdw7ees4gv5s0xJBOCI5BKHw4Wlr3UFXWvpREMOjD48NvMZoYJvy54vlp0x6tdQhjl4UCFok7tkKvsOna/BijTJ47mzNs3QV5gsCGTw6L+vOKuqqM9K7CJ6GGscOIwmXBXvtRqEnEwSwjDmMxshLOrS5kyroVT503j1HlHBMpwWq24LDYw4bMegwmssAsO1KQ7PpQlhPU4HKhRh2BqeW8Ke1HXndkaq6WwprZqGRg21/C1TEwVdsFrUSkMtTIO+3CgRh6CqcW98Qvc1/5sO2ObHdKqI2rydqQY0lCSiHw897WqXjH04piRKE6VsedwoBaH5mSC685sjbRVPJghnFoO+/it/Qf3NL1GGmozwQ21x3AT8C+1KIgZueJ2zOXyxTN4/VA/N23ooMlJsOoB9/iQKIZOBjMRXuu9HZUWHjTqUJspreI+BhHZVupbwDxVrftvw1D3MVgCrWjl33+g5nMWtSif356Neg+dDGafRlh7O3Jr/52EcCDl3Uhn+yMaRy33MRwJ/AWwp/A9gF8PomyRCtqaarTg0Sjlzb//h/rTqCqjm5IlP4uuP73Opq5XmDquhZPfMqUu/7ZSezYO9BbnQqoVv89vMHMvYU2e509kb9+1j1UbO2PRuzPhCBIYHgDGqeqWwm+IyC9rXqIQBV39EnWahWrFubyFvYPC+w+lD7G5/ue/5a4nXhy4zkkI3zhvUej/tloNnQQN1qU+v8HMvYQ5X5ObyF40axLLFhzVEA0RMzhBJp//TlU3+X1DVQvPVoi1IAm0Gm0de5zLW7g34cdPvug7iZmT/1l0/el1T1AASGeUq9duo+tPr4e6nj4359Hk+K/A7k2nK1a0QfdllPv8BrNPo16J6uJ2YJKprSA9hrXAO0XkYVU9LewChSlIa6rR1rHHrby5VvLYZqeod3D7o3+g3HaX/M9iy869vtdkVENNDJcrf+vRE0iUKGpGYXPXKyXft5p9GZU+v8HsRbCUD2aoggSGhIjcAMwTkaIzE1T167UvVjiCrH4Je+lkrecC4rTUM39IJJXOIAULG5odh8tPfTOrf9lVco4hd08Wz5rk+x592RQPvenqzlAOct+Ly+//Wn1pLfu+1QTrIJ9fkL0Ihf++Rkj50CjzYiNRkMBwPu4ZDElgHN4mX8OlwajUmgpz6WQYcwFxWerp10ou1JfJcOGJs7nwxNkVVyLNPXI8l5w8m7sePzyclBBoSkjVieGC3Pcg5c9X7n1nTh7NoX7vyp1D/f7DT/mfnyNCXzrDdWdUt29i3ZZdXJN3tOit58RnjqmUOM+LmQCBQVWfBW4Wkd8Ck4BjAYcGToFRqTUVRlc8zJOj4jB04NdKHtWUIJNRz7BPrmxBdk+vPOvPuOSkYwZWJb3tqPFVJ4YLet/Llb/JSQysSCr3vvnDaIXLwMstCx/YN/FAJ83JBKs2djJ+VDJQRdmzP8VV920d6EkBXHnf1lifSFaPU9SsNzI01Wxw+xywF/gNcCic4sRHrbviYc8FRD10UGolz4Of+3MO9KYH/Qc698jxzD1y/MDjcr2joSz5rFT+7bv3seqB0ks0C4ehHBH68zrUo5uSJT/rnv0pVm3spLc/Q282kX3QirJj9z5PUAB3qKtj9z5OnXdE2Z+NSth/C8OtNxJFkKsmMMxU1WWhlWSYi9NcQBhKDWnlV+q1UKp3VM2Sz1R/mrHNTlXlXzRrEsvm+y/RDDqMVuqzHmxF2bM/xY6XD5T4bnw79GH+LQy3M52jCnLVBIZfi8ifqepvQyvNMBaXuYAw1WtIq7B3VKkyyN13zSiptJJICGfevqnojyzI/JNf76TaYbRCg6kocxVG0mfpVAKYPnFUyZ+NWph/C3FbpTcUUQa5agLDKcClIvJHIIXbJFFVXRhKyYahOMwFhG2oQ1rVdJtz1+472Icj3goyf0/EnClj+dEnTuDC7/1fQAfSRfj9kQUpf1EK6jNahzSMVm1FmV9h+MkAy/7xMb5+3uKqWpe1GLII+hph/S0Mp555lEGumsDwwdBKMYJEPRcQB6Uqj2q6zYWpNYrG2TMZtu/ax0fWPO6O+/enSST8g0c1n4dfK27Vxk6uO7O1aA6immG0airKSsesAvRn4Oq1wSehazFkUe1rhPG3MJx65lEGucCBQVVfCLMgZmQoVXn4VbhXr93KpDFNzJ8+seox/S++fx6rNnZ6r/EJHtX+kZVqxS2YPpHNK5YOqQUctKKslLIjx5HgqTuGOmQRp7H94dIzjzLI1fKgHmPKKld5+FW4qX7l0z/6DZmCjKaVWsxjWxzeNLYZzXgDgSPgOAlanMH9kfXsT7HvYB+9af9WXL16g4UVRm86TUYp6jWlNVjgq3bIolYJ/8I0XHrmUQW50AODiCwD/hF378Mdqvq1gu9fCtwK5JLJ3K6qd4RdLlMs7GVx5SqPUq3gN3wymlZqMaczyjFTxng2woHbYfjpJ0+gKelU/W/M7+mkMxmaHGFUdmL5ujNbB+Yz6vWHu3zxDFqPnsCWnXtZPGsSnf/1Glfm7WdIJuDWcxYNugdSqjdVy4R/JpgoglyogUFEHGA18AGgG3hKRNaramfBpT+1E+CiVY9lcX6VR286zb6DvcycPHqgFZxAeKPPu6Esv/VZ2GL2S63RlHQY1ZQoOpegKemwqES6jVL8ejotSVh90fHsfPVg0dyC367qagNupZ/x+7ye+PJpdOzeBwjzp08I/F5BhyyCrv5q9LF9E36P4QSgS1V3AIjIPcBZQGFgMBGq1/hwYeVxsK+fjMJn7n5moCLZvGIpHbtf47K72kn1H67UC1ufhV1s8KbWKJV5dTAtWL+eTtJxs8QWzmMU3rcwTmAr9XltXrF00JvaggxZhJHwz8TTkM58DmAGsDPvcXf2uUJni8g2EVkrIrNCLpMpECQdea0sXzyDzSuWsvqid+AkEvSl1ZNuGuDUedO49ZzKqaOnjDuc+jn/69z3bjl7IS1JYUyTQ0tSSrZgK52P7NfTOZBK8/h/9hTdtwRCx+7XBl632pToQX5mMJ9XkDOgC+9hoaAJ/ywdd+MLu8fgt/2yMGnMBuAnqpoSkU8DPwCWFr2QyOXA5QCzZ8+udTmHrSDDGPUeH54yroWJo5todhKeXkGtW5/uL5pkd9z47wQO0qKfMq6F685s5dqfbfc8//3Nz6PqvW9v9KW57K52bj1nIZPGNFE4FZLJKBu27uZDi6YP9Gzy/41BJnGr/bxqNUxow0UjR9iBoRvI7wHMBHbnX6CqPXkPvwvc7PdCqroGWAPumc+1LebwFDSraPeeg77r8MP8gx9KuukgwS7X8s4PPIXDPNUMoS2YPpGxzY4nmZ6Iu1+gUKo/w5X3bUUzSn/Bb2pvWrlxQyd/t7GTC0+Yzb1Pd3vu+ZK5UwPdl6AVdK2HCW24aGQIOzA8BRwnIsfirjo6H/Cc+iYiR6vqf2UfLgd+F3KZhqXCyjJIheC3g3fBjImD/oOvZpJ1sK3PoK3fIC3vas9NKFymWmrnMRQvHS3Un2HghLrCeYIg9yW/gs4FrNypb9Xeh2oNl6WgprRQA4Oq9ovIFcBDuMtVv6+qHSKyEmhX1fXA50RkOdAPvApcGmaZhiO/ynLOlLG+FULH7teYOLrJ94S1VRs72bxi6aD+6O9+4gU3bbQj9Gc00HBFta3Palq/QXokla4pPK+6XOrsWshV2EHvy5RxLWzqeqVsoLRlpGYwQt/HoKoPAg8WPHd93tdfBr4cdjmGq1KV5QNXnFJUIRzs6+eyu9rdsX2fE9YG25K8+4kXuPbn7vh7tWmjq2l9Bm39Bh0eK9drKQy2n3nvXEY3JXk91V+2jC3JBBnVij0GP6l0ZiDra5D7EiRQ2ryAGQzb+dzgSlWWB3rTvrtjU/0Zz7h7vsG0JHv2p7hpQ0fR8wkRHv39S7zvbUcEToZXqdcQpPVb7fCYX+vcr8K9/dEughxYmBC3Z5E7TS3/+UTenEQyAReeOJt727sBd1hKVH2zvpYSNFDavICplgWGBleuslw0a9JAhbDvYK+7XyB9uMVbTWroUrr3HKTJcQNPvjd609ywvoOvrNvuyYcU9BwFP5Vav34V+soHOvn1l8oPjxW2zv0q3GYn4Tmvui+ToW3OZDZ19Xhe62B23iGZUFqSCc9xm0vmTi3agHbJScdw+jd/BeDu1K5wnnS+aoaJbF7AVMMCQ4OrVFnmKoSe/alQTlibOXk06RJj7wfy0lm8fqifVRu9wzpL5k4tORQC/mdBl2v9+udbynDHr3aw4oNvr+rf5Ffh5p9XPbbZcY8ZLaEl6XD1X7yVN08b59mFfOq8IwYCZO4etSQdevMCdtAhvdxnf3Xeec+5z96OtjRDYYFhGAgyVFAqgAz1hLX813VESPWncRJCKm+dpiPCTQ+4R1fmB4A1H23zHQq5+8kX+XZey9yvx5FLa9H1p9cH8gX5rRwC+Kf/2MHMN43hohPneM5lLhUQgwTbrTv3lk3kd6A3zW2/eLZoIj7IWQ7VDOlp7v8qA4/WbdnFNXnB4tZz4nG0pQWrxiFhr7QIQ1tbm7a3t0ddjIYUxh9nz/5UdrevMn3iaM68fZNnKWezIzQnE+xPHR5uGt+SZPVFx3P5D5/2XNuSTADqCSwtSeETS47l+5ufp9k5XKFu7nqZB7f/aeC6S06ezdjmJP/0HzuKyticTHDDma2sfKATVehNZ2hxBElIyeGrcveqZ3+KJTc/UnbJas6opgSbV7h7Ngt/ZlRTwneSPEhF7leGlqQUZVptcoQnvnxapJXxcDuHuVGJyNOq2lbxOgsMZij8/uABT2v7ujNaB3IK5bQkE/z6S0vZ3PWK59rPvHcuax7bUXH1TylNCaEvU/w7PaY5QW+/0u/zvVzFXW2Cu+279rFqYydOQjiQSpe8PhcEX3otxbU/+60n62uLI9z76XcPHBFaTcDeunMvF9/xpOdejUomOOSzuOCuT7zLk0epnq13vwBW7T03tRE0MNhQkhm0csncCg+tGT8q6UkLnc5k2Nz1im8yvNW/7Bp0mfyCArgtaL+gAO5QV9AVVEVDQWe20uIkWHH/tsJzgAYc6k/zyTufIpmQolTgqbSy89U3PMNjQfnNhaRLpiM/nBKk3q33uJ3VYCoLO4meGcbKJXMrTKa2ZO5U8k/W7M8wkByuMBnedWe00uwIo5JD//Uc3ZRgVFOCT51ybMlrDvSmuXFDB0tufoT1W3aVvM4vwd2qBzqZPKbJNyiMbnJocoS+tNKXgYOF+TGyrrl/W8n3Lpf8LjcXkp9s8MblCyi8bckEzJ8+oeS/oVJiv6GyTXaNx3oMpkgt9hUUvkbH7tdwJAEcHnLxazWu27KLVRs7aU66CfZyFetgtCSFW89ZyITRTUyfOJo1j+0o2arPzX+UWypaquW7tXvpENU8AAAUe0lEQVSf72tefNIsvr/p+Yrl9DuMCMq37HP3d8ncqb69s6vXbsWRBGnNeA7siaL1bpvsGo8FBuNRi30FhWkaznvnTH7a3l20sa437W01dv3pda6+byu9ebV3MuHORyQT4klgl+MkhIwqhVNlLckEH2mbyVVrD5dj2fyj2Lj9/5X995erJGdOHs3BPu/cx8G+fk49birffKR4+OvNU8eVDESFhwgVvne5Xc2V0mCUW6UWtPVe6zkI22TXWCwwmAGDycTpN0eQm2jMvUYuWVyh/HmGdVt2cfXabZ6gADC6Kcnqi97BS68d4ob12znQ663UEgIrly8Y2CPRm07ziVOOZf7RE7nyvq2k8pbI/vvvX6LZkaL3yFeqx5MjcnhZaO7xs3/aT0IgfwrjkpNnM2PyGN/3uPTkOSx9+5FlDyMq1bLv2L0v0GdUakNbkNZ7WHMQtsmucVhgMAMGO8yQ/wdfaX1/vtw8Q+vRE1hx/zZ6fVbT9GUyzJ8+gfnTJ/A/flZcoTcnEyyYMXFgOCW3UiiBFPVQRKC/YBnnBSfM4t727rI9nlzFePeTLxYNazUnE9y0ocMTFJod4fOnzQPcHk9+MRyBz552HFPGtXDrOaUr6LHNDql+bw/JbenLkIeCyrXe63Wan4k3CwxmQC0mCf1eoxzNKFtKBJNmx3vq2g0fmj+QrC+nP62eyu0jax4vubeg8PmEwOdPm8fnT5tXtsdzzf3bmD5xFN96+Lmi1+zr16KUIEknwQ9+/TzLF03n6+ctLjneX6qCzrXYEwmBtHr2W8yfPqEmE7mlWu+2gsiArUoaEYIc6wj+q1yqnST0e43z2mbS7Eh285pXKq0cM2VMUWXXnEzw4Of+3DOEcdFJc/jqhxeQzFvelBuOAv9VUgBjmh2akwlaHO8pbs2OU7SCyu81NKNccMeT+MWbT55yTFFKkDd603zzkS7e/43H2NT1Mt+9pI3vfPSd/PpLp/meDpe/esvTYs++YQb40SdOYPniGTX5jMqxFUQGrMcw7FU7XlyLScL819i+ax83bujIDsH4by5rSjqB03Usm38UKzd0DAzP5IajlsydWrK38rF3z+Hs42e6uY3yhoL8Kjy/1yjce5DjCHzqz9/M24+ewDX3byMhMrDCKOfe9l1s3Pb/SGuwMyr8Wux9aeXCO57ktnMXsXzxDJYvnkHr0RMGUoEMNa1JPltBZMACw7A22PHiWkwS5n7+vH9+vOJy08JMsOUCUveegzQ7Dqn+4qRzi2ZN4ovvn8f//Lffe37mXzY/z6dOeXOgCi+/YtSMlgwKgDvUw+FA+INfP++7OulAieWope6FX3DrTStXr93KpDFN7Hz1YFFCwlpuULMVRMYCwxDFOTFY1OPF3XsO4iTE93tjmh0yqp7KOUhAchPlFU/Kjm12+ObDf+BbjxTPAxSejJbL6zR/+kTf98i1yE//1ibKncEwKnl4KGrKuBaWL5ruGxgKy1EpKN9y9kKuWls8GZ/qV/77D58eSO0d5uSwrSAa2SwwDEHcE4NFPV48c/Joz2E1Oc0OfOfidzB/+sSqK59NXa94VgAlE3DeO2dyxrc2VTyAqGd/iruffJHVj3YNJOMr9Zkd6E3T4iR8V0oVvm7O3CPHc8nJs7nrcf/luYXXl2pUDASmb/6qaGntQZ+JDpscNrVmk8+DFEVqgWqFPVEZ5P0/8q6ZnucEuO3cxZw6r3JeokK5e54/NOUkEvy0fWfJoNCcTAwsQX331x7m6//nOVL9lT+zmZNHc6i/dGK8lqT43suVZ/0Z//6FU7ntnIX87WlzaU4mGNviFN37dVt2seTmR7j4jid902HMPXI8t527iFFNCcY0OWXvi00Om1qzHsMgRT1ME1SU48U9+1MDR1fmNCcTAwfxVMvvnjsJyZ5FUFyJNzvCg589hcljm1ly8yOeVN455T4zv8zDLckEV7xvLheeOLvkvZx75Hg6/us1vvPYDjfba3+GGz4035POIsjcT/7QV+FmOPAfjjOmFqzHMEhRD9NUo3BJZL34Lf1sdhIDp5dVyz+bqJLW4t6CI3DbuYuYe+T4kstYofRn1r3nIKObvO2mMU0O372kbWCDWin5Ff+B3jS9aWXVxs6Bnkm55IOFpoxr4dR507j1HLfnl1ty25IU+tNuSvM4DV+a4cECwyDVepgm6F6DRlLr4DllXAvXndnqGZ659ZyFXP+h+UXXJp3DPZNSK31akqU/M7+fyaADWUqh9GdWqeIfzH1ZvngGD1xxCipuYEj1a1HAiYvh+Ls80thQ0hDUapgm7pPY+apZhVXrNfHrtuxi5YZOHIHe/gw3Zodntu7cO3BUZ07S8Z6xkF+O3nSaK953XNnhoEplL/eZVar4B3tf/CbE4zZ82Ui/y6Y0O8EtYo10utVg/+iHuqTXPTp0H5+48ylP3qHckZVQfGQmwNjmBH0ZuOFDrZ7znqsph9/PBPnM1m/ZVVTxF96rassT99+VuJfP2AluDaNRJrGDTJiWquiGsiY+d7A9eJPRgbsjuGP3Pk6dd8RACzz/mM1cJtZrf7Yd1E2pUW05/Moe5DML0pus9r7EfVdyo/wum8osMESsUSaxK/3RhzGE0LM/xVV5x4H68+4+fvT3L/mm575pQwfLFhxVkwqqmgOKal0hxnlXcqP8LpvKQp98FpFlIvKsiHSJyJfKXHeOiKiIVOzmDCdR7zUIqlJlGMaejo7d+8oGhfwjK8G9l+972xG+ye6ahrAaqlCpz2xT1ytl9ybUSlSrzCpplN9lU1moPQYRcYDVwAeAbuApEVmvqp0F140HPgc8GWZ54irOrcCccsMYfmcw1GYIwT+dRrMjiOBJYZ1fzhs+1OoOH+VJq9a05RrkgKKReI5BI/wum8rCHko6AehS1R0AInIPcBbQWXDdKuAW4KqQyxNbjZCbptQffanexNhmh6079w66gpg/fQKOeBKi4gjc8bG2suk0LjpxDqg7fNTkJEirct0ZrQM9hlrd5/zPbDDBMc55toaiEX6XTXlhB4YZwM68x93AifkXiMjxwCxVfUBESgYGEbkcuBxg9uzZIRTVBOH3R+/XmzjvnW6aa0eEvrS78/eik+ZU9V6bul4h/yTNZAK+fp6bTqOSi06aw7IFR3lOdQtzCWW14+uNtKxzuAYwU1rYgcFvLGCg/SciCeAbwKWVXkhV1wBrwF2uWqPymRrJ702MbXY48/ZNnmWL1/58O0i2NR9Abt7CcyxmQqpKp5GrxHKnuoWdjTToiqFGOj6zkQKYqZ2wA0M3MCvv8Uxgd97j8cAC4JfuIescBawXkeWqOjw2Kowgud7E1p17caS4TXDThk6WzQ+2MshvFVT+iWtBlVtNlft+kJZwkFZz0PH1RlnW2UgBzNRW2IHhKeA4ETkW2AWcD1yY+6aq7gMGmoAi8kvgKgsKjW3m5NH0pX3SQzsSuPKr1dLHUq+zfdc+PrLm8UAt4WpazUHPlGiEZZ2NEsBM7YW6XFVV+4ErgIeA3wH3qmqHiKwUkeVhvreJjrsyqDh/UToTfGXQUJc+5vL1AEWvc90Zraza2BloeW0YS3EbZVlnowQwU3uhb3BT1QeBBwueu77Ete8NuzymPi46aQ6IO3zU5AjpTPXpoQe79NGvhb95xdKB16mmJRxWq7kRlnXGfae1CY/tfB4hSuX8CbNiuujEOSybf9SQ3qPapY+lxsU3r1jKolmTBq4L2hIOs9XcCMs6GyGAmdqzwDAC+LWgFeqy2qTelV+QFn6llnBhwBzpreZGCGCmtiwwDHN+Leir124FhFT/8FttErSFX6olXGqiuXCX82PPvQQI86dPaPh7ZkwhCwzDnO9xmJIo2mEyXFabVNPCL2wJV1qemUsWmJ/YL7fprtrelm0aM3FmgWGY8z0OUzPZc5IPG06rTQY7Ll5pGKpnf4pr1m7zJPbrz7g9sGp6W7ZpzMSdHe0ZsbCPQfRbGnnrOYsGzhCOy3LJWt+HoBlI89+30jBU956DOInijXuOBM/cGlYmWmNqyXoMEapXy7FUCzouq00q3Yewhl383rfcMNTMyaNJZ4qzsaQ1eG/LNo2ZRmBHe0bEjkF0VboPYQXPcu8LpVNlrN+yiyuHMMdgn7uJkh3tGXPWcnRVymUUVq6ecu9bbggq1/vq2L2PwaxKsuWvphFYYIiIpRtwlbsPYQbPodz/KeNaAqX+LsU2jZm4s8nniDRKvpywlbsPYe86vuXshbQkE4xpdmhJ1vf+x/V4TmPAegyRspajq9R9mDKuhevObB1SvqVyNPd/zTsNyBhjgSFqlm7A5Xcf1m3ZxaoHOmlKCH397ilwtVq1lVs2mupXIA0Mn93fxgyVDSWZWMpf73+gN01vWlm1sbNm6/1z8xf58ie9jRnJLDCYWAq74rbJf2NKs8BgYinsitsm/40pzeYYTCzVY72/Tf4b488Cg4mtelTcNvlvTDELDCbWRkrFbWm4TZxYYDAmYpaG28SNTT4bE6F6puEOO8W7GT6sx2BMhOqVTNF6JaYa1mMwJkL12E9hhwOZallgMCZC9dhPYbu8TbVsKMmYiIW9LNd2eZtqhd5jEJFlIvKsiHSJyJd8vv9pEfmtiGwRkU0i0hp2mYyJmzDTcNsub1OtUI/2FBEHeA74ANANPAVcoKqdeddMUNXXsl8vB/5GVZeVe93hcLSnMfVmeyVMXI72PAHoUtUd2ULdA5wFDASGXFDIGoslxjcmFCNls6AZurADwwxgZ97jbuDEwotE5DPAF4FmYKnfC4nI5cDlALNnz655QY0xxrjCnmMQn+eKegSqulpV3wKsAL7i90KqukZV21S1bdq0aTUupjHGmJywA0M3MCvv8Uxgd5nr7wH+MtQSGWOMKSvswPAUcJyIHCsizcD5wPr8C0TkuLyHZwB/CLlMxhhjygh1jkFV+0XkCuAhwAG+r6odIrISaFfV9cAVIvJ+oA/YA3wszDIZY4wpL/QNbqr6IPBgwXPX5339+bDLYIwxJjhLiWGMMcbDAoMxxhgPCwzGGGM8Qk2JERYReRl4IepyhGwq8ErUhYgJuxcuuw+H2b04rJp7MUdVK24Ea8jAMBKISHuQnCYjgd0Ll92Hw+xeHBbGvbChJGOMMR4WGIwxxnhYYIivNVEXIEbsXrjsPhxm9+Kwmt8Lm2MwxhjjYT0GY4wxHhYYIhbg6NMvikiniGwTkYdFZE4U5ayHSvci77pzRERFZFiuSglyH0TkvOzvRYeI/LjeZayXAH8fs0XkURF5Jvs3cnoU5QybiHxfRF4Ske0lvi8i8s3sfdomIu8Y0huqqv0X0X+4iQX/E3gz7iFFW4HWgmveB4zJfv3XwE+jLndU9yJ73XjgMeAJoC3qckf0O3Ec8AwwOfv4iKjLHeG9WAP8dfbrVuD5qMsd0r04FXgHsL3E908H/g33DJyTgCeH8n7WY4jWwNGnqtqLex7FWfkXqOqjqvpG9uETuGdaDEcV70XWKuAW4FA9C1dHQe7DZcBqVd0DoKov1bmM9RLkXigwIfv1RMqf99KwVPUx4NUyl5wF3KWuJ4BJInL0YN/PAkO0/I4+nVHm+k/itgqGo4r3QkSOB2ap6gP1LFidBfmdmAfME5HNIvKEiCyrW+nqK8i9uBG4WES6cbM4f7Y+RYudauuSskJPu23KCnT0KYCIXAy0Ae8JtUTRKXsvRCQBfAO4tF4FikiQ34kk7nDSe3F7kL8SkQWqujfkstVbkHtxAXCnqv6DiJwM/DB7LzLhFy9WAtclQViPIVqBjj7NHmR0LbBcVVN1Klu9VboX44EFwC9F5HnccdT1w3ACOsjvRDewTlX7VPWPwLO4gWK4CXIvPgncC6CqjwOjcHMHjTTVHqNclgWGaAU5+vR44J9xg8JwHUuGCvdCVfep6lRVPUZVj8Gdb1muqu3RFDc0FX8ngJ/jLkpARKbiDi3tqGsp6yPIvXgROA1ARN6OGxhermsp42E9cEl2ddJJwD5V/a/BvpgNJUVIgx19eiswDrhPRABeVNXlkRU6JAHvxbAX8D48BPw3EekE0sDVqtoTXanDEfBeXAl8V0S+gDt0cqlml+kMJyLyE9yhw6nZ+ZQbgCYAVf0O7vzK6UAX8Abw8SG93zC8h8YYY4bAhpKMMcZ4WGAwxhjjYYHBGGOMhwUGY4wxHhYYjDHGeFhgMCNedu13rP4WRMSJugxm5IrVH4Mx9SIix4jI70Tk28BvgO+JSHs2jfVNedd9LS/t+W1lXu9cEdkuIltF5LHsc46I3CYiv83+/Gezz5+WTRP922w65Zbs88+LyPUisgk4V0TeIiL/W0SeFpFficjbQr0pxmTZBjczkr0V+Liq/o2IvElVX8221B8WkYW4aQY+DLxNVVVEJpV5reuBv1DVXXnXXQ4cCxyf3az1JhEZBdwJnKaqz4nIXbjp1P9X9mcOqeopACLyMPBpVf2DiJwIfBtYWtM7YIwP6zGYkeyFbIpigPNE5De45xzMx83t/xpueu87ROSvcHeUlrIZuFNELsPdpQvwfuA7qtoPoKqv4gajP6rqc9lrfoCbaz/npwAiMg54N+6O9y24aVEGnUbZmGpYj8GMZAcARORY4CrgXaq6R0TuBEZlW/kn4ObiOR+4ghItdlX9dLZVfwawRUQW42a8LEwt4JcFs6hMuI22vaq6uPp/ljFDYz0GY9yDXg4A+0TkSOCDMNBqn6iqDwJ/C5SspEXkLar6pKpeD7yCm+nyF8CnRSSZveZNwO+BY0RkbvZHPwr8R+HrqeprwB9F5Nzsz4qILKrJv9aYCqzHYEY8Vd0qIs8AHbhZSjdnvzUeWJedFxDgC2Ve5lYROS573cO4x1Bux818uk1E+oDvqurtIvJx3CGiJG4G0e+UeM2LgH8Ska/gJky7J/u6xoTKkugZY4zxsKEkY4wxHjaUZEwVRORa4NyCp+9T1a9GUR5jwmBDScYYYzxsKMkYY4yHBQZjjDEeFhiMMcZ4WGAwxhjjYYHBGGOMhwUGY4wxHv8fjXOX8RqEgjIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "all_results.plot(kind='scatter', x='ras_score', y='nf1_score');"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:expression-classification]",
   "language": "python",
   "name": "conda-env-expression-classification-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}