fml_12.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4f540bb9-03a7-43e4-8b1a-98f57b63e093",
   "metadata": {},
   "source": [
    "# Fundamentals of Machine Learning - Exercise 12\n",
    "Goal of the excercise is to learn how to save trained models and use selected advanced libraries like Plotly or Optuna.\n",
    "\n",
    "\n",
    "![meme01](https://github.com/rasvob/VSB-FEI-Fundamentals-of-Machine-Learning-Exercises/blob/master/images/fml_12_meme_01.png?raw=true)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36f950f9-dc19-4c8f-a745-5f490492fed1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# For Google Colab\n",
    "!pip install optuna"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b83617d-af10-4f63-956b-9d5a52107893",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import plotly.express as px\n",
    "import optuna\n",
    "import joblib\n",
    "\n",
    "import sklearn.datasets as skd\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.metrics import make_scorer, accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc021811-5e15-481f-a3b9-58ad0c791b3a",
   "metadata": {},
   "source": [
    "# 📊 Plotly\n",
    "https://plotly.com/python/getting-started/\n",
    "\n",
    "* The plotly Python library is an interactive, open-source plotting library that supports over chart types covering a wide range of statistical, financial, geographic or scientific use-cases\n",
    "* Built on top of the Plotly JavaScript library (plotly.js)\n",
    "* Plotly enables Python users to create **interactive web-based visualizations** that can be displayed in Jupyter notebooks"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f7e606f-7a2d-4db1-b215-cc139b296b94",
   "metadata": {},
   "source": [
    "## 📒 Here we have some examples of commonly used plots\n",
    "* 💡 Express API is easy to grasp and it is very similar to Seaborn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2d9de4b-97c0-48b6-9ba2-0ad29192d06d",
   "metadata": {},
   "source": [
    "## Scatter plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69535c97-c357-4b73-811a-c0d4d78fe359",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = px.data.iris()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "18474621-967d-4713-8593-4abfdc2e7322",
   "metadata": {},
   "outputs": [],
   "source": [
    "px.scatter(df, x=\"sepal_width\", y=\"sepal_length\", color=\"species\", symbol=\"species\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27ae6f3e-5d40-4ebe-816d-da2d5ca3b2a1",
   "metadata": {},
   "source": [
    "## Line plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2f8154a-9b74-49fe-a7a9-bf078d3b757e",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = px.data.gapminder().query(\"continent == 'Oceania'\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f992a05-d084-46fc-88a4-d9b278a95a84",
   "metadata": {},
   "outputs": [],
   "source": [
    "px.line(df, x='year', y='lifeExp', color='country', markers=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4038b220-b7f1-4a79-b418-55cd2122c41a",
   "metadata": {},
   "source": [
    "## Bar plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "357b8ad7-81a6-4710-9e08-4a7d649810e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = px.data.medals_long()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58311eaa-ee09-437d-a86e-222f35daa3d0",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "px.bar(df, x=\"medal\", y=\"count\", color=\"nation\", text=\"nation\", barmode='group')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "115dfae3-0b83-4bbb-949a-64907c269962",
   "metadata": {},
   "source": [
    "## Box plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a112fc82-9dc5-4f2c-9a10-38bf04754f16",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = px.data.gapminder().query(\"continent == 'Oceania'\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73caea75-de8e-4e12-a204-1247267f3110",
   "metadata": {},
   "outputs": [],
   "source": [
    "px.box(df, x='country', color=\"country\", y=\"lifeExp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24ae6a8e-b011-458d-b19c-8147b067c252",
   "metadata": {},
   "source": [
    "## Heatmap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db134117-96a8-4d43-9d09-30812a7785e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = px.data.iris()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6fcfb5bc-51ff-46aa-bdf5-1a13193e906a",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_corr = df.iloc[:, :-2].corr()\n",
    "df_corr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "754941d2-926e-4722-a1d5-a20381a82705",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.imshow(df_corr, text_auto=True, color_continuous_scale=\"blues\", aspect=\"auto\")\n",
    "fig.update_xaxes(side=\"bottom\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e66ecd5-8422-4590-b2cc-4eb5434ba56c",
   "metadata": {},
   "source": [
    "## 📌 Parallel categories diagram\n",
    "* How to read it?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee64109d-7e81-4b74-8820-32f7043b3b78",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/rasvob/VSB-FEI-Fundamentals-of-Machine-Learning-Exercises/master/datasets/titanic.csv', index_col=0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e931662b-b71a-4175-bd77-606cf5023e7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "px.parallel_categories(df, dimensions=['Embarked', 'Sex', 'Survived'], color=\"Survived\", color_continuous_scale=px.colors.diverging.Spectral)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0696900d-0562-440d-a08d-ef20b2ee765f",
   "metadata": {},
   "source": [
    "![meme02](https://github.com/rasvob/VSB-FEI-Fundamentals-of-Machine-Learning-Exercises/blob/master/images/fml_12_meme_02.jpg?raw=true)m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98944256-4b48-461c-b785-0a2cf035d97b",
   "metadata": {},
   "source": [
    "# 🚀 Optuna\n",
    "https://optuna.org/\n",
    "\n",
    "* An open source hyperparameter optimization framework to automate hyperparameter search\n",
    "* You can use it with any machine learning or deep learning framework\n",
    "    * Scikit-learn, TF2, PyTorch, Keras, ...\n",
    " \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e0d7665-c6c1-4cec-b4dd-731aa3f548ab",
   "metadata": {},
   "source": [
    "## ⚡ Using Optuna is very simple\n",
    "* You just need to define the `objective` which will be used for each trial\n",
    "* Then you define the parameter ranges through `suggest_XYZ` function and use is as a regular parameter\n",
    "* After that you can start tuning the parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fc620e70-0e85-41b3-a5a1-b4df133663d8",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "X, y = skd.load_iris(return_X_y=True, as_frame=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e34e1b58-bd1d-4cb3-8315-8816b4452ed3",
   "metadata": {},
   "outputs": [],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4769e3d9-4e29-4f66-bcac-270d18e2dbd2",
   "metadata": {},
   "outputs": [],
   "source": [
    "y.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d046115-66e8-4970-a10b-1e29479d60b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "def objective(trial, X, y):\n",
    "    n_estimators = trial.suggest_int('n_estimators', 2, 20)\n",
    "    max_depth = int(trial.suggest_int('max_depth', 1, 32))\n",
    "    criterion = trial.suggest_categorical('criterion', [\"gini\", \"entropy\"])\n",
    "    random_state = 13\n",
    "    \n",
    "    clf = RandomForestClassifier(n_estimators=n_estimators, max_depth=max_depth, criterion=criterion, random_state=random_state)\n",
    "\n",
    "    acc_scorer = make_scorer(accuracy_score)\n",
    "    cv_res = cross_val_score(clf, X, y, n_jobs=-1, cv=5, scoring=acc_scorer)\n",
    "\n",
    "    return np.mean(cv_res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1529b9f-7e23-40aa-9215-d1b72cc34249",
   "metadata": {},
   "outputs": [],
   "source": [
    "study = optuna.create_study(direction='maximize', storage=\"sqlite:///db.sqlite3\", study_name=\"Iris-RF-Tuning\")\n",
    "study.optimize(lambda trial: objective(trial, X, y), n_trials=100)\n",
    "\n",
    "trial = study.best_trial\n",
    "\n",
    "print('Accuracy: {}'.format(trial.value))\n",
    "print(\"Best hyperparameters: {}\".format(trial.params))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b6bb2be0-3ff4-495f-8eb1-1f24337778aa",
   "metadata": {},
   "source": [
    "## 💡 Dashboard\n",
    "* Logs are hard to read - it is usually better to vizualize the tuning process\n",
    "* You have two options with `Optuna`\n",
    "    * You can use the basic online tool https://optuna.github.io/optuna-dashboard/\n",
    "    * You can run local instance of https://github.com/optuna/optuna-dashboard for more advanced usage\n",
    "\n",
    "![meme03](https://github.com/rasvob/VSB-FEI-Fundamentals-of-Machine-Learning-Exercises/blob/master/images/fml_12_meme_03.jpg?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fda650ac-7511-43f4-9be3-2ff17bedb90a",
   "metadata": {},
   "source": [
    "# ⚡ Model deploy\n",
    "* How are ML/DL models used in production?\n",
    "    * Do we train it every time from scratch?\n",
    "* How would you deploy the model?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7224173-ac8e-4fc9-8e1e-3fc7cb2f00ac",
   "metadata": {},
   "source": [
    "## Train the model on full data with the best parameter setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d022cd5-4b1d-4c7b-bd70-176f7062a383",
   "metadata": {},
   "outputs": [],
   "source": [
    "params = study.best_trial.params\n",
    "params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4575f722-9c7b-416c-b792-2d14fee46de4",
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = RandomForestClassifier(**params, random_state=13)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e9e0168-89ec-4320-91ea-0f6a034d2591",
   "metadata": {},
   "outputs": [],
   "source": [
    "clf.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "596b3423-aaf4-45ad-9dc7-fda138cb9820",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_feat_imp = pd.DataFrame({'Feature': X.columns, 'Importance': clf.feature_importances_}).sort_values(by='Importance')\n",
    "df_feat_imp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7563042b-e133-40a7-9802-c10742b034d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "px.bar(df_feat_imp, y='Feature', x='Importance', orientation='h')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8def1f78-5dad-4493-99db-940346ba5338",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = clf.predict(X)\n",
    "accuracy_score(y_true=y, y_pred=y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c47fdbc-8ea3-4046-af7d-f3b7ddc29820",
   "metadata": {},
   "source": [
    "# Save the model using `joblib`\n",
    "* There are other alternatives\n",
    "    * https://skops.readthedocs.io/en/stable/\n",
    "    * https://onnx.ai/sklearn-onnx/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2c9a4f19-dc4b-492d-a2b7-d652458673a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "filename = 'rf_best.bin'\n",
    "joblib.dump(clf, filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97152003-c58f-4dfd-9cff-d05cf7cdd489",
   "metadata": {},
   "source": [
    "# 📈 Load the model from disk "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0dd8cc52-1110-4cd2-879a-53460beac8c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "loaded_model = joblib.load(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d1a352a-7f13-4623-9aa9-d060c63dca8e",
   "metadata": {},
   "source": [
    "## Check if everything works fine 🙂"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "197b1b04-94e0-4159-8c1e-d4970065adc9",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = loaded_model.predict(X)\n",
    "accuracy_score(y_true=y, y_pred=y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ba74a9a-e7c2-4721-a263-53f606ca4015",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_feat_imp = pd.DataFrame({'Feature': X.columns, 'Importance': loaded_model.feature_importances_}).sort_values(by='Importance')\n",
    "df_feat_imp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7bc4852-33c0-485a-81d4-ed327f0e70e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "px.bar(df_feat_imp, y='Feature', x='Importance', orientation='h')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6523828-fe6f-44a6-a23e-3702782d8cd1",
   "metadata": {},
   "source": [
    "![meme04](https://github.com/rasvob/VSB-FEI-Fundamentals-of-Machine-Learning-Exercises/blob/master/images/thats_all.jpg?raw=true)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}