Example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PySuggestify\n",
    "\n",
    "## Example of building recommendation system for movies\n",
    "\n",
    "At first, import necessary modules.\n",
    "\n",
    "From PySuggestify we import class PMF, which is simple implementation of Probabilitic Matrix Factorization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "executionInfo": {
     "elapsed": 962,
     "status": "ok",
     "timestamp": 1687523322039,
     "user": {
      "displayName": "Mateusz Soczewka",
      "userId": "14486733780861258567"
     },
     "user_tz": -120
    },
    "id": "-zdrOUQsm-OK"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "from pysuggestify.PMF import PMF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's load the data.\n",
    "\n",
    "The dataframe contains the movie ID, user ID and rating of the movie by the given user on a scale of 1-10."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>47</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>50</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userId  movieId  rating\n",
       "0       1        1     8.0\n",
       "1       1        3     8.0\n",
       "2       1        6     8.0\n",
       "3       1       47    10.0\n",
       "4       1       50    10.0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings = pd.read_csv('data/ratings.csv')\n",
    "ratings.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at what our dataframe contains.<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Number of users: 610\n",
      "Number of movies: 9724\n"
     ]
    }
   ],
   "source": [
    "print(' Number of users:', np.unique(ratings['userId']).shape[0])\n",
    "print('Number of movies:', np.unique(ratings['movieId']).shape[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have **610 users** who can rate **9724 movies**. Of course, each user only saw some of them.\n",
    "\n",
    "Let's assume that we are the owners of a streaming service and we want to offer our users the most suitable movies. How to do it?\n",
    "\n",
    "We can use popular algorithm named **Probability Matrix Factorization**!\n",
    "\n",
    "If you created a matrix of movie ratings by users, it would be very large and mostly filled with gaps (rather no one is able to watch all movies). In PMF, the idea is to factorize or break down the user-item matrix into two smaller matrices: one for users and another for items. This is like breaking down the big matrix into smaller pieces to extract useful information.<br>\n",
    "By doing this factorization, PMF tries to find hidden or latent space in the data. These latent space capture underlying patterns or characteristics that influence user preferences. For example, in a movie recommendation system, these latent features could represent genres like action, romance, or comedy.\n",
    "\n",
    "Using this algorithm, we can *fill gaps* in our rating matrix and thus find out what is **the expected rating of a given movie is for each user**.\n",
    "\n",
    "Let's see how we can use PySuggestify for that!\n",
    "\n",
    "At first, let's create instance od `PMF` class.<br>\n",
    "We need to set the dimension of latent space and the regularization parameter befor we start training our model.<br>\n",
    "The larger the latent space dimension, the more accurate results can be obtained. We use `n_dims = 40`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "PMF_model = PMF(n_dims = 40, lambda_U = 0.3, lambda_V = 0.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we use the `prepare_data()` method, the data for training the model will be automatically prepared.\n",
    "We only need to provide a dataframe as `ratings` in the example above, and indicate which columns are ID for users and items and which column contains ratings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Rating matrix prepared!\n",
      "Number of users:  610\n",
      "Number of items:  9724\n"
     ]
    }
   ],
   "source": [
    "PMF_model.prepare_data(ratings, 'userId', 'movieId', 'rating')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have the data ready, we can start training the model.\n",
    "\n",
    "The only thig is to specify the number of iterations of the algorithm. The higher the number, the more accurate the results will be, but it will take more time to get them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50 \n",
      "  Time: 6.2s, Log p posteriori: -624735.1215, RMSE: 2.8234\n",
      "Epoch 2/50 \n",
      "  Time: 6.3s, Log p posteriori: -141739.1549, RMSE: 1.1257\n",
      "Epoch 3/50 \n",
      "  Time: 6.4s, Log p posteriori: -93051.3433, RMSE: 0.8168\n",
      "Epoch 4/50 \n",
      "  Time: 6.5s, Log p posteriori: -73380.8777, RMSE: 0.6839\n",
      "Epoch 5/50 \n",
      "  Time: 6.5s, Log p posteriori: -62104.7708, RMSE: 0.6062\n",
      "Epoch 6/50 \n",
      "  Time: 6.8s, Log p posteriori: -54637.9614, RMSE: 0.5539\n",
      "Epoch 7/50 \n",
      "  Time: 6.4s, Log p posteriori: -49238.6703, RMSE: 0.5157\n",
      "Epoch 8/50 \n",
      "  Time: 6.4s, Log p posteriori: -45120.9461, RMSE: 0.4864\n",
      "Epoch 9/50 \n",
      "  Time: 6.4s, Log p posteriori: -41886.7305, RMSE: 0.463\n",
      "Epoch 10/50 \n",
      "  Time: 6.5s, Log p posteriori: -39281.0396, RMSE: 0.4438\n",
      "Epoch 11/50 \n",
      "  Time: 6.4s, Log p posteriori: -37123.1818, RMSE: 0.4278\n",
      "Epoch 12/50 \n",
      "  Time: 6.4s, Log p posteriori: -35298.2562, RMSE: 0.4141\n",
      "Epoch 13/50 \n",
      "  Time: 7.5s, Log p posteriori: -33732.5289, RMSE: 0.4025\n",
      "Epoch 14/50 \n",
      "  Time: 7.6s, Log p posteriori: -32372.5318, RMSE: 0.3923\n",
      "Epoch 15/50 \n",
      "  Time: 7.6s, Log p posteriori: -31176.5332, RMSE: 0.3834\n",
      "Epoch 16/50 \n",
      "  Time: 7.6s, Log p posteriori: -30113.1202, RMSE: 0.3756\n",
      "Epoch 17/50 \n",
      "  Time: 7.6s, Log p posteriori: -29159.2776, RMSE: 0.3685\n",
      "Epoch 18/50 \n",
      "  Time: 7.5s, Log p posteriori: -28297.4533, RMSE: 0.3622\n",
      "Epoch 19/50 \n",
      "  Time: 7.6s, Log p posteriori: -27513.2225, RMSE: 0.3564\n",
      "Epoch 20/50 \n",
      "  Time: 7.7s, Log p posteriori: -26794.4888, RMSE: 0.3511\n",
      "Epoch 21/50 \n",
      "  Time: 7.7s, Log p posteriori: -26131.3169, RMSE: 0.3462\n",
      "Epoch 22/50 \n",
      "  Time: 7.6s, Log p posteriori: -25515.7377, RMSE: 0.3417\n",
      "Epoch 23/50 \n",
      "  Time: 7.6s, Log p posteriori: -24941.4293, RMSE: 0.3375\n",
      "Epoch 24/50 \n",
      "  Time: 7.6s, Log p posteriori: -24403.3675, RMSE: 0.3336\n",
      "Epoch 25/50 \n",
      "  Time: 7.5s, Log p posteriori: -23897.5226, RMSE: 0.33\n",
      "Epoch 26/50 \n",
      "  Time: 7.7s, Log p posteriori: -23420.5601, RMSE: 0.3265\n",
      "Epoch 27/50 \n",
      "  Time: 7.6s, Log p posteriori: -22969.5466, RMSE: 0.3233\n",
      "Epoch 28/50 \n",
      "  Time: 7.6s, Log p posteriori: -22541.8788, RMSE: 0.3202\n",
      "Epoch 29/50 \n",
      "  Time: 7.6s, Log p posteriori: -22135.3719, RMSE: 0.3173\n",
      "Epoch 30/50 \n",
      "  Time: 7.7s, Log p posteriori: -21748.2575, RMSE: 0.3146\n",
      "Epoch 31/50 \n",
      "  Time: 7.7s, Log p posteriori: -21379.0883, RMSE: 0.312\n",
      "Epoch 32/50 \n",
      "  Time: 7.6s, Log p posteriori: -21026.6291, RMSE: 0.3095\n",
      "Epoch 33/50 \n",
      "  Time: 7.6s, Log p posteriori: -20689.776, RMSE: 0.3071\n",
      "Epoch 34/50 \n",
      "  Time: 7.6s, Log p posteriori: -20367.5189, RMSE: 0.3048\n",
      "Epoch 35/50 \n",
      "  Time: 7.6s, Log p posteriori: -20058.9304, RMSE: 0.3026\n",
      "Epoch 36/50 \n",
      "  Time: 7.6s, Log p posteriori: -19763.1579, RMSE: 0.3005\n",
      "Epoch 37/50 \n",
      "  Time: 7.6s, Log p posteriori: -19479.4082, RMSE: 0.2984\n",
      "Epoch 38/50 \n",
      "  Time: 7.6s, Log p posteriori: -19206.9318, RMSE: 0.2965\n",
      "Epoch 39/50 \n",
      "  Time: 7.6s, Log p posteriori: -18945.0143, RMSE: 0.2946\n",
      "Epoch 40/50 \n",
      "  Time: 7.6s, Log p posteriori: -18692.9746, RMSE: 0.2928\n",
      "Epoch 41/50 \n",
      "  Time: 7.6s, Log p posteriori: -18450.1693, RMSE: 0.2911\n",
      "Epoch 42/50 \n",
      "  Time: 7.8s, Log p posteriori: -18215.9983, RMSE: 0.2894\n",
      "Epoch 43/50 \n",
      "  Time: 7.7s, Log p posteriori: -17989.9107, RMSE: 0.2878\n",
      "Epoch 44/50 \n",
      "  Time: 7.6s, Log p posteriori: -17771.4079, RMSE: 0.2862\n",
      "Epoch 45/50 \n",
      "  Time: 7.6s, Log p posteriori: -17560.0448, RMSE: 0.2847\n",
      "Epoch 46/50 \n",
      "  Time: 7.6s, Log p posteriori: -17355.4269, RMSE: 0.2833\n",
      "Epoch 47/50 \n",
      "  Time: 7.6s, Log p posteriori: -17157.2056, RMSE: 0.2819\n",
      "Epoch 48/50 \n",
      "  Time: 7.6s, Log p posteriori: -16965.0694, RMSE: 0.2805\n",
      "Epoch 49/50 \n",
      "  Time: 7.5s, Log p posteriori: -16778.7351, RMSE: 0.2792\n",
      "Epoch 50/50 \n",
      "  Time: 8.0s, Log p posteriori: -16597.9387, RMSE: 0.278\n"
     ]
    }
   ],
   "source": [
    "PMF_model.fit(n_epochs=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The learning process can be verified on charts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA88AAAGDCAYAAAAVjiL9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABXZ0lEQVR4nO3deZxcVZn/8e9TS3dV0kk32XcSSBDCkgCRRUQRVIIbqCjghoCDPwW3cVQcdfSn4owyLsPIqPyUERRZxAVUEFkERWUJGJawhs0kZN+T7q6u5fn9cU91Vzq9Jt19q7s+79erXnXvuafuPXV7OfXUOfe55u4CAAAAAADdS8TdAAAAAAAAqh3BMwAAAAAAvSB4BgAAAACgFwTPAAAAAAD0guAZAAAAAIBeEDwDAAAAANALgmcAe8XM/tXMfrgXr19mZicMXIt6PJab2dw9fO0LZvbabrYdb2ZPdVV3b88PAAAAqgPBM9CFngKlkcTMfmxmX92bfbj719z9A3vx+oPd/a69aUPc3P3P7v6ybra1nx8zmx0C+NTQthAAakPov1vMbIeZrQn9XMMA7LfX/jL8f19X+T/ezNKhzPe2Df1lZnVm9k0zWxnOxwtm9p2hbkd/VfSVOzo9zoi7bQDBM4A9tjdB4EAHkASkAIDgze7eIGmhpMMlfXYIj71Z0ikV66eEsjh8VtIiSUdJGiPpBEkPDXUj9qJ/bnL3horHdd3sP7k3x+PzA/qD4BnoBzOrN7PvmNlL4fEdM6uv2P5pM1sdtn2gp2nCZnaOmT1hZtvN7Dkz+2Avx37BzD5rZo+b2WYz+18zy1Rs/yczW25mm8zsJjObFsrNzL4dvvneZmaPmtkhZna+pHdL+nT4Rvc3of40M/uFma03s+fN7KMVx/iSmd1gZj81s22S3h/KflpR5y1hKvYWM7vLzA7q9B4+Y2aPSNppZqlepkP/2My+b2a3hfN0t5ntW7HdzewCM3tG0jM9nYcKbwjne4OZXWJmifC6/c3sTjPbGLZdbWZNnV778q7Ov5mdYGYru3kPlefnT+F5Szjnrw7tPLSi/iQzazaziV3tDwDQN+6+RtKtioJoSb32UQeFsi2hzltCeZf9ZTd+Iul9Fevvk3RVZQUzazSzH4XPC6vM7KvlALC3vij0mf9iZo+Y2VYzu67ys0AnL5f0K3d/ySMvuPtVFfs63MweCv3rdWZ2rYXRdTN7v5nd06nd7Z9pzOyNZvb38LlihZl9qaJeeeT4PDP7h6Q7Q/m5Fn3u2Wxmt1b25/0RPht8z8xuNrOdkl7TzeeLfn0e2ZO2oPYQPAP98zlJxyjqiBco+jb385JkZosl/bOk10qaq+gb3p6sk/QmSWMlnSPp22Z2RC+vebekkyXtL+mAimOfKOnfJb1T0lRJL0q6Nrzm9ZJeFeo3hjob3f1ySVdL+kb4RvfNIZD8jaSHJU2XdJKkj5vZyRVtOFXSDZKawuvbmdkBkq6R9HFJEyXdLOk3ZlZXUe0sSW9U9I1yoZf3W37PX5E0QdLSzseUdJqkoyXN7+U8lL1V0TfxR4T3cm65+eG10yQdJGmmpC910Zbdzn8/vCo8l79Nvzu07z0Vdc6SdIe7r+/nvgEAFcxshqKR3+Vhvds+yszSivq/P0iaJOkjkq42s5d11V/2cNhfS3qVmTWZ2T6Sjpd0Y6c6P5ZUUPRZ4XBF/XT58qe+9EXvlLRY0hxJh0l6fzdtuVfSP5vZh83sUDOzinNTF9r6E0njJP1c0tt7eF+d7VT0xUCToj79Q2Z2Wqc6rw7v4WQzO1XSv0p6m6Jz/2dFP4s99S5JFysaUS8H+e2fLyTtp4H/PAIQPAP99G5JX3b3dSG4+b+S3hu2vVPS/7r7Mndv1u6d3S7c/Xfu/mz4NvhuRR328b0c/7vuvsLdNynqNM6qaNcV7v6Qu+cUTdU61sxmS8or6lwOlGTu/oS7r+5m/y+XNNHdv+zube7+nKT/J+nMijp/c/dfu3vJ3Vs6vf4MSb9z99vcPS/pPyVlJb2ios6l4T10fm13fufufwrv63Phfc2s2P7v7r4p7K+n81D29VD/H5K+o3AO3X15aHcu/Gy/pajjr9Td+d8bV0o6q+JDzXsVfZgBAOyZX5vZdkkrFH1R/cVQ3lMfdYykBkn/Efq/OyX9Vv3/P9+qKAg/IzxuCmWSJDObLOkNkj7u7jvdfZ2kbyv0s33siy4No8mbwrEWdtOWf5f0dUV94xJJq8zs7LDtGElpSd9x97y73yDpgb6+SXe/y90fDZ8FHlEUqHZu55fCe2yR9H8U9ddPhED1a5IW9jL6vCGMGpcfB1Vsu9Hd/xKOXz6/lZ8vBuPzCEDwDPTTNEWjmWUvhrLythUV2yqXd2Nmp5jZvWHa7hZFnemEsO0W60iQ8e5u9tn52O3tcvcdkjZKmh4+AHxX0mWS1pnZ5WY2tptm7StpWmVnpeib4sl9fF+d21EK9af38fVdaa8f3tcmdbzvzvvr9jx0U7/9HJrZ5DBlbZVFU9J/qvDz6O21e8Pd75PULOkEMztQ0UjETXu7XwCoYae5e/ka3wPV8b+8pz5qmqQVoazsRe3af/TVVYpGZXebsq2on01LWl3Rz/5A0Wh3X/uiNRXLzYqC/t24e9HdL3P34xSNxl4s6YoQhE6TtMrdKxOZvdjFbrpkZkeb2R8tusRrq6LguKc+c19J/1XxnjcpGmXv6fxOcPemiscT3ey7q7LB+DwCEDwD/fSSog6gbFYok6TVkmZUbKscHd2FRddJ/0LRN6GT3b1J0ZQikyR3P6UiQUblNOXKfVYee5d2mdloSeMlrQr7u9Tdj5Q0X9F040+Fqp2zf66Q9HynzmqMu7+hok5PGUM7t8NCm1f18fVdaX/PFmVMHaeO9915fz2eh877067n8GthX4e6+1hFU6lNu+rutX3V3Xu/MhzvvZJuqPgWHQCwh8Ksrh8r6mulnvuolyTNDJcvlc1SR//Rn77rz4ouHZqsjinFZSsk5bRrYDjW3Q8O2/vSF/Wbu7e4+2WKkpfNV/SZZXrlVG5F77dsp6RR5RUzm9Jplz9T9EXvTHdvlPT9LtpZec5WSPpgp88XWXf/656+pV7KBuPzCEDwDPQgbWaZikdK0bSkz5vZRDObIOnfFH0rLEnXSzrHooQjoyR9oYd910mql7ReUsHMTlF0zVNvLjCzGWY2TtEU5nLmyWvCsReGwPxrku5z9xfM7OXhG+K0os6wVVL5m/W1iq4LKrtf0vaQRCNrZkmLkou9vA9tk6Jz8EYzOykc75OKPiTsaecoRQm+XhmuU/qKpHvdvbtvi7s9DxV1PmVm+4Sp3x9TxzkcI2mHpK1mNl0dXzBU6u7899V6Red+v07lP1V0LfZ7tPsoBQBgz31H0uvMbIF67qPKs4A+bdHtpU6Q9GZ15M3o3F92K4zmvlnSWzqN7CpcNvUHSd80s7FmlrAoSVh5ynNf+qI+MbOPW5TQMmtRAq2zw/7/Lulviq67/mh4v29TlMel7GFJB4f+NKPdL0UbI2mTu7ea2VGKrkHuyfclfdbMDg5tazSzd+zpe+uDwfg8AhA8Az24WVJLxeNLkr6q6LqhRyQ9quiWD1+VJHe/RdKlkv6oKDnJvWE/uc47dvftkj6q6J/7ZkWdTl+m6v5MUaf7nKRnK459u6Jg/ReKvk3eXx3XKY9VdN3yZkVTmDZKuiRs+5GiRFtbzOzX7l5UlMRsoaTnJW2Q9ENFicZ65e5PKQoA/zu89s2KbhnS1pfXd+Nniq5X2yTpSO2aXKvz8Xs6D2U3SnpQUfKx3yk6B1J0/foRkraG8l9205bdzn9feXQt/MWS/hLO+TGhfIWi3yVXNGIBABgA4brhqyT9W099VOin3qwowdgGSf8j6X3u/mTY1S79ZR+Ou8zdl3Wz+X2KvkR/XFHffIOikWqpb31RXzVL+qaiad4bJF0g6e3u/lx4v29TlGxsk6JrhNuP5e5PS/qypNsV3c2i8wj6hyV9OVxb/m+KPs90y91/pej662vDdPTHtOstvbpSvjNF+fHPvb/l9uMNxucRQNbpCzEAAyRcU/SYpHofgCyOZvaCpA+EALEmmNmPJa109/5mtR52zOwKSS/VwnsFAFSfWupzgT3FyDMwgMzsrRbdC3ofRd+w/mYgAmeMbBZlA3+bOkbBAQAAUGUInoGB9UFFt8V4VlJR0ofibQ6qnZl9RdEMhUvc/fm42wMAAICuMW0bAAAAAIBeMPIMAAAAAEAvCJ4BAAAAAOhFKu4GDCcTJkzw2bNnx90MAMAI8eCDD25w94lxt2M4o28GAAyknvpmgud+mD17tpYsWRJ3MwAAI4SZvRh3G4Y7+mYAwEDqqW9m2jYAAAAAAL2o6eDZzBab2VNmttzMLoq7PQAAAACA6lSzwbOZJSVdJukUSfMlnWVm8+NtFQAAAACgGtXyNc9HSVru7s9JkpldK+lUSY/3Zyf5fF4rV65Ua2vrIDSxemUyGc2YMUPpdDrupgAAAADYA7Uay0h7Fs/UcvA8XdKKivWVko7uXMnMzpd0viTNmjVrt52sXLlSY8aM0ezZs2Vmg9TU6uLu2rhxo1auXKk5c+bE3RwAAAAAe6AWYxlpz+OZmp223Vfufrm7L3L3RRMn7p6xvLW1VePHj6+pXzYz0/jx42vyGyoAAABgpKjFWEba83imloPnVZJmVqzPCGX9Vmu/bFJtvmcAAABgpKnVz/V78r5rOXh+QNI8M5tjZnWSzpR0U8xtAgAAAICakUwmtXDhQh1yyCF685vfrC1btkiSXnjhBZmZPv/5z7fX3bBhg9LptC688EJJ0lNPPaUTTjhBCxcu1EEHHaTzzz9fknTXXXepsbFRCxcubH/cfvvte93Wmg2e3b0g6UJJt0p6QtL17r4s3lbtmYaGhribAAAAAAD9ls1mtXTpUj322GMaN26cLrvssvZtc+bM0e9+97v29Z///Oc6+OCD29c/+tGP6hOf+ISWLl2qJ554Qh/5yEfatx1//PFaunRp++O1r33tXre1ZoNnSXL3m939AHff390vjrs9AAAAAFCrjj32WK1a1XEl7ahRo3TQQQdpyZIlkqTrrrtO73znO9u3r169WjNmzGhfP/TQQwe1fbWcbXvA/d/fLNPjL20b0H3OnzZWX3zzwb1XVJQ17tOf/rRuueWW9ikOZ5xxhkqlki688ELdeeedmjlzptLptM4991ydfvrpXe5n9uzZeuc736lbbrlF2WxWP/vZzzR37tyBfFsAAAAAqkjcsUyxWNQdd9yh8847b5fyM888U9dee60mT56sZDKpadOm6aWXXpIkfeITn9CJJ56oV7ziFXr961+vc845R01NTZKkP//5z1q4cGH7fn7xi19o//3336v3Q/A8gvzyl7/U0qVL9fDDD2vDhg16+ctfrle96lX6y1/+ohdeeEGPP/641q1bp4MOOkjnnntuj/tqbGzUo48+qquuukof//jH9dvf/naI3gUA7B13V8mlYslVclex5Cq6q1iMnkvl9ZKrVFL7cvlR+ZpSqXJZ7WVHzNpHjaO4z/1IsbUlr4f+sVmHTW/U+Ib6uJsDADWlpaVFCxcu1KpVq3TQQQfpda973S7bFy9erC984QuaPHmyzjjjjF22nXPOOTr55JP1+9//XjfeeKN+8IMf6OGHH5YUTdse6BiG4HkA9fVblcFyzz336KyzzlIymdTkyZP16le/Wg888IDuueceveMd71AikdCUKVP0mte8ptd9nXXWWe3Pn/jEJwa76QD2krurUHIViq5CqRSeo8AvXyypWOpYL5RKoXzX9UIpCjAr6xXa65QD0FLF9o7nUvt6ScWSVCx11CuWOu/DdwtYC6WSSiVFbfHw+mJFIFsRwBbCMUruKhRL7YFye5Bc8kE/37/40LE6ct9xg34cDI3n1u/QOf/7gP73nJfrNS+bFHdzACAWccUy5Wuem5ubdfLJJ+uyyy7TRz/60fbtdXV1OvLII/XNb35Tjz/+uG66adccz9OmTdO5556rc889V4cccogee+yxQWsrwTO6VJm6vVbT16M2lEqufKmkfDEKxNqK0XK+UFKhVFJbIQo+y8uFUkn5cp1iFOCV1wulyvKOsqhOx37Kx8q3B6u7vr5Q3DW4zVcEsfn2beG5IgCNUzJh0cNMqYQpmYyeE2E9kQjl7Y+EkglF9ZMJJS0qr0ul2uskKvZV3l4+RiJhSiakVCIR1UtG9ZMJRfsOy4lEp9cmyvV23U90rET7cjK0ufza9teE7fMmj4n1fGNgZdJJSVIuX4y5JQBQu0aNGqVLL71Up512mj784Q/vsu2Tn/ykXv3qV2vcuF2/uP7973+vk046Sel0WmvWrNHGjRs1ffp0Pfnkk4PSRoLnEeT444/XD37wA5199tnatGmT/vSnP+mSSy5RLpfTlVdeqbPPPlvr16/XXXfdpXe961097uu6667TRRddpOuuu07HHnvsEL0DjGTFkqutUFKuUAzP0aOyrK0YrZeXc/mScsWS8p225YvhtWFbvlje7ruUReWutkKxPXjNF8v7KAezgxt0JkPQmE4mlE5GgWI6ET2nQoCZSnRsSyVMo+qiADKdDAFd0pQOAWdULyovB5bpRKL9OKlwnMr1yuVyANvxHLWjvU55X8nd67WvJ3ctTxhfsmF4y4bguYXgGQBidfjhh+uwww7TNddco+OPP769/OCDD94ly3bZH/7wB33sYx9TJpORJF1yySWaMmWKnnzyyd2uef785z/fbc6nviJ4HkHe+ta36m9/+5sWLFggM9M3vvENTZkyRW9/+9t1xx13aP78+Zo5c6aOOOIINTY29rivzZs367DDDlN9fb2uueaaIXoHGArFkqs1X1RrvqiWfFGt+VL7evtyIVrOFTrKcvmicoWw3MVzZSCcK4TANyy3FQYuSDWT6pIJ1aUSHc+phNLJaD2dSqguaapLJdSQSXWUJ6MAtlw3HeqkEuWycoDb/XIqaapLJtqD0Oj1u25Ph2A0HeokEgSVQLXL1oXgua0Uc0sAoPbs2LFjl/Xf/OY37ctdTcF+//vfr/e///2SpG9961v61re+tVudE044QVu3bh3YhorgeUQo/8KZmS655BJdcsklu2xPJBL6z//8TzU0NGjjxo066qijek3j/qlPfUpf//rXB63N6J67K1coqbmtqJ25QvTcVlBzLnpuaSuqua2o5vJyvhjKorrloLhcr3K9NR+N0O6pTDqh+lSy/bk+lVAmnVRdKqFMOqHGbFr1qYTqQ0Bbn+rYVpdMqj4dBbIdz8noOdURBFcGxF1tSyVr+g57AAZBhpFnAEAfEDzXiDe96U3asmWL2tra9IUvfEFTpkyJu0kjTqFY0o5cQdtaCtrWmteOXEE7WgvansuH52h9R8XzzraCduSiIHlnLiprbiv26/rVdNKUTSc1qi6lbF1S2XRS2bqkRtenNL6hPloPZZmwnEknovVUUpm6pDIhCM6EbZl02JaOAtxMCHaZmgtgJMqkoy/lWgmeAQA9IHiuEXfdddduZW9961v1/PPP71L29a9/XS+88MLQNKoKlUquba15bW7Oa3Nzm7Y257W1Ja8tzW3a0hItl8u2tebbA+VtLXntbOv9Q1cqYWrIpNRQHz1G16fUmE1relNGo+ui9dH1UeA7OgTDo+tSGlUfnuuS4RFtG1WXVJqRWADYK3XJ6Np9gmcAQE8InmvYr371q7ibMOhKJdfWlrw27Mhp/Y6cNu5o04bwvHFnTpt2tmnzzrw2Nbdp8842bW5uU0+DvmPqUxqbTasxPPYdP0pjs2mNzaQ1NpsKz2k11Kc0NpNSQyalMZlofUwmpfoUo7cAUG3Mohk8LX34EhQARhp3r8nPp+79z8dD8DwAavEXbk9+2Qb6+Fua83ppa4vWbctp7bZWrd2W07rtlc+t2rijrctEVcmEadzoOo0bVad9Rqd1wOQG7TOqTuNG16lpVJ3GjU6rKVunxlFpNYVAeWw2zSgvAIxQ2bok1zwDqDmZTEYbN27U+PHjayqecXdt3LixPUt3XxE876Va/IXb01+2/h5j/facXtjYrJWbm/XSlhat2tKqVVta9FJ4NHcxQjBudJ0mjanX5LEZvWzyGE0aW68JDfUa31CvCQ11mhiWm7JpsiADANrVp5JqzZNtG0BtmTFjhlauXKn169fH3ZQhl8lkNGPGjH69huB5L9XqL9ye/LJ15u5atz2nZ9ft0Asbm/Xixp16YeNOvbixWS9ubN5tBGD86DpNa8pq7sQGvWreRE1rymh6U1aTGzOaPDajiQ31qksxMgwA6L9sXZJrngHUnHQ6rTlz5sTdjGGD4Hkv8QvXO3fXhh1tembtdj29drueXrdDT6+Jlre1Ftrr1aUSmjVulGaPH6VX7D9BsyeM0qxxozRz3ChNb8q230oEAICBlk0zbRsA0DOCZwy4TTvb9PDKLXp4RfR4dNVWbdjR1r69MZvWyyaP0ZsXTNMBk8do3qQGzZ4wWlPGZphKDQCIRTbNyDMAoGcEz9grxZLrsVVbdf/zm7R05RY9snKLVmxqkSSZSfMmNeiEl03S/KljdcDkMTpgcoMmjqmvmevDAQDDQ306oR25Qu8VAQA1i+AZ/eLu+semZv35mQ36y/IN+uuzG7W1JS9Jmt6U1YKZjXrP0ftqwcwmHTK9UQ31/IoBAKpfNp3U+u25uJsBAKhiRDboVUtbUXc9tU53P71e9yzfoJWbo5HlaY0ZnXzwZB03d4KO3X+8Jo0ZvOzbAAAMJhKGAQB6Q/CMLuWLJd3zzAbd9PBL+sOyNdrZVtSYTErH7jdeH3zVfjpu7gTNmTCa6dcAgBEhuuaZW1UBALpH8Ix2pZLrgRc26aaHX9LNj67W5ua8GrNpvWXhNL15wTQdNXucUkluBQUAGHkyZNsGAPSC4Bna2pLXlX99Qdfc/w+t3tqqbDqp186frFMXTNOrDpjIvZMBACMewTMAoDcEzzVs446crvjL87rqry9qe66gVx0wURedcqBee9BkjSbRFwCghmTTSbUVSiqWXElumwgA6AIRUg1au61Vl//pOf3svn+otVDUGw6Zqg+/Zn8dPK0x7qYBABCLbF00yypXKGpUHR+PAAC7o3eoISs3N+v7dz+r6x9YqaK7Tl0wTR9+zf6aO2lM3E0DACBWmXRSUnSHCYJnAEBX6B1qgLvryr++oK/d/KRcrtOPnKkPvXp/zRo/Ku6mAQBQFdqDZ657BgB0g+B5hNu8s02fuuER3f7EWp144CR99bRDNK0pG3ezAACoKtkQPHOvZwBAdwieR7B7n9uoj1+7VJt2tunf3jRf5xw3m/syAwDQhY7gmXs9AwC6RvA8AhWKJV1653J9985ntO/40frl2a/QIdNJBgYAQHeYtg0A6A3B8wjz0pYWfezav+uBFzbr7UfM0JdPPZjbTgEA0Itytu2WNoJnAEDXiKpGkHue2aALfvaQiiXXd85YqNMOnx53kwAAGBYYeQYA9IbgeYR4bv0OfeinD2r6Pll9/z1HavaE0XE3CQCAYYOEYQCA3hA8jwA7cwV98CcPKp1K6Ir3v5xs2gAA9FOG4BkA0ItE3A3A3nF3ffoXj+jZ9Tv03bMOJ3AGAGAPlEeeueYZANAdgudh7kf3PK/fPbJan158oF4xd0LczQEAYFjK1pWveeZWVQCArhE8D2N/e3aj/v2WJ7X44Cn64Kv2i7s5AAAMW/Wp6CMR07YBAN0heB6mVm9t0YU/e0izx4/SJe84TGYWd5MAABi2zEyZdILgGQDQLYLnYShXKOpDP31IrfmifvDeIzUmk467SQAADHvZdJJbVQEAukXwPAx9+TePa+mKLbrkHQs0d9KYuJsDAECvzGymmf3RzB43s2Vm9rEu6pxgZlvNbGl4/NtQtjGbTpIwDADQLW5VNcz8fMkKXX3fP/TBV+2nNxw6Ne7mAADQVwVJn3T3h8xsjKQHzew2d3+8U70/u/ubYmifMumkWgskDAMAdI2R52Fk7bZWff7Xj+nY/cbrUye/LO7mAADQZ+6+2t0fCsvbJT0haXq8rdpVhpFnAEAPYguezewdYdpWycwWddr2WTNbbmZPmdnJFeWLQ9lyM7uoonyOmd0Xyq8zs7pQXh/Wl4fts3s7RjX7/WNrlCuU9JXTDlYqyfceAIDhKfTHh0u6r4vNx5rZw2Z2i5kd3M3rzzezJWa2ZP369QPWrmxdkoRhAIBuxRmBPSbpbZL+VFloZvMlnSnpYEmLJf2PmSXNLCnpMkmnSJov6axQV5K+Lunb7j5X0mZJ54Xy8yRtDuXfDvW6PcZgvdGBcuuyNdp/4miucwYADFtm1iDpF5I+7u7bOm1+SNK+7r5A0n9L+nVX+3D3y919kbsvmjhx4oC1jYRhAICexBY8u/sT7v5UF5tOlXStu+fc/XlJyyUdFR7L3f05d2+TdK2kUy26R9OJkm4Ir79S0mkV+7oyLN8g6aRQv7tjVK3NO9t03/ObdPLBU+JuCgAAe8TM0ooC56vd/Zedt7v7NnffEZZvlpQ2swlD1T5uVQUA6Ek1zv2dLmlFxfrKUNZd+XhJW9y90Kl8l32F7VtD/e72tZvBmhrWX3c8uU7FkhM8AwCGpfDl9Y8kPeHu3+qmzpRQT2Z2lKLPKRuHqo0ZRp4BAD0Y1GzbZna7pK6ivc+5+42DeeyB4u6XS7pckhYtWuRxteP3j63R1MaMDpvRGFcTAADYG8dJeq+kR81saSj7V0mzJMndvy/pdEkfMrOCpBZJZ7r7kPW92XRSrSQMAwB0Y1CDZ3d/7R68bJWkmRXrM0KZuinfKKnJzFJhdLmyfnlfK80sJakx1O/pGFWnua2gPz+zXmcdNUvhC3kAAIYVd79HUo+dmLt/V9J3h6ZFu8vWMfIMAOheNU7bvknSmSFT9hxJ8yTdL+kBSfNCZu06RQm/bgrfSP9R0bfVknS2pBsr9nV2WD5d0p2hfnfHqEp3P7VeuUJJrz94ctxNAQBgxMqkk2rNc59nAEDXBnXkuSdm9lZFmTQnSvqdmS1195PdfZmZXS/pcUkFSRe4ezG85kJJt0pKSrrC3ZeF3X1G0rVm9lVJf1d0TZXC80/MbLmkTYoCbvV0jGp067I12mdUWkfNHhd3UwAAGLHK1zy7OzO9AAC7iS14dvdfSfpVN9sulnRxF+U3S7q5i/Ln1EW2bHdvlfSO/hyj2rQVSrrjyXU6+eAp3NsZAIBBlE1Hd63MFUrKpKv+DpYAgCFGNFbl7n1uo7a3FrSYLNsAAAyqbDr6WNRC0jAAQBcInqvc75et0ai6pF45b8hucwkAQE0qjza3FgieAQC7I3iuYqWS67bH1+qEl01k+hgAAIMsWxf1tYw8AwC6QvBcxf6+YrPWb8/pZKZsAwAw6MpfVHO7KgBAVwieq9ity9YqnTS95sBJcTcFAIARr5wwrJXgGQDQBYLnKuXuunXZGh27/wSNzaTjbg4AACNe+zXP3OsZANAFgucq9dTa7XpxYzNZtgEAGCLlkWeueQYAdIXguUr9/rE1MpNeN39y3E0BAKAmZOvCraqYtg0A6ALBc5W6ddlaHTlrH00cUx93UwAAqAkkDAMA9ITguQqt2NSsJ1ZvI8s2AABDqBw85wieAQBdIHiuQrcuWyNJBM8AAAyhLCPPAIAeEDxXoVuXrdFBU8dq1vhRcTcFAICa0T5tu41s2wCA3RE8V5n123Na8uJmnXwwicIAABhKyYSpLpVg5BkA0CWC5ypz2+Nr5c6UbQAA4pBJJdRK8AwA6ALBc5X523MbNa0xowOnjIm7KQAA1JxsXZLgGQDQJYLnKrMzV9D4hnqZWdxNAQCg5mTTSaZtAwC6RPBcZXKFoupT/FgAAIhDJp1USxvBMwBgd0RpVaY1X1J9mh8LAABxyKSTai2QbRsAsDuitCoTjTwn424GAAA1KZtOqpWRZwBAFwieq0wuX2LaNgAAMcnWcc0zAKBrRGlVJlcoKZNm5BkAgDhk0tznGQDQNYLnKkPCMAAA4pNJc6sqAEDXiNKqTK7AtG0AAOKSJXgGAHSDKK3KtOaLqmfaNgAAschyqyoAQDcInquIuzPyDABAjDLpKGGYu8fdFABAlSFKqyL5ostdJAwDACAm2bqkSh71yQAAVCJ4riK5QjRNjJFnAADiUf4Cm4zbAIDOiNKqSK5QkkTwDABAXLIheCZpGACgM6K0KlLuqOtTTNsGACAOmXT00YjgGQDQGcFzFWkfeU7zYwEAIA5Zpm0DALpBlFZFcnmmbQMAEKdMXQieuV0VAKATorQq0p4wjGzbAADEgpFnAEB3CJ6rCAnDAACIVznbdnk2GAAAZURpVaQjeGbkGQCAODDyDADoDsFzFenIts2PBQCAOLQHz1zzDADohCitipRHnjNk2wYAIBaZuqgPZuQZANAZUVoVyXGfZwAAYlW+5pn7PAMAOiN4riLc5xkAgHhlCZ4BAN0gSqsiJAwDACBe6WRCqYQxbRsAsJvYgmczu8TMnjSzR8zsV2bWVLHts2a23MyeMrOTK8oXh7LlZnZRRfkcM7svlF9nZnWhvD6sLw/bZ/d2jDiRMAwAgPhl00m1tHGrKgDAruKM0m6TdIi7HybpaUmflSQzmy/pTEkHS1os6X/MLGlmSUmXSTpF0nxJZ4W6kvR1Sd9297mSNks6L5SfJ2lzKP92qNftMQb5/faK+zwDABC/+nRSrQVGngEAu4otSnP3P7h7IazeK2lGWD5V0rXunnP35yUtl3RUeCx39+fcvU3StZJONTOTdKKkG8Lrr5R0WsW+rgzLN0g6KdTv7hixyhWKqk8lFDURAADEIVuXUCu3qgIAdFItQ5znSrolLE+XtKJi28pQ1l35eElbKgLxcvku+wrbt4b63e0rVrl8iVFnAABilk0nueYZALCb1GDu3MxulzSli02fc/cbQ53PSSpIunow27KnzOx8SedL0qxZswb1WLlCSfXp2GePAwBQ0wieAQBdGdTg2d1f29N2M3u/pDdJOsndPRSvkjSzotqMUKZuyjdKajKzVBhdrqxf3tdKM0tJagz1ezpG5/dwuaTLJWnRokXeVZ2BkssXGXkGACBm9ekkt6oCAOwmzmzbiyV9WtJb3L25YtNNks4MmbLnSJon6X5JD0iaFzJr1ylK+HVTCLr/KOn08PqzJd1Ysa+zw/Lpku4M9bs7RqxyBaZtAwAQt2jkmWzbAIBdxRmpfVfSGEm3mdlSM/u+JLn7MknXS3pc0u8lXeDuxTCqfKGkWyU9Ien6UFeSPiPpn81suaJrmn8Uyn8kaXwo/2dJF/V0jMF+w72JEoYxbRsAMPKY2Uwz+6OZPW5my8zsY13UMTO7NNxK8hEzOyKOtmbTSRKGAQB2M6jTtnsSbh/V3baLJV3cRfnNkm7uovw5dZEt291bJb2jP8eIU65QUibNyDMAYEQqSPqkuz9kZmMkPWhmt7n74xV1TlE0G2yepKMlfS88D6lsHdc8AwB2R6RWRaJs24w8AwBGHndf7e4PheXtimaRdb7TxamSrvLIvYpymkwd4qYqk05wzTMAYDcEz1UkVyiqnpFnAMAIZ2azJR0u6b5Om/p0K0kzO9/MlpjZkvXr1w94+zJk2wYAdIFIrYq0cp9nAMAIZ2YNkn4h6ePuvm1P9uHul7v7IndfNHHixIFtoMI1zwTPAIBOiNSqCAnDAAAjmZmlFQXOV7v7L7uo0udbSQ6mbDqpfNFVKJJxGwDQgeC5ipAwDAAwUpmZKboLxhPu/q1uqt0k6X0h6/Yxkra6++oha2SQSUdfZLcWCJ4BAB1iy7aN3UX3eWbkGQAwIh0n6b2SHjWzpaHsXyXNkiR3/76iO2q8QdJySc2Szhn6ZkqZuqgvbmkrqqGej0oAgAg9QhXJ5Ytc8wwAGJHc/R5J1ksdl3TB0LSoe9nyyDPXPQMAKhCpVZHWQols2wAAxKx8CRUZtwEAlYjUqkShWFKx5EzbBgAgZow8AwC6QvBcJXIhKQkJwwAAiFc5eG5pI3gGAHQgUqsS5eCZkWcAAOLVnjCMkWcAQAWC5yqRK0QdNAnDAACIVybFtG0AwO6I1KpEaz6MPDNtGwCAWGXrysEz93kGAHQgUqsSHSPPTNsGACBO7dc8M/IMAKhA8FwlcuWRZ6ZtAwAQKxKGAQC6QqRWJTqybTPyDABAnOq5zzMAoAsEz1WChGEAAFSH+lRCZlKO4BkAUIFIrUp0TNtm5BkAgDiZmbLpJCPPAIBdEDxXidbyyDPZtgEAiB3BMwCgMyK1KkHCMAAAqkcmnVRLG7eqAgB0IFKrEiQMAwCgemTSifZZYQAASATPVYOEYQAAVI9sXVKt3KoKAFCBSK1KlEeeSRgGAED8uOYZANAZwXOVaA0ddB0jzwAAxC5D8AwA6IRIrUrkCiWlk6ZkwuJuCgAANS+TTqo1T8IwAEAHgucqkcuXmLINAECVyKaT7bPCAACQCJ6rRq5QVIZ7PAMAUBWy6aRaSBgGAKhAtFYlcgVGngEAqBaZdIJrngEAuyB4rhKt+SK3qQIAoEpk6pi2DQDYFdFalcgVSmTaBgCgSmTTSeUKJZVKHndTAABVgmitSuQKJdWnmbYNAEA1yIY+ubXA6DMAIELwXCVy+aIyjDwDAFAVMiF4JmkYAKCMaK1KMPIMAED16Bh55l7PAIAIwXOViLJt8+MAAKAaZOoYeQYA7IporUrkyLYNAEDVaB95JuM2ACAgWqsS3OcZAIDqkUlHH5G41zMAoIzguUrkCsX2jhoAAMSLkWcAQGdEa1Uil2fkGQCAakG2bQBAZwTPVSLKts2PAwCAapAtJwxj5BkAEBCtVYFiydVWJNs2AADVIsO0bQBAJ7FFa2b2FTN7xMyWmtkfzGxaKDczu9TMloftR1S85mwzeyY8zq4oP9LMHg2vudTMLJSPM7PbQv3bzGyf3o4Rh7ZwD0mmbQMAUB06rnnmPs8AgEicQ52XuPth7r5Q0m8l/VsoP0XSvPA4X9L3pCgQlvRFSUdLOkrSF8vBcKjzTxWvWxzKL5J0h7vPk3RHWO/2GHHJFaJvtRl5BgCgOpSDZ6ZtAwDKYovW3H1bxepoSR6WT5V0lUfuldRkZlMlnSzpNnff5O6bJd0maXHYNtbd73V3l3SVpNMq9nVlWL6yU3lXx4hFLow8l6eIAQBQjczsxIrlOZ22vW3oWzR4yl9okzAMAFAW61CnmV1sZiskvVsdI8/TJa2oqLYylPVUvrKLckma7O6rw/IaSZN7OUYscvnytG1GngEAVe0/K5Z/0Wnb54eyIYMtkTDVpxJqLRA8AwAifY7WzGw/M/uNmW0ws3VmdqOZ7dfLa243s8e6eJwqSe7+OXefKelqSRfu3VvpWRiV9l4rdmJm55vZEjNbsn79+kFoWcW0bbJtAwCqm3Wz3NX6sJetS6qVkWcAQNCfaO1nkq6XNEXSNEk/l3RNTy9w99e6+yFdPG7sVPVqSW8Py6skzazYNiOU9VQ+o4tySVpbno4dntf1coyu3sPl7r7I3RdNnDixp7e7x1rzJAwDAAwL3s1yV+vDXjad5JpnAEC7/gTPo9z9J+5eCI+fSsrs6YHNbF7F6qmSngzLN0l6X8iIfYykrWHq9a2SXm9m+4REYa+XdGvYts3MjglZtt8n6caKfZWzcp/dqbyrY8SChGEAgGFiPzO7ycx+U7FcXp/T24uHm0w6qRaybQMAglQ/6t5iZhdJulbRt8tnSLo5ZMGWu2/q57H/w8xeJqkk6UVJ/yeU3yzpDZKWS2qWdE55/2b2FUkPhHpfrjjmhyX9WFJW0i3hIUn/Iel6MzsvHOOdPR0jLiQMAwAME6dWLP9np22d14e9TDrJfZ4BAO36EzyXA88Pdio/U1Ew3eP1z525+9u7KXdJF3Sz7QpJV3RRvkTSIV2Ub5R0Un+OEQdGngEAw4G73125bmZpRf3vKndf1/Wrhq9sOkHwDABo1+fg2d17nI5lZq9z99v2vkm1pz3bNgnDAABVzMy+L+m/3X2ZmTVK+pukoqRxZvYv7t5jLpThJluX5FZVAIB2AxmtfX0A91VTWttHnpm2DQCoase7+7KwfI6kp939UElHSvp0fM0aHJkUCcMAAB0GMngecbeoGCrc5xkAMEy0VSy/TtKvJcnd18TSmkGWqeOaZwBAh4GM1kbcLSqGSjlhGMEzAKDKbTGzN5nZ4ZKOk/R7STKzlKKknSNKNp1sv50kAAD9SRiGQVJOGEa2bQBAlfugpEslTZH08YoR55Mk/S62Vg0S7vMMAKg0kMHzCwO4r5rCtG0AwHDg7k9LWtxF+a2Sbh36Fg2uTDpBwjAAQLs+B89m9rYuirdKetTd17l7V9vRB62FopIJUypJ8AwAqF5mdmlP2939o0PVlqGQTSfVWijK3WVGahcAqHX9GXk+T9Kxkv4Y1k+Q9KCkOWb2ZXf/yQC3rWbk8iVGnQEAw8H/kfSYpOslvaR+JAs1syskvUnSOnc/pIvtJ0i6UdLzoeiX7v7lvWzvXsnUJeUe5Sbh0ioAQH+C55Skg9x9rSSZ2WRJV0k6WtKfJBE876FcgeAZADAsTJX0DklnSCpIuk7SDe6+pQ+v/bGk7yr67NCdP7v7m/ayjQMmGwLm1nyR4BkA0K9s2zPLgXOwLpRtkpQf2GbVllyBThkAUP3cfaO7f9/dX6PoPs9Nkh43s/f24bV/krRpkJs4oMp9M0nDAABS/0ae7zKz30r6eVg/PZSNlrRloBtWSxh5BgAMJ2Z2hKSzFN3r+RZFl3ENhGPN7GFFU8L/xd2XDdB+90jHyDO3qwIA9C94vkDS2yS9MqxfKekX7u6SXjPQDasl0TXPjDwDAKqbmX1Z0hslPSHpWkmfdffCAO3+IUn7uvsOM3uDpF9LmtdNO86XdL4kzZo1a4AOv7v2kWcybgMA1I/g2d3dzO6R1CbJJd0fAmfspdZCUfVpRp4BAFXv84oSei0Ij6+FLNSm6KPCYXu6Y3ffVrF8s5n9j5lNcPcNXdS9XNLlkrRo0aJB+yySrWPaNgCgQ39uVfVOSZdIuktRJ/nfZvYpd79hkNpWM8i2DQAYJuYM1o7NbIqkteHL+qMU5WXZOFjH64tM6JtbCZ4BAOrftO3PSXq5u6+TJDObKOl2SQTPeylXKGp0fX9+FAAADD13f7GrcjNLKLoGusvtoc41im5zOcHMVkr6oqR02O/3FeVS+ZCZFSS1SDoz7hlu5ZFngmcAgNS/4DlRDpyDjepftm50I1coadxoTiUAoLqZ2VhFOVCmS7pJ0m2SLpT0SUkPS7q6u9e6+1k97dvdv6voVlZVI0u2bQBAhf4Ez783s1slXRPWz5B088A3qfZE2bZJGAYAqHo/kbRZ0t8kfUDSvyq6lOs0d18aY7sGBQnDAACV+pMw7FNm9nZJx4Wiy939V4PTrNrSmi9yzTMAYDjYz90PlSQz+6Gk1ZJmuXtrvM0aHJk007YBAB36daGtu/9C0i8GqS01K1cokW0bADAc5MsL7l40s5UjNXCWKq955j7PAIA+BM9mtl3Rral226TothRjB7xVNSaXLzJtGwAwHCwws/ItpUxSNqyPyM8E5WzbXPMMAJD6EDy7+5ihaEgtY+QZADAcuHtNfdObSiZUl0wQPAMAJJEtO3buTsIwAACqVH06QcIwAIAkgufY5QrRdVQkDAMAoPpk00nlCgTPAACC59gRPAMAUL2ydUlGngEAkgieY1f+Nrs+zbRtAACqTTad5JpnAIAkgufY5cLtLzKMPAMAUHXq00m1cKsqAIAInmPXPm2bkWcAAKpONp1QKyPPAAARPMeufdo2I88AAFSdbDpJ8AwAkETwHLvWPAnDAACoVpk0CcMAABEitph1jDwzbRsAgGpDwjAAQBnBc8zK1zxn0vwoAACoNpm6ZPssMQBAbSNii1mufdo2I88AAFQbrnkGAJQRPMes4z7P/CgAAKg2mXRCLfmi3D3upgAAYkbEFrMcCcMAAKha2XRSxZIrXyR4BoBaR8QWMxKGAQBQvTLpqH9uLTB1GwBqHcFzzMoJw5i2DQBA9cnWheCZ21UBQM0jYotZe7ZtRp4BAKg65f6Z21UBAAieY5bLF2UmpZMWd1MAAEAn5ZFngmcAAMFzzFoLJdWnEjIjeAYAoNpky9c8c69nAKh5BM8xy+WLJAsDAKBKlROGtXDNMwDUvNiDZzP7pJm5mU0I62Zml5rZcjN7xMyOqKh7tpk9Ex5nV5QfaWaPhtdcamEY18zGmdltof5tZrZPb8cYarkw8gwAAKpPJiT0bGXaNgDUvFijNjObKen1kv5RUXyKpHnhcb6k74W64yR9UdLRko6S9MVyMBzq/FPF6xaH8osk3eHu8yTdEda7PUYccoVS+7faAACgunDNMwCgLO4hz29L+rQkryg7VdJVHrlXUpOZTZV0sqTb3H2Tu2+WdJukxWHbWHe/191d0lWSTqvY15Vh+cpO5V0dY8jlCkVGngEAqFId1zwTPANArYstajOzUyWtcveHO22aLmlFxfrKUNZT+couyiVpsruvDstrJE3u5RhdtfN8M1tiZkvWr1/fl7fWL7l8iXs8AwBQpcrBMyPPAIDUYO7czG6XNKWLTZ+T9K+KpmwPCXd3M/Pea+72usslXS5JixYt6vfre9NaIGEYAADVqp6EYQCAYFCDZ3d/bVflZnaopDmSHg65vWZIesjMjpK0StLMiuozQtkqSSd0Kr8rlM/oor4krTWzqe6+OkzLXhfKuzvGkMvlSRgGAEC1Yto2AKAslqjN3R9190nuPtvdZyuaNn2Eu6+RdJOk94WM2MdI2hqmXt8q6fVmtk9IFPZ6SbeGbdvM7JiQZft9km4Mh7pJUjkr99mdyrs6xpAj2zYAANUrnTQlE8Z9ngEAgzvyvIdulvQGScslNUs6R5LcfZOZfUXSA6Hel919U1j+sKQfS8pKuiU8JOk/JF1vZudJelHSO3s6RhxyhSLZtgEAqFJmpmw6yTXPAIDqCJ7D6HN52SVd0E29KyRd0UX5EkmHdFG+UdJJXZR3e4yhxsgzAADVLZNOEDwDAGK/VVXNa82TMAwAgGqWSSfVSsIwAKh5BM8xyxW4VRUAANUsm06qtUDwDAC1jqgtZmTbBgCgumXrktyqCgBA8BwndydhGAAAVS6TImEYAIDgOVaFkqvkYuQZAIAqlqlLcqsqAADBc5xyhagjJmEYAADVa9yotNZua427GQCAmBE8x6g1TAEjYRgAANXr0BlNWr21VesIoAGgphG1xahj5JkfAwAA1WrBjEZJ0sMrt8bcEgBAnIjaYpQLI88kDAMAoHodPK1RyYTpkZVb4m4KACBGBM8xYuQZAIDql61L6oDJY7R0xZa4mwIAiBFRW4xIGAYAwPCwcGajHlm5Ve4ed1MAADEheI5Re8IwRp4BAKhqh81o0taWvF7c2Bx3UwAAMSFqi1H7yDPZtgEAqGoLZjRJkh7mumcAqFlEbTHKtY88M20bAIBqdsDkBmXSCT28gozbAFCrCJ5jVB55zjDyDABAVUslEzpkWiMjzwBQw4jaYkTCMAAAho8FM5u07KWtyhdLcTcFABADgucYkTAMAFArzOwKM1tnZo91s93M7FIzW25mj5jZEUPdxt4cNqNRrfmSnl67Pe6mAABiQNQWI0aeAQA15MeSFvew/RRJ88LjfEnfG4I29cvCmU2SpEdWct0zANQigucY5Qph5JlrngEAI5y7/0nSph6qnCrpKo/cK6nJzKYOTev6Zta4UWoaldbDK7bE3RQAQAyI2mKUy5dHnvkxAABq3nRJKyrWV4ayqmFmOmxGk5YSPANATSJqi1GuUFJdKiEzi7spAAAMG2Z2vpktMbMl69evH9JjL5jRqGfW7VBzW2FIjwsAiB/Bc4xyhSKjzgAARFZJmlmxPiOU7cbdL3f3Re6+aOLEiUPSuLIFM5pULLmWvbRtSI8LAIgfkVuMWvMlkoUBABC5SdL7QtbtYyRtdffVcTeqs8NmNkoS1z0DQA1Kxd2AWsbIMwCgVpjZNZJOkDTBzFZK+qKktCS5+/cl3SzpDZKWS2qWdE48Le3ZpDEZTWvM6GEybgNAzSF4jlGuUCLTNgCgJrj7Wb1sd0kXDFFz9sqCmU16ZOWWuJsBABhiRG4xyuVLyjBtGwCAYeWwGU16cWOzNu9si7spAIAhRPAco1yhyMgzAADDzILydc+MPgNATSFyi1EuX+KaZwAAhplDpzfKTHqE654BoKYQucUoShjGtG0AAIaTMZm09p/YQMZtAKgxBM8xyhUYeQYAYDhaMKNJD6/cqijPGQCgFhC5xShXKCmTZuQZAIDhZsHMRm3YkdNLW1vjbgoAYIgQPMcol+c+zwAADEcLZjRJkh5h6jYA1Awitxi1cp9nAACGpQOnjlE6aVpKxm0AqBlEbjGKRp6Ztg0AwHBTn0pq/tSxJA0DgBpC8BwjEoYBADB8HTajSY+t2qZiiaRhAFALiNxiUiiWVCg5CcMAABimFsxs0o5cQc+t3xF3UwAAQ4DgOSZtxZIkMfIMAMAwtXBmoyTp4ZVbY24JAGAoELnFJJcneAYAYDjbb0KDGupTXPcMADWCyC0mrYWiJKmeadsAAAxLiYTp0OmNeoSM2wBQEwieY8LIMwAAw99hMxv1+OptyoUvxQEAI1dskZuZfcnMVpnZ0vB4Q8W2z5rZcjN7ysxOrihfHMqWm9lFFeVzzOy+UH6dmdWF8vqwvjxsn93bMYZKrlAOnhl5BgBguFo4o0n5ouuJ1dvjbgoAYJDFPez5bXdfGB43S5KZzZd0pqSDJS2W9D9mljSzpKTLJJ0iab6ks0JdSfp62NdcSZslnRfKz5O0OZR/O9Tr9hiD/3Y7lL+hzqTj/hEAAIA9deTsfZRKmH7991VxNwUAMMiqMXI7VdK17p5z9+clLZd0VHgsd/fn3L1N0rWSTjUzk3SipBvC66+UdFrFvq4MyzdIOinU7+4YQ4aRZwAAhr9JYzI67fDpuub+f2j99lzczQEADKK4g+cLzewRM7vCzPYJZdMlraioszKUdVc+XtIWdy90Kt9lX2H71lC/u33txszON7MlZrZk/fr1e/Yuu9CaLycMi/tHAAAA9saHT9hf+WJJP/zzc3E3BQAwiAY1cjOz283ssS4ep0r6nqT9JS2UtFrSNwezLXvK3S9390XuvmjixIkDtl8ShgEAMDLsN7FBbzpsmn5y74vatLMt7uYAAAbJoEZu7v5adz+ki8eN7r7W3YvuXpL0/9QxbXqVpJkVu5kRyror3yipycxSncp32VfY3hjqd7evIcO0bQAARo4LT5yr5rai/vcvz8fdFADAIIkz2/bUitW3SnosLN8k6cyQKXuOpHmS7pf0gKR5IbN2naKEXze5u0v6o6TTw+vPlnRjxb7ODsunS7oz1O/uGEOGhGEAAIwcB0weo8UHT9GP//KCtrbk424OAGAQxBm5fcPMHjWzRyS9RtInJMndl0m6XtLjkn4v6YIwQl2QdKGkWyU9Ien6UFeSPiPpn81suaJrmn8Uyn8kaXwo/2dJF/V0jMF+w5UYeQYAYGS58MS52p4r6Kq/vhB3UwAAgyDVe5XB4e7v7WHbxZIu7qL8Zkk3d1H+nLrIlu3urZLe0Z9jDJX2hGFc8wwAwIhwyPRGnXjgJP3oL8/rnFfOUUN9bB+zAACDgMgtJu0jz0zbBgBgxLjwxLna0pzX1fe+GHdTAAADjMgtJh3Ztpm2DQDASHHErH30yrkT9P/+/Jxa2ob0ijAAwCAjeI5JrlBUOmlKJizupgAAgAH0kRPnasOONl37wD/ibgoAYAARPMckVygx6gwAwAh09H7jddSccfrB3c+1310DADD8ETzHJFcokiwMAIAR6iMnztWaba264cGVcTcFADBAiN5i0povETwDADBCvXLuBC2c2aTv3fWs8sVS3M0BAAwAoreY5Aol1aeZtg0AwEhkZvrIiXO1cnOLfv33VXE3BwAwAAieY5LLM20bAICR7MQDJ2n+1LH6zu3PaNPOtribAwDYS0RvMWHkGQCAkc3M9LW3Har1O3L60E8fVFuB6dsAMJwRPMeEhGEAAIx8C2c26ZLTD9N9z2/SF296TO4ed5MAAHsoFXcDalVrvqQxGU4/AAAj3akLp+vptdt12R+f1bxJY3TuK+fE3SQAwB5g6DMm3OcZAIDa8cnXvUyvnz9ZX/3d47rrqXVxNwcAsAcInmOSKxRVn+b0AwBQCxIJ07fPWKiXTRmrj/zs71q+bkfcTQIA9BPRW0xy+ZIyjDwDAFAzRten9MOzF6k+ndAHrnxAW5rJwA0AwwnBc0yibNucfgAAasn0pqx+8N5FemlLqz589UPKF8nADQDDBdFbTLjPMwAAtenIfffRv7/tUP312Y360k3LyMANAMME6Z5jQsIwAABq19uPnKGn123XD+5+Tg31KX168YFKJizuZgEAekDwHINSydVWLDHyDABADfv0yQdqZ66gH/zpOT21drv+68zD1ZhNx90sAEA3iN5i0Baub+KaZwAAalcyYfrqaYfqa289VH9ZvkGnXfYXLV+3Pe5mAQC6QfQWg1w+Cp7Jtg0AAN519Cxd80/HaHtrQadd9lfd9vjauJsEAOgCwXMMcoWiJEaeAQBAZNHscfrNR47TfhNH65+uWqJL73hGpRKJxACgmhC9xaA1jDyTMAwAAJRNbczq+g8eq7cdPl3fuu1pffjqh7QzV4i7WQCAgOA5Bu0jzyQMAwAAFTLppL75zgX6/BsP0h8eX6O3fPce/XX5hribBQAQwXMscoXyyDOnHwAA7MrM9IHj99NPzjtabcWS3vXD+/Thqx/Uqi0tcTcNAGoa0VsMyiPPmTTTtgEAQNeOmztBt33i1frk6w7QnU+u00nfvEuX3vGMWvPFuJsGADWJ4DkGuTwjzwAAoHeZdFIfOWme7vjkCTrxwEn61m1P63Xfvlt/WLZG7iQUA4ChRPQWg9b2bNuMPAMAaoeZLTazp8xsuZld1MX295vZejNbGh4fiKOd1Wh6U1b/8+4jdfUHjlYmldT5P3lQZ//vA1r20ta4mwYANYPgOQaMPAMAao2ZJSVdJukUSfMlnWVm87uoep27LwyPHw5pI4eB4+ZO0M0fO15feNN8/f3FzXrjpffoPT+8T3c/vZ6RaAAYZERvMSBhGACgBh0labm7P+fubZKulXRqzG0altLJhM575Rzd85kT9ZnFB+rptdt19hX365T/+rNueHCl2sLnDADAwCJ6iwEJwwAANWi6pBUV6ytDWWdvN7NHzOwGM5vZ1Y7M7HwzW2JmS9avXz8YbR0WGkel9aET9tc9nzlR//mOBXKX/uXnD+v4b9yp79/9rLa25ONuIgCMKATPMWDkGQCALv1G0mx3P0zSbZKu7KqSu1/u7ovcfdHEiROHtIHVqC6V0OlHztDvP368fnzOyzV3UoP+45YndczX7tDHrv277npqnQpFRqMBYG+l4m5ALWq/5pmRZwBA7VglqXIkeUYoa+fuGytWfyjpG0PQrhHDzHTCyybphJdN0mOrtupn9/9Dv3tktW5c+pImNNTrLQum6W1HTNfB08bKzOJuLgAMOwTPMSjfn5GRZwBADXlA0jwzm6MoaD5T0rsqK5jZVHdfHVbfIumJoW3iyHHI9EZ97a2H6otvnq8/Prlev/r7Sv3k3hd0xV+e19xJDXrr4dP1xkOnavaE0XE3FQCGDYLnGOQKJSVMSiX41hcAUBvcvWBmF0q6VVJS0hXuvszMvixpibvfJOmjZvYWSQVJmyS9P7YGjxD1qaQWHzJFiw+Zoi3Nbfrdo6v167+v0iW3PqVLbn1K+00crRNfNkknHjRJL589TukkX+wDQHcInmOQKxRVn0oyZQoAUFPc/WZJN3cq+7eK5c9K+uxQt6tWNI2q07uP3lfvPnpfrdjUrDufXKc7nlynq/72on54z/Mak0npVQdM1EkHRlO/x42ui7vJAFBVCJ5jkCuUlEnzzS4AAIjHzHGjdPYrZuvsV8zWzlxB9yzfoDufiILp3z0SzZw/cMoYHbPfeB2z3zgdNWc8wTSAmkfwHINcvqT6FMnCAABA/EbXp3TywVN08sFTVCq5Hntpq+5+ar3ue36TrntghX781xckSS+bPEZH7zdOx+w3Xov23UeTxmbibTgADDGC5xi0FoqqZ+QZAABUmUTCdNiMJh02o0kfkdRWKOnRVVt073ObdO9zG/XzJSt11d9elCRNbczosBmNWjCzSQtnNOnQGY0ak0nH+wYAYBARPMcgGnkmeAYAANWtLpXQkfuO05H7jtMFr5mrfLGkR1dt1dJ/bNHDK7fo4RVbdOuytZIkM2n/iQ06bEaj5k8dqwOnjNWBU8doQkN9zO8CAAZGrMGzmX1E0gWSipJ+5+6fDuWflXReKP+ou98ayhdL+i9FWTp/6O7/EcrnSLpW0nhJD0p6r7u3mVm9pKskHSlpo6Qz3P2Fno4xFMoJwwAAAIaTdDKhI2btoyNm7dNetnlnmx5ZtVWPrIgC6nue2aBfPtRxC++JY+p14JQxOmjqWB00dYzmTRqj/SaO1qg6xnAADC+x/dcys9dIOlXSAnfPmdmkUD5f0b0fD5Y0TdLtZnZAeNllkl4naaWkB8zsJnd/XNLXJX3b3a81s+8rCoq/F543u/tcMzsz1Duju2O4e3Eo3jsJwwAAwEixz+g6vfqAiXr1ARPbyzbuyOnJNdv1xOptenLNdj25Zpt+/NcX1FYotdeZ3pTVfhNHa/+JDdp/UoPmTmzQ/hNHa+KYeu5IAqAqxfmV34ck/Ye75yTJ3deF8lMlXRvKnzez5ZKOCtuWu/tzkmRm10o61cyekHSipHeFOldK+pKi4PnUsCxJN0j6rkX/jbs7xt8G6b3uIlcoKZtm5BkAAIxM4xvqddzceh03d0J7WaFY0vMbdmr5uh1avm6Hnl2/Q8+u36nrl6xQc1vH+MWouqRmjRulWeNGad/xo7Tv+NHR87jRmtaUUYp7UQOISZzB8wGSjjeziyW1SvoXd39A0nRJ91bUWxnKJGlFp/KjFU3V3uLuhS7qTy+/xt0LZrY11O/pGIOuNV9UU5aEGgAAoHakkgnNmzxG8yaP2aXc3bVmW6ueXbdTz67foRc3Nusfm3bquQ07ddfT63cZrU6YNLUxq+lNWU3fZ/fnqY0ZpoMDGDSD+t/FzG6XNKWLTZ8Lxx4n6RhJL5d0vZntN5jt2RNmdr6k8yVp1qxZA7LPXKFEtm0AAABJZqapjVlNbczqlfMm7LKtVHKt3d6qFzc268WNO7Vyc4tWbm7Rqs0tuv/5TVq9tUUl33V/jdm0pjZmNKUxoyljo+epjRlNHpvRpDEZTRpbr3Gj6pRIMDUcQP8MavDs7q/tbpuZfUjSL93dJd1vZiVJEyStkjSzouqMUKZuyjdKajKzVBh9rqxf3tdKM0tJagz1ezpG5/dwuaTLJWnRokXeVZ3+ImEYAABA7xKJjsD6mP3G77a9UCxpzbZWrdrcolVbWrR6a6vWbG3Vmm3R82OrtmnDjtxur0slTBPH1GvSmHpNHJPR5LH1mtBQrwlj6jVhdF303FCvCQ11aqhPcQ02AEnxTtv+taTXSPpjSAhWJ2mDpJsk/czMvqUomdc8SfdLMknzQmbtVYoSfr3L3d3M/ijpdEUZt8+WdGM4xk1h/W9h+52hfnfHGBK5PAnDAAAA9lYqmdCMfUZpxj6juq3TVihp7bZWrdveqrXbclq3rVXrtue0bntOa7e1auXmZj30j83a3Nwm72KYpD6V0ISGeo0bXadxo+s0PjyPaygv12vc6LSaRtVpn1F1asymlWRUGxiR4gyer5B0hZk9JqlN0tlhFHqZmV0v6XFJBUkXlLNgm9mFkm5VdKuqK9x9WdjXZyRda2ZflfR3ST8K5T+S9JOQEGyTooBb7t7tMYZCrlBi5BkAAGAI1KUSmjlulGaO6z7AlqJR7E0727R+R04bdrRpw/acNuyIHht3tGlTc5s27WzT8nU7tGlnm1ryXX90NIumju8zqk5No8JzNq3GUWk1ZevUmE2paVRdWE+rMZvW2GxaYzNp1aUYXAGqWWzBs7u3SXpPN9sulnRxF+U3S7q5i/Ln1JGRu7K8VdI7+nOMoRBN2+afIwAAQLVIJROaNDajSWMzfarf0lbUxp05bdrZps3NeW1pbtPmnW3aFJY37WzTlua81m5r1dNrt2trc17bc4Ue95lNJzU2m4oC6kwUWI/JpDS2/JxJa0wmrbHZlMZkymUpNdRHy6PqkkwxBwYR6QiHmLurNV8ieAYAABjGsnVJzajrecp4Z/liSdta8trakteWlry2NkfL21o7Lbfkta2loNVbW/X0ury2txa0rSW/W3K0zhImNdRHgXVDfUoNmZRG16c0pj6lhvpouSGTUkN9MlquT2l0Xap9eVR9sr3eqHSSpGpAJwTPQ6ytGN1uoZ77PAMAANSUdDKh8Q31Gt9Q3+/Xurua24ra1toRTG/PFbSjtaDtrQXtyEXlHY+8drYVtLUlr1Wbm7UjV9DOXFE7ehn9rjSqLqlRdSmNrg/PdUmNqo+es3VJja5L7VInW5fUqLqksulyebks1b6cTSeV5l7dGKYInodYLtyrkJFnAAAA9JWZaXQYFZ7auOf7KZVcO9s6AunmtkJ7YL0zVwjbovXmtoJ2thXVnCtoR1jf2tym1VuKam6L1pvbiu2fb/sqnTRl0uVAO9mxXLGeTXdar0sqk0pEz+mKR2VZKqlMOqH6dPRcl0wwjR0DiuB5iOXyjDwDAAAgHomEheul0wO2z2LJ1dxWUEtbMQq2w3JzW1Et+WL7crm8JR+tt+Y71ylo8868WgtFtZbL80W15vsXnLe/V5PqQ0BdDrbrU4ldnsvbKsvLr6lPJVWfTigTnsvb6lOJsB4t16U6LyeUYnR9RCJ4HmK5QpSZkZFnAAAAjATJQQjIK5VKrlyhpNZ8ZUBdfpTaA/LWfFGthZJyFduismi5vI/WfFG5fElbmtuiOoVoPVcottfp7fry3iQTprpkFGTv8pxKtgfYlc+V2+oqyjrXqdyWTiVUH8rSyd1fl25/NkbhBwjB8xArf3NG8AwAAAD0LpGwaAp3XVL7DMHx3F2FELDnQkDeFoLqclmu0BGMt4XltkJHeVRWrFjevWx7a0EbCyW1FaPytlDeFsryxb2M4DtJJ60ioA4BeChLVwTj6VRH2S51UgmlExXLybBeXq7cV1hOhe11YTkV9plKWkedZMd+UxWvSyas6gJ+guch1jHyzLRtAAAAoNqYdQR2DfXxhUulkqutGAXSnQPryud8xXOuEAXdleWd65UD83xx19e2FV35Qkmt+ZJ2tBai9VAnX4i2txWKKpRchaK3J0IeTLsE4SG4TiWi4DzVHpBHZd84/TAdMHnMoLaH4HmIZdJJHT9vgiaN7X+WRQAAAAC1IZEwZRLRtdrVqDxCn+8UjJcD60JlWSkKzPPhuVCKgvFCOTgPdQtFV74UPReKlfW72F6KXlcI+08Nwa3VCJ6H2P4TG/ST846OuxkAAAAAsMcqR+hrRe28UwAAAAAA9hDBMwAAAAAAvSB4BgAAAACgFwTPAAAAAAD0guAZAAAAAIBeEDwDAAAAANALgmcAAAAAAHpB8AwAAAAAQC8IngEAAAAA6AXBMwAAAAAAvSB4BgAAAACgFwTPAAAAAAD0guAZAAAAAIBemLvH3YZhw8zWS3qxj9UnSNowiM0ZSThX/cP56jvOVd9xrvpuIM/Vvu4+cYD2VZPomwcN56p/OF99x7nqO85V3w1J30zwPEjMbIm7L4q7HcMB56p/OF99x7nqO85V33Guhi9+dn3Hueofzlffca76jnPVd0N1rpi2DQAAAABALwieAQAAAADoBcHz4Lk87gYMI5yr/uF89R3nqu84V33HuRq++Nn1Heeqfzhffce56jvOVd8NybnimmcAAAAAAHrByDMAAAAAAL0geB4EZrbYzJ4ys+VmdlHc7akmZnaFma0zs8cqysaZ2W1m9kx43ifONlYLM5tpZn80s8fNbJmZfSyUc746MbOMmd1vZg+Hc/V/Q/kcM7sv/C1eZ2Z1cbe1WphZ0sz+bma/Deucq26Y2Qtm9qiZLTWzJaGMv8Nhhr65e/TNfUff3Hf0zf1H39x3cfXNBM8DzMySki6TdIqk+ZLOMrP58baqqvxY0uJOZRdJusPd50m6I6xDKkj6pLvPl3SMpAvC7xLna3c5SSe6+wJJCyUtNrNjJH1d0rfdfa6kzZLOi6+JVedjkp6oWOdc9ew17r6w4jYY/B0OI/TNvfqx6Jv7ir657+ib+4++uX+GvG8meB54R0la7u7PuXubpGslnRpzm6qGu/9J0qZOxadKujIsXynptKFsU7Vy99Xu/lBY3q7on+l0cb5245EdYTUdHi7pREk3hHLOVWBmMyS9UdIPw7qJc9Vf/B0OL/TNPaBv7jv65r6jb+4f+uYBMeh/hwTPA2+6pBUV6ytDGbo32d1Xh+U1kibH2ZhqZGazJR0u6T5xvroUpjotlbRO0m2SnpW0xd0LoQp/ix2+I+nTkkphfbw4Vz1xSX8wswfN7PxQxt/h8ELf3H/8jveCvrl39M398h3RN/dHLH1zaqB3COwNd3czIwV8BTNrkPQLSR93923RF5ERzlcHdy9KWmhmTZJ+JenAeFtUnczsTZLWufuDZnZCzM0ZLl7p7qvMbJKk28zsycqN/B1ipON3fHf0zX1D39w39M17JJa+mZHngbdK0syK9RmhDN1ba2ZTJSk8r4u5PVXDzNKKOuer3f2XoZjz1QN33yLpj5KOldRkZuUvCflbjBwn6S1m9oKiqasnSvovca665e6rwvM6RR/+jhJ/h8MNfXP/8TveDfrm/qNv7hV9cz/F1TcTPA+8ByTNC9nx6iSdKemmmNtU7W6SdHZYPlvSjTG2pWqEa11+JOkJd/9WxSbOVydmNjF8qy0zy0p6naLr0P4o6fRQjXMlyd0/6+4z3H22ov9Pd7r7u8W56pKZjTazMeVlSa+X9Jj4Oxxu6Jv7j9/xLtA39x19c9/RN/dPnH2zuTOrZKCZ2RsUXbeQlHSFu18cb4uqh5ldI+kESRMkrZX0RUm/lnS9pFmSXpT0TnfvnLik5pjZKyX9WdKj6rj+5V8VXVvF+apgZocpSgyRVPSl4PXu/mUz20/RN7jjJP1d0nvcPRdfS6tLmBr2L+7+Js5V18J5+VVYTUn6mbtfbGbjxd/hsELf3D365r6jb+47+uY9Q9/cuzj7ZoJnAAAAAAB6wbRtAAAAAAB6QfAMAAAAAEAvCJ4BAAAAAOgFwTMAAAAAAL0geAYAAAAAoBcEzwCGnJmdYGa/jbsdAAAgQt8M9I7gGQAAAACAXhA8A+iWmb3HzO43s6Vm9gMzS5rZDjP7tpktM7M7zGxiqLvQzO41s0fM7Fdmtk8on2tmt5vZw2b2kJntH3bfYGY3mNmTZna1mVlsbxQAgGGCvhmID8EzgC6Z2UGSzpB0nLsvlFSU9G5JoyUtcfeDJd0t6YvhJVdJ+oy7Hybp0YryqyVd5u4LJL1C0upQfrikj0uaL2k/SccN8lsCAGBYo28G4pWKuwEAqtZJko6U9ED44jkraZ2kkqTrQp2fSvqlmTVKanL3u0P5lZJ+bmZjJE13919Jkru3SlLY3/3uvjKsL5U0W9I9g/6uAAAYvuibgRgRPAPojkm60t0/u0uh2Rc61fM93H+uYrko/h8BANAb+mYgRkzbBtCdOySdbmaTJMnMxpnZvor+b5we6rxL0j3uvlXSZjM7PpS/V9Ld7r5d0kozOy3so97MRg3lmwAAYAShbwZixLdJALrk7o+b2ecl/cHMEpLyki6QtFPSUWHbOkXXXknS2ZK+Hzrg5ySdE8rfK+kHZvblsI93DOHbAABgxKBvBuJl7ns6qwNALTKzHe7eEHc7AABAhL4ZGBpM2wYAAAAAoBeMPAMAAAAA0AtGngEAAAAA6AXBMwAAAAAAvSB4BgAAAACgFwTPAAAAAAD0guAZAAAAAIBeEDwDAAAAANCL/w8QDWivqh+HOwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "PMF_model.plot_history()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can show the predictions.<br>\n",
    "This array contains predicted ratings for each film and each user."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[7.61285827 9.7008641  8.24477333 ... 1.         1.         1.        ]\n",
      " [7.36409559 6.60971559 8.73170649 ... 1.         1.         1.        ]\n",
      " [3.74121054 1.         1.44741694 ... 1.49265373 1.49265373 1.        ]\n",
      " ...\n",
      " [5.39949936 3.72039516 3.83382713 ... 1.         1.         1.        ]\n",
      " [6.0364435  3.50469717 4.25813621 ... 1.         1.         1.        ]\n",
      " [8.54859188 3.99996115 1.         ... 1.         1.         2.15478004]]\n"
     ]
    }
   ],
   "source": [
    "print(PMF_model.predict())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also examine the error of our model, i.e. how far from the truth it is."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 18671,
     "status": "ok",
     "timestamp": 1687523048468,
     "user": {
      "displayName": "Mateusz Soczewka",
      "userId": "14486733780861258567"
     },
     "user_tz": -120
    },
    "id": "JGK9PnIqElFZ",
    "outputId": "cfbd6fff-0bc3-4520-beb4-750301079466"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-------------------------\n",
      "RMSE: 0.27797442619788887\n"
     ]
    }
   ],
   "source": [
    "print('-------------------------')\n",
    "print('RMSE:', PMF_model.history['rmse'][-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also get prediction for specific user and specific movie by passing them ID's."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 286,
     "status": "ok",
     "timestamp": 1687523084640,
     "user": {
      "displayName": "Mateusz Soczewka",
      "userId": "14486733780861258567"
     },
     "user_tz": -120
    },
    "id": "0f7SZwQHE6V4",
    "outputId": "921d5436-8f62-401b-b345-e198683f244d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "round(PMF_model.predict_one(user_id=1, item_id=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "According to our model, user 1 is likely to rate the video with id=2 with a rating of 10.\n",
    "\n",
    "Function below compare true user rating with our predictions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "ugTzJ0HhFRGJ"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     True rating: 8.0\n",
      "Predicted rating: 7.612858270920231\n"
     ]
    }
   ],
   "source": [
    "def compare_pred_vs_true(user_id, item_id):\n",
    "    print('     True rating:', PMF_model.R[PMF_model.user_to_row[user_id], PMF_model.item_to_column[item_id]])\n",
    "    print('Predicted rating:', PMF_model.predict_one(user_id=user_id, item_id=item_id))\n",
    "\n",
    "compare_pred_vs_true(user_id=1, item_id=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can copmare predictions for some random users ratings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     True rating: 10.0\n",
      "Predicted rating: 10.0\n",
      "\n",
      "\n",
      "     True rating: 9.0\n",
      "Predicted rating: 8.227362600349798\n",
      "\n",
      "\n",
      "     True rating: 2.0\n",
      "Predicted rating: 2.032618706589141\n",
      "\n",
      "\n",
      "     True rating: 10.0\n",
      "Predicted rating: 9.912388552967919\n",
      "\n",
      "\n",
      "     True rating: 10.0\n",
      "Predicted rating: 9.88467252141719\n",
      "\n",
      "\n",
      "     True rating: 4.0\n",
      "Predicted rating: 4.316526464483501\n",
      "\n",
      "\n",
      "     True rating: 6.0\n",
      "Predicted rating: 5.924530344751906\n",
      "\n",
      "\n",
      "     True rating: 10.0\n",
      "Predicted rating: 9.889626359269368\n",
      "\n",
      "\n",
      "     True rating: 7.0\n",
      "Predicted rating: 6.980524520014333\n",
      "\n",
      "\n",
      "     True rating: 4.0\n",
      "Predicted rating: 3.812907609892198\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Compare some predictions with true ratings\n",
    "for i in range(10):\n",
    "    row_sample = PMF_model.df.sample(1)\n",
    "    user_id = row_sample['userId'].values[0]\n",
    "    item_id = row_sample['movieId'].values[0]\n",
    "    compare_pred_vs_true(user_id=user_id, item_id=item_id)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also get information about predictions only for those movies that have not been rated by a given user.\n",
    "\n",
    "We can treat it as a mask that will allow us to decide what movies, which he has not seen yet, to propose to each user.\n",
    "\n",
    "Values 0 in this matrix mean that the user has already rated the movie. Other values show the predicted rating for the user-video pair."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.         9.7008641  0.         ... 1.         1.         1.        ]\n",
      " [7.36409559 6.60971559 8.73170649 ... 1.         1.         1.        ]\n",
      " [3.74121054 1.         1.44741694 ... 1.49265373 1.49265373 1.        ]\n",
      " ...\n",
      " [0.         0.         0.         ... 1.         1.         1.        ]\n",
      " [0.         3.50469717 4.25813621 ... 1.         1.         1.        ]\n",
      " [0.         3.99996115 1.         ... 1.         1.         2.15478004]]\n"
     ]
    }
   ],
   "source": [
    "print(PMF_model.get_masked_preds())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's put our model into practice!\n",
    "\n",
    "Let's try to find out the title of the movie that, according to our model, user xxx will like the most.<br>\n",
    "To do this we have to read movies metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                               title  \\\n",
       "0        1                    Toy Story (1995)   \n",
       "1        2                      Jumanji (1995)   \n",
       "2        3             Grumpier Old Men (1995)   \n",
       "3        4            Waiting to Exhale (1995)   \n",
       "4        5  Father of the Bride Part II (1995)   \n",
       "\n",
       "                                        genres  \n",
       "0  Adventure|Animation|Children|Comedy|Fantasy  \n",
       "1                   Adventure|Children|Fantasy  \n",
       "2                               Comedy|Romance  \n",
       "3                         Comedy|Drama|Romance  \n",
       "4                                       Comedy  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_metadata = pd.read_csv('data/movies.csv')\n",
    "movies_metadata.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take a look on user with `userId=435`<br>\n",
    "Which row corresponds to this user?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "434\n"
     ]
    }
   ],
   "source": [
    "test_user_id = 435\n",
    "print(PMF_model.user_to_row[test_user_id])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can check the column number with the highest predicted score and how high is that score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "92\n"
     ]
    }
   ],
   "source": [
    "print(PMF_model.get_masked_preds()[PMF_model.user_to_row[test_user_id]].argmax())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Highest score: 10.0\n"
     ]
    }
   ],
   "source": [
    "print('Highest score:', PMF_model.get_masked_preds()[434][92])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the ID of film in column 702?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "104\n"
     ]
    }
   ],
   "source": [
    "print(PMF_model.transpose_dict('item')[92])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What movie is hidden under the `movieId=104`?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>104</td>\n",
       "      <td>Happy Gilmore (1996)</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    movieId                 title  genres\n",
       "92      104  Happy Gilmore (1996)  Comedy"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_metadata[movies_metadata.movieId == 104]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "authorship_tag": "ABX9TyOUUR69HUBJzuXsSZkwenJH",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}