rnn.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "rnn.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.8"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/DLOHai/DLQA_Torch/blob/master/rnn.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WuU7OHX3XxsY",
        "colab_type": "text"
      },
      "source": [
        "# Recurrent Neural Network: Neural Weather Forecaster\n",
        "\n",
        "이번 실습의 목표는 다음과 같습니다.\n",
        "- RNN을 설계하여 지난 며칠 동안의 날씨 정보를 기반으로 24시간 이후의 기온을 예측한다.\n",
        "- 다양한 속성의 시계열 정보를 활용하기 위해 적절한 전처리 과정을 적용한다. \n",
        "- 설계한 모델의 성능을 검증하기 위해 베이스라인 모델을 도입한다.\n",
        "\n",
        "실습코드는 Python 3.6, Pytorch 1.0 버전을 기준으로 작성되었습니다.\n",
        "\n",
        "**본문 중간중간에 Pytorch 함수들에 대해 [Pytorch API 문서](https://pytorch.org/docs/stable/) 링크를 걸어두었습니다. API 문서를 직접 확인하는 일에 익숙해지면 나중에 여러분이 처음부터 모델을 직접 구현해야 할 때 정말 큰 도움이 됩니다.**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o4-9XdVcXxsa",
        "colab_type": "text"
      },
      "source": [
        "## 1. Package load\n",
        "필요한 패키지들을 로드합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3s3IfDT4Xxsc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import os\n",
        "import glob\n",
        "import csv\n",
        "import numpy as np\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.utils.data as data\n",
        "import matplotlib.pyplot as plt\n",
        "from torch.utils.data import DataLoader\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DQA2n5iif46c",
        "colab_type": "code",
        "outputId": "d3395d69-4102-4f52-dc0d-e017fe74cb6c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "!unzip /content/rnn.zip"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Archive:  /content/rnn.zip\n",
            "replace check_util/__pycache__/checker.cpython-36.pyc? [y]es, [n]o, [A]ll, [N]one, [r]ename: N\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N5RVN8QOf5NC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from check_util import checker"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "raHJb6AfXxsk",
        "colab_type": "text"
      },
      "source": [
        "## 2. 하이퍼파라미터 세팅\n",
        "학습에 필요한 하이퍼파리미터의 값을 초기화해줍니다.\n",
        "\n",
        "미니배치의 크기, 학습할 Epoch(세대) 수, Learning rate(학습률) 값들을 다음과 같이 정합니다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ItYSGEIGXxsm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "batch_size = 100\n",
        "num_epochs = 30\n",
        "learning_rate = 0.00003"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R3QaGlwagaUM",
        "colab_type": "code",
        "outputId": "034e8cb9-7625-408b-a6c0-aac389e12aa6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        }
      },
      "source": [
        "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
        "device"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'cuda'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dmNSwZCMXxst",
        "colab_type": "text"
      },
      "source": [
        "## 3. 데이터 전처리 함수 정의\n",
        "\n",
        "우리는 이번 실습에서 지난 10년간의(2009년~2018년) 서울시 기후 데이터를 활용해 기온을 예측하는 모델을 학습시킬 것입니다. 데이터셋은 [기상자료개방포털](https://data.kma.go.kr/)에서 받은 자료입니다. 이번 실습에서 사용하는 데이터 이외에도 기상자료개방포털에서 기상과 관련된 다양한 자료들을 내려받으실 수 있습니다. \n",
        "\n",
        "'./data/climate_seoul' 경로의 디렉토리를 보시면, test / train / val 디렉토리에 csv파일이 각각 1개 / 8개/ 1개 담겨있음을 확인하실 수 있습니다. 각 csv파일은 1년간의 서울시 기후 데이터를 담고 있으며, 1시간 간격으로 기록된 정보입니다. (사실 아주 가끔씩 30분 간격으로 기록한 구간도 있기도 하지만, 이후 본문에서는 편의상 모두 1시간 간격으로 기록된 것으로 간주하겠습니다.) 매 시간마다 기록되는 정보는 기온, 강수량, 풍속 등을 포함한 총 25가지 속성으로 이루어져 있습니다. 이 중에서 우리는 기온, 강수량, 풍속, 습도, 증기압을 포함한 총 9가지의 속성만을 사용하여 기온 예측 모델을 학습시켜 보겠습니다.\n",
        "\n",
        "그렇다면 왜 하필 이 9가지의 속성을 선택한 것일까요? 이렇게 9가지의 속성을 선택한 배경에는 어떠한 전문적인 지식도 고려되지 않은 것입니다. 25가지의 속성을 모두 사용해볼 수도 있겠죠. 어떤 속성들을 활용할지는 설계자의 몫입니다. 그런데 여러분이 기상과 관련된 전문적인 지식을 갖고 있지 않는 이상 이중에서 어떤 속성이 기온 예측에 가장 중요한지, 또는 어떤 속성이 가장 불필요한 속성인지 알지 못할 것입니다. 하지만 고맙게도 딥러닝은 이러한 속성 선택 문제에 덜 예민한 학습 방식입니다. 더 정확하게 말하면, 다소 불필요한 정보가 입력으로 주어진다고 해서 극단적으로 학습이 이루어지지 않는 일은 일어나지 않을 가능성이 큽니다. 학습과정에서 인공신경망이 필요한 특징(feature)을 알아서 추출하기 때문입니다. 그러니까 우리는 어떤 속성을 활용할지를 너무 심각하게 고민하지 않아도 되는 것입니다. 다만 이번 실습에서는 매번 빠짐없이 잘 기록된 속성들을 위주로 9가지를 선택한 것 뿐입니다.\n",
        "\n",
        "아래에 정의한 **preprocess** 메소드는 csv파일들을 읽어 9가지 속성 정보만을 numpy 배열에 저장해 반환하는 역할을 합니다. 이 메소드는 이후에 구현할 Dataset class에서 활용할 것입니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "muO89_P3Xxsu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def preprocess(all_files):\n",
        "    data_0 = [] # 기온\n",
        "    data_1 = [] # 강수량\n",
        "    data_2 = [] # 풍속\n",
        "    data_3 = [] # 습도\n",
        "    data_4 = [] # 증기압\n",
        "    data_5 = [] # 이슬점 온도\n",
        "    data_6 = [] # 현지 기압\n",
        "    data_7 = [] # 해면 기압\n",
        "    data_8 = [] # 지면 온도\n",
        "    for f in all_files:\n",
        "        with open(f, encoding='euc-kr') as c:\n",
        "            csv_reader = csv.reader(c, delimiter=',')\n",
        "            header = True\n",
        "            for col in csv_reader:\n",
        "                if header:\n",
        "                    header = False\n",
        "                    continue\n",
        "                data_0.append(float(col[2])) if col[2] != '' else data_0.append(0.0)\n",
        "                data_1.append(float(col[3])) if col[3] != '' else data_1.append(0.0)\n",
        "                data_2.append(float(col[4])) if col[4] != '' else data_2.append(0.0)\n",
        "                data_3.append(float(col[6])) if col[6] != '' else data_3.append(0.0)\n",
        "                data_4.append(float(col[7])) if col[7] != '' else data_4.append(0.0)\n",
        "                data_5.append(float(col[8])) if col[8] != '' else data_5.append(0.0)\n",
        "                data_6.append(float(col[9])) if col[9] != '' else data_6.append(0.0)\n",
        "                data_7.append(float(col[10])) if col[10] != '' else data_7.append(0.0)\n",
        "                data_8.append(float(col[22])) if col[22] != '' else data_8.append(0.0)\n",
        "\n",
        "    data = np.zeros((len(data_0), 9))\n",
        "    for i, d in enumerate(data):\n",
        "        data[i, 0] = data_0[i]\n",
        "        data[i, 1] = data_1[i]\n",
        "        data[i, 2] = data_2[i]\n",
        "        data[i, 3] = data_3[i]\n",
        "        data[i, 4] = data_4[i]\n",
        "        data[i, 5] = data_5[i]\n",
        "        data[i, 6] = data_6[i]\n",
        "        data[i, 7] = data_7[i]\n",
        "        data[i, 8] = data_8[i]\n",
        "    return data"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oYXzVCjhXxs2",
        "colab_type": "text"
      },
      "source": [
        "## 4. Dataset 정의 및 DataLoader 할당\n",
        "\n",
        "이제 우리가 사용할 데이터셋에 대해 정의할 차례입니다. Pytorch 의 Dataset과 DataLoader에 대해 잘 기억나지 않는다면 ['Lab-04-2'](https://www.youtube.com/watch?v=B3VG-TeO9Lk&list=PLQ28Nx3M4JrhkqBVIXg-i5_CVVoS1UzAv&index=8&t=0s) 강의를 참고하시기 바랍니다.\n",
        "\n",
        "생성자(**\\__init__**)는 미리 구현을 해두었습니다. 여기서 사용되는 각 변수의 의미는 다음과 같습니다.\n",
        "- **seq_len**: 우리가 이후에 설계할 RNN의 입력으로 줄 데이터 시퀀스의 길이, 즉 총 타임스텝의 길이를 뜻합니다. 다시 말하면, 현재로부터 24시간 뒤의 기온을 예측하기 위해 얼마만큼의 과거 정보를 참고할지 결정하는 것입니다. 기본값을 480로 해두었는데, 이는 480개의 과거 데이터를 살펴보겠다는 것이고, 데이터 1개는 1시간마다 기록되기 때문에 결과적으로 지난 20일간의 데이터를 기반으로 24시간 후의 기온을 예측하겠다는 의미입니다. \n",
        "- **target_delay**: 우리가 예측할 시점이 입력 시퀀스의 마지막 타임스텝으로 얼만큼 이후인지 결정하는 것입니다. 24로 기본값을 해두었고, 이는 24시간 이후의 기온이 우리가 예측할 대상임을 의미하는 것입니다. \n",
        "- **stride**: 데이터를 모델에게 입력으로 주기 위해 우리는 **seq_len** 길이 만큼의 정보를 우리가 가진 전체 데이터에서 임의로 선택해야 합니다. 임의로 선택된 그 시작 지점을 시작 인덱스라고 부른다면, **stride**는 그 시작 인덱스 후보들 간의 간격을 의미합니다. **stride=1**이라면 모든 시점의 데이터가 시작 인덱스가 될 수 있는 것이고, **stride=2**이면 아래의 그림처럼 시작 인덱스 후보가 하나씩 건너띄어 존재하므로, 전체 데이터 포인트에서 절반만이 시작 인덱스가 될 수 있습니다. **stride**가 작을수록 모델의 서로다른 입력간에 정보가 중복되는 정도가 크겠죠. (**stride=1**을 '01234', '12345', '23456' 이라고 생각하고 **stride=2**를 '01234', '23456', '45678' 이라고 생각하면 이해가 되실 겁니다.) 어떤 값으로 정할지는 역시 설계자인 우리의 몫입니다. 이번 예제에서는 기본값을 5로 정하겠습니다.\n",
        "<img src=\"./img/stride.png\" width=\"80%\" height=\"60%\">\n",
        "- **all_files**: 정의할 Dataset에 사용할 모든 csv파일의 경로를 담고 있습니다. **data_dir**은 데이터셋의 디렉토리 경로를 의미하고 **mode**는 정의하고자 하는 Dataset에 따라 'train' 또는 'val' 또는 'test'으로 구분될 것입니다. \n",
        "- **self.data**: 위에서 정의한 데이터 전처리 메소드인 **preprocess**에서 데이터를 전처리한 결과가 저장됩니다. **self.data**의 shape은 (데이터의 총 길이, 9)가 됩니다.\n",
        "- **normalize**: 입력으로 사용하는 데이터의 각 속성은 저마다 값의 범위가 다릅니다. 예를들어 기온은 보통 -15에서 35사이의 값을 갖지만, 강수량과 풍속은 음수 값이 존재하지 않고, 기압의 경우에는 1000 내외의 값이 일반적입니다. 이러한 경우 각각의 속성들을 저마다의 평균과 표준편차를 통해 값을 정규화해주는 것이 바람직합니다. \n",
        "\n",
        "이제 다음을 읽고 코드를 완성해보세요.\n",
        "- **\\__len__**은 미리 구현을 해두었고, 이제 여러분이 직접 **\\__getitem__**을 구현해야 합니다. 모델의 입력으로 줄 데이터 시퀀스를 **sequence** 변수에 저장하고, 그 **sequence**의 가장 마지막 타임스텝으로부터 24시간 후의 기온을 **target** 변수에 저장하여 코드를 완성해보세요.\n",
        "- **sequence**의 shape은 (**seq_len**, 9)가 되어야 합니다. **index**는 **\\__len__**에서 정의한 데이터의 총 길이를 값의 범위로 합니다. 예를 들어 현재 가진 데이터셋의 총 길이와 주어진 **seq_len**, **stride**를 고려했을 때 존재할 수 있는 시작 인덱스 후보가 1000개라면 **\\__len__**는 1000을 반환할 것이고 **index**는 0~999 사이의 값을 가지게 됩니다. \n",
        "- **target**의 shape은 (1)이 되어야 합니다. \n",
        "- **힌트**: **sequence**를 정의하기 위해서는 아마 우리가 가진 전체 데이터, 즉 **self.data**에서 시작 인덱스부터 **seq_len** 길이만큼을 인덱싱을 해서 가져와야 할 것입니다. 이때 시작 인덱스로 **index**를 곧장 사용해서는 안됩니다. 앞서 언급한 것처럼 **index**는 0과 시작 인덱스 후보의 총 갯수 사이의 값이지, 그 자체로 시작 인덱스를 의미하는 것이 아닙니다. 시작 인덱스를 구하기 위해서는 **index**에 **self.stride**를 곱해주어야 합니다.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cQUdWC5oXxs4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Dataset(data.Dataset):\n",
        "    def __init__(self, data_dir, mode, seq_len=480, target_delay=24, stride=5, normalize=True):\n",
        "        self.seq_len = seq_len\n",
        "        self.target_delay = target_delay\n",
        "        self.stride = stride\n",
        "        all_files = glob.glob(os.path.join(data_dir, mode, '*'))\n",
        "        self.data = preprocess(all_files)\n",
        "        if normalize:\n",
        "            mean = self.data.mean(axis=0)\n",
        "            std = self.data.std(axis=0)\n",
        "            self.data = (self.data - mean) / std\n",
        "    \n",
        "    def __getitem__(self, index):\n",
        "        # 코드 시작\n",
        "        index *= self.stride\n",
        "        sequence = self.data[index:index + self.seq_len, :]\n",
        "        target = self.data[index + self.seq_len + self.target_delay - 1]\n",
        "        target = target[0]\n",
        "        target = np.expand_dims(target, 0)\n",
        "        # 코드 종료\n",
        "        return sequence, target\n",
        "    \n",
        "    def __len__(self):\n",
        "        max_idx = len(self.data) - self.seq_len - self.target_delay\n",
        "        num_of_idx = max_idx // self.stride\n",
        "        return num_of_idx"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sbY2p-vgXxs-",
        "colab_type": "text"
      },
      "source": [
        "이제 학습용, 검증용, 테스트용 Dataset과 DataLoader를 각각 할당합니다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xTIaRWhaXxtA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data_dir = './data/climate_seoul'\n",
        "train_data = Dataset(data_dir, 'train')\n",
        "val_data = Dataset(data_dir, 'val')\n",
        "test_data = Dataset(data_dir, 'test')\n",
        "\n",
        "train_loader = DataLoader(train_data, batch_size=batch_size, shuffle=True, drop_last=True)\n",
        "val_loader = DataLoader(val_data, batch_size=batch_size, shuffle=False, drop_last=True)\n",
        "test_loader = DataLoader(test_data, batch_size=batch_size, shuffle=False, drop_last=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rtEG6QF2XxtI",
        "colab_type": "text"
      },
      "source": [
        "아래의 코드를 실행해 코드를 성공적으로 완성했는지 확인해보세요. \n",
        "\n",
        "별다른 문제가 없다면 이어서 진행하면 됩니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1tv0QAK4XxtK",
        "colab_type": "code",
        "outputId": "3ab2bc93-813b-496d-9142-9078355a9ff1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 75
        }
      },
      "source": [
        "checker.customized_dataset_check(train_data.seq_len, train_data)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "__getitem__ 함수가 반환하는 target이 24시간 후의 기온이 아닙니다. 인덱싱을 올바르게 했는지 다시 확인하시기 바랍니다.\n",
            "__getitem__ 함수가 반환하는 sequence가 self.data로부터 올바르게 인덱싱되고 있지 않습니다. stride를 고려하여 시작 인덱스를 올바르게 구했는지 다시 확인하시기 바랍니다.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "prz5ixBeXxtY",
        "colab_type": "text"
      },
      "source": [
        "## 5. 데이터 샘플 시각화\n",
        "\n",
        "**train_data**의 첫번째 **sequence**를 그래프 형태로 시각화합니다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FxbCIXINXxta",
        "colab_type": "code",
        "outputId": "17eb5993-d47e-4ebc-d480-809a21f51ded",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 269
        }
      },
      "source": [
        "temp = train_data[0][0]\n",
        "temp = temp[:, 0]\n",
        "plt.plot(range(len(temp)), temp)\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsvXmYJFd55vue3CL3zKrKrK271VW9\nq1toQY1WQALEJmssg40H22DsMZbX6/F4G3yNZx7b11zu4GFs44WR8eAFZjCbkA3CAoFWZIFaey9q\n9Va9VXfXvuQWW577R8SJjMyMyDhRS1Zm1vk9Tz1dnRVREVUVeb7zbe9HKKUQCAQCgSCw0TcgEAgE\ngs5AGASBQCAQABAGQSAQCAQmwiAIBAKBAIAwCAKBQCAwEQZBIBAIBACEQRAIBAKBiTAIAoFAIAAg\nDIJAIBAITEIbfQOtyOVydGxsbKNvQyAQCLqG5557boZSml/JuR1tEMbGxnDo0KGNvg2BQCDoGggh\nZ1d6rggZCQQCgQCAMAgCgUAgMBEGQSAQCAQAhEEQCAQCgYkwCAKBQCAAIAyCQCAQCEyEQRAIBAIB\nAGEQBAKBYEOoqDq+eOg8OmmMcUc3pgkEAkGv8hffPYm/ePQkklIId79uZKNvB4DwEAQCgWBDmC8p\nAICZgrzBd1JDGASBQCDYAKRQEACgaNUNvpMawiAIBALBBiCFjeVXFgZBIBAINjdR00OQVX2D76TG\nqgwCIaSfEPJtQsgJ898+l+OuIoR8ixByjBBylBAytprrCgQCQbcTIMa/lR7yED4C4DuU0t0AvmP+\n34l/APAJSunVAG4CMLXK6woEAkFXw0JFRVnb4DupsVqDcC+Avzc//3sAP9J4ACFkP4AQpfTbAEAp\nLVBKS6u8rkAgEHQ1smaEipYrvWMQhiill8zPLwMYcjhmD4AFQshXCSEvEEI+QQgJrvK6AoFAsCao\nehV/+shrWCyrbb0u8xCWKu29bis8DQIh5BFCyGGHj3vtx1Gj3c6p5S4E4E0AfgvAGwDsAPAzLa53\nHyHkECHk0PT0tJ+fRSAQCHzzlecu4E8fOYH/+fiptl5XVg2D8NjxaVxerLT12m54GgRK6V2U0msc\nPh4EcIUQMgIA5r9OuYELAF6klJ6mlGoAvgbg9S2udz+l9CCl9GA+v6KxoAKBQMDNiakCACAhtVe4\ngYWMAOBLh8639dpurDZk9M8APmR+/iEADzoc8yyALCGEre5vBXB0ldcVCASCNeHcnJHSDLKynzYh\na1XsGUoiEwtjarkzupVXaxA+DuDthJATAO4y/w9CyEFCyGcAgFKqwwgXfYcQ8goAAuBvVnldgUAg\nWBMuzJcBAMttjuXLWhVSKIjBlISp5c4IGa3KR6KUzgJ4m8PrhwB82Pb/bwO4djXXEggEgvVgyUwm\nF9pc7SNrOqRQAKloCNM94iEIBAJBV8M8g3aXf1bUKqRwAIMpCdMdInAnDIJAINi0UEpRMBvDljbA\nQ4iGghhMRzG1JHfEXARhEAQCwaalpOiomutwQW5zDsH0EPJJCbJWbbtBckIYBIFAsGmxh4naHTJi\nSeVsPAyglsvYSIRBEAgEmxbmFQQIrNBRu6gllU2D0AEdy8IgCASCTQsL04xmYxvkIRhVRkBnaBoJ\ngyAQCDYtrNR0W18ci2UVerU9iV1Nr6JQ0ZCMhiyD0O6yVyeEQRAIBJsWtivfM5SEXqVtaxC7uFCG\nVqXYPpBA0pTMWG5zUtsJYRAEAsGmhfUg7BlOAQAmF9pjEM7MFAEA47mElUMQHoJAIBBsIPMlwyDs\nG04DAC4tln1/j5X0D0yYBmH7QNwKGYmyU4FAINhAjl9ewkgmil2DSQDA5II/g0ApxfjvPoQ/+Jcj\nvs47N1dGPBJEPilBCgUQDpK2Vzk5IQyCQCDYtByZXMKB0TTS0RCSUsh3yIiplH72exO+zivIKjKx\nMAghIIQgFQ23XVzPCWEQBALBpqSi6jg1XcD+kTQIIRjJRH2HjI5OLgEAEhF/QyArqlFyykhKIVF2\nKhAIBBvFbFFBlQJb+mIAgJFsDJd8Ti47MrkIABjKRH2dJ2s6ouGaEUlFhUEQbCK++Ox5HL64uNG3\nIRBYsKoeNiltSzbqO4dwfs44XtWrvs4zlE5rBiETC7d9prMTwiB0GYcvLuLpkzMbfRu++Z2vvIx7\nPvXURt+GQGDBkrisD2AkE8NMQakbbenFXEkBACwU/S3mFVWvCxn1JyKYKyq+vsd6sGqDQAjpJ4R8\nmxBywvy3z+W4/0YIOUIIOUYI+XNCSHvn1XUQL55fWHEDzD2fego/+Znvr/EdrS/VNnV/CgR+KDYZ\nBCPs42fgPVvEl2UNisbvJchatS5kNJCIYLYDZiKshYfwEQDfoZTuBvAd8/91EEJuA3A7jKlp1wB4\nA4A71uDaXcmP/OX38PZPPrGq79EJDw8vFR87LoGgXTAPgYWMWOnpI8emuL+HfVe/UObf4Td7CBKW\nKprv0NNasxYG4V4Af29+/vcAfsThGAogCiACQAIQBnBlDa7ddbAmlpXEC+07kKOXltbsntaboiwM\ngqDzaAwZXb8ti9t3DeDTj5/i9mpnCzJyyQgAYN5H2Ehp8BD6re+xsWGjtTAIQ5TSS+bnlwEMNR5A\nKf03AI8CuGR+PEwpPbYG1+46yurKF8dzcyXr81cvLa/F7bSFkrLx1RMCQSONISNCCN57w1ZML8tc\nGy5VN4ba7MwbnsV8yZ+HELV5CAMJwyDMdoNBIIQ8Qgg57PBxr/04amx/m0wrIWQXgKsBbAWwBcBb\nCSFvcrnWfYSQQ4SQQ9PT075/oE7H3o1YVvwZh3NzRevzK0vt0VxZCzqhA1MgaKSxyggA7tibBwA8\nxVG4wXbzO81Qk5/dfUUzpqUx+k2DsNGJ5ZD3IQCl9C63rxFCrhBCRiillwghIwCcAnDvAfAMpbRg\nnvNNALcCeNLhWvcDuB8ADh482HPZSLuA1dm5oqWhwsOczSVlHZLdQMlm+Cil2MT1BIIOoqBoiIQC\niNh26rmkhGw8jIvz3uWnbDe/y/QQ5nx4CLJqzFOuXdcwCDMbnBtci5DRPwP4kPn5hwA86HDMOQB3\nEEJChJAwjITypgwZ2ePpswV/uwGWd9g1mMR0FxmEot0rWkXITCBYSwoVzQoX2cknJa7313kzhHvN\nlgwAYKHEn0OoNOQQRjJGc9yL5xfw7//nv/nuh1gr1sIgfBzA2wkhJwDcZf4fhJCDhJDPmMd8GcAp\nAK8AeAnAS5TSf1mDa3cd9vCJn5gjUJu5ujOfaJtu+1pg9xCWyiJ8JOgMirKGhNQsOTGYlrjeXxOz\nRgh371AK8UiQO9yj6VXoVVpXZZSQQhhMSfjs9ybw/TNz+MyTZzh/irWFK2TUCkrpLIC3Obx+CMCH\nzc91AL+w2mv1AnaD4GdHARgeQioawnA6in87NbvWt7Zu2D2ExbKKYZ9t/gLBelCQdSQizh7Cc+fm\nPc8/M1NCXzyMTDyMvniEe4NXMasF7R4CAIzlEhseChadym2mcXH0w1LZUEjMp4ya5UqXhF/sHsJG\nJ80EAkZBVh1DRoPpKKaWZM85B2dnixjLJQAAfYkwd1KZvW/tSWUAGBuIW59Pb1AuQRiENlPvIfjP\nIaSjYfQnJAD+Q04bhf1n3qgHXSBopCjr1nAaO/mkBFmreg6subhQxtY+YxE3PAS+DZ7MPIRQvYdw\n7das9fnZ2SI2AmEQ2gxbHDOxMPcDxFiqGB4Ci3t2S8OXvQ9hqovKZQW9TUHWkDTHV9rhLQGdLShW\n/4AfcTo3D+F9B7daHsvETHFFk9hWy6pzCAJ/FGUNhBi6KSvJIezIJa24Z7c0fL18YRH5lITFkio8\nBMG6UZA1BAlBjHM2wbJLlRE7v1VIVtZ0FGTNMgixcJA7hCurhocgNXgIUiiIF//L27Fc0SCFAxtS\nni08hDazVDbiln3xCBZ9aJ8AZsgoFrIaabrBQzhxZRlPnpjBz71xHPkUXzmfQLASrvmvD+OuTz7O\nfXxBVh1DRrGwt0Fg3gOTnIhFgtwl1UzbKxpuXn5DwQD6EhHEHZLd7UAYhDZzbq6EbX1xZOP+Q0aL\n5caQUed7CJfNENGN2/uEQRCsGyy8cpGzfl/Vq6ioVUcPgYVyKqq70BzrIbJ7CLzKA1bIKORvylo7\nEAahzZydLWEsF0cqGqrrWvZC1nRU1CoysbC1eyh2QciIhcWyZnWUMAiC9cBv9VqjjpEdVg7aSqXX\n8hDMAo9oOAhZq3KJ4llJZQcPYaPpvDvqYTS9inNzJYwNJJCQQr52+Kyhy+4hlHxqIW0ErJIqEw8j\nEwtbzXUCwVoy4bMqh42rTDqEjFj1j8wTMjI9BGZEZI6ZCOz7NvYhdALCILSRyYUKtCrFWC6BpBRC\nQdG4KwlYBUPa7iF0Qcio5iFEjEHiXXDPgu5jYqamBMyzS2fVfilHD8FYFlvlBJhBqIWMvM9hMKNh\n71TuFDrvjnoYVmEzmJKQlEKglH+XbzcIiUj3lJ0ulFUkIkFEQgEkTa9oI8rpBL3NBZsYHc+mw5qF\n4OQhWEll991+4/msMonHIFSEhyAAamWiScleKcS3Y16qGAYhEwsjFAxACgW6oux0vqQgGzd2UQkp\nhCpt/UZzglKKf3r2nK/RhoLNhV0Mjqfhk+XvHMtOOaqMFK2KAAHCQWMJZYs7T2KZPf/CIGxy2OIf\nj4SsB5F3VgCLvafNRpqEFOqKpPJiyaiMAoCkmftYlv3lEf718GX856+8gj995LU1vz9BbzC5aDcI\n3s8X8yKcyk55PARFr9bJZvMYEYassSqjzlt+O++OehgW4klIQd+9BCxkxBbXeCSIUpeEjPoSNSMG\n+A91/e8fnAMAaJxjDQWbj0uLFeRTRsXPAkfhQs1DaO5UZgt1y8Y0VUckWFs+oz4MQkUVOQQBaiEj\nu4fAu1teajAIiUiXeAim/hJQc8/9JMMppTgyaYwz7KYpcYL2QSnFpYUyrh4xhk1xhYzM952T/HUg\nQBAJBVqWnSp6FZIt5OM3hxAKEISCnbf8dt4d9TBFM76YlEK2xZHfQ4iFg5abmpCCVulcJyNrurV7\nsoygj/u+vFSxKjo2amiIoLMpyBqKim5NLuMJwxaYt+7SERwNBSyJCSdkrVrnIcQ4wkz2czsxfwAI\ng9BWmI5RNBzw3W3MZCsY/QmpK6SkZbVqucZ+E+mAoYMEANdtzeDSYkVUKAmaYJuq0WzU/D+HQTB1\njAIBZ72gqIc2kaxV60I+VlKZ00PoxKY0YJUGgRDyPkLIEUJIlRBysMVx7yKEHCeEnCSEfGQ11+xm\niuZADkKIVa7GW5fPZCsYuWQEMz5HcG4E9uSbZRB8hLqePDGNeCSId14zjJKi+54hIeh92POUSxo5\nhAKH1+02C4HhpU2kaPVJZbbAVziqjAxj0psewmEA7wXwhNsBhJAggL8E8G4A+wH8BCFk/yqv25WU\nFA3xSH34hLvstKw1GAQJc0WZqwlnI7F7CKyiw0/I6InXZnDbzhyGUsbuTxgEQSMlq1gjhEQkyOch\nyJpjDwIjGmrtISgNHkLMp4fQKH3dKazqriilxyilxz0OuwnASUrpaUqpAuALAO5dzXW7laKiW7vk\nWDiIAPEZMrJptw8kI6jSzh+S4+ghcP7MepXi/HwJV4+kVmRMBJuDWrFGEElOjTA36WtGNBxomQ+Q\nNb2+7NRXUrnaNBynU2iHmdoC4Lzt/xfM1zYdJdtQb0IIElKIe4FrDhkZ7vFsB+cRasPEjZ85EQki\nFCBcZYGAUVlFqTGNiu3mWIOeQMBg3f7xiFHOXeAISRZkzbEHgSF55BCUhrBPLByEFApwjdGUtS72\nEAghjxBCDjt8rMsunxByHyHkECHk0PT09HpcYsMoyFqdznnSh8DdUllFOlbvIQDATAerhzLNFraT\nIoSgLxHhnj07V6oJiDHvyI9CrGBzUDMIIe73VMHDQ0hHQy1HaDY2phFCMJKJcslvyx3sIXhOYaCU\n3rXKa1wEsM32/63ma27Xux/A/QBw8ODBzg6Q+6Sk6MiZCznA322sVymW5focQt70EGY62ENQHES8\nBhIRbq+GGY6+RESEjASuFG0ho0SE0yDIrQ1CPiXhxfOLrl+X1fqyUwAYycRwiUNeRdZ09CUinsdt\nBO3wW54FsJsQMk4IiQB4P4B/bsN1O46iUu8h8IaMlis1YTvGADMIbfIQzs2W8OjxKWg6vw5RTdWx\nthvqT0S4y2XZAKG+eNi31Idg88D0gxJSCMko33uqUGmdVM6nopgtyq7Pu9GYVr98jmZjuMThIfRs\nDoEQ8h5CyAUAtwL4BiHkYfP1UULIQwBAKdUA/CqAhwEcA/BFSumR1d12dyKr9Q0pKU73tlG2AjAG\nzgQDBLPF9hiEn/27H+BnP/ssHjl2hfscpSFkBBi7fW6DwDyEeAQpM2S0LHIIggbsHkKSw+uuqDoK\nSr3H3Ug+JYFS98E7itbsIYxmo7i8VPHcNHV1DqEVlNIHKKVbKaUSpXSIUvpO8/VJSundtuMeopTu\noZTupJT+8WpvultpbEhJSEGuTmX7cBxGIEAwkIhgZrk9ISMW5jlxpcB9jpOI10AigtkCnxFjFVR9\niQgiIUPhVcxTEDRSknUEiPGc8bynjl9eBqXAvuGU6zEsJDvl4oHLDX0IALC1L4YqBc7Pt/YSVJ02\nGZNOoTPvqkcxDELNQ0hIIa4QiDULocHFHUhKbfMQ2G5/YrbkcWSNxqQyYISMlioaVI7Q01xJQSQY\nsOY/pDjDAYLNRUnREWcNn1IYyxW1ZUc708Y6MJpxPYYJ5bmNfG2sMgKAm8YHABjNlK1Q9CrCHShs\nBwiD0DYopaho1ToPIeXTIGTi9S5uLhnBNGe38m9/6SW879NP+7jjGopWtSo5/IwqdJoMxSZM8VQa\nLRRVZONhEGLIC6SiYWEQBE3YGz5zyQhUnbasEHp2Yg6paAhb+2KuxwyaBsFNULGxUxkAxnMJbB+I\n4/HjrQ2CqjeHmzqFzryrHkTVKfQqrUsmsbnKXvo8TjkEwOhF4A2/fOm5C3h2Yh53fOJR/NVjJ33d\nu707eGLGj0FgIaPaz5wxh+Xw9BPMlRRrZi3APASRQxDUY3gIxjPmtbM/dmkJD754ET/6+q3WRsOJ\n4UwUwQDB+flmj7hapU1lp4yD2/vx0gX36iTAMCbhoPu1NxJhENoEk9JtDBlpVeo5mNs+Lc2OoWck\n+xJ8OztbwreO8CeGgZpB2D+SxmxR4W4Oc0oqs5+BR4JioaQga/OK+hMRzHaBfpOgvZRs1Xu12L/z\nzv5jDx1DOhbGr9+1u+X3DAcD2NYXcwyRKrr7PIP9o2nMFGTX6wOGhxAWHkLvUFF1fOyhY5jh3J2z\ncwAgGqkZBN5SysWyilCAWHopjN2DKVTUKl69vMx9HwCfXnzj9QHgum1ZAMDZGb48glPIiOVBWKK8\nFXPFeg9hOG1UcQgEdoqybikADKabPYSTU8v4T//0In7+Hw7hyRMz+NCtY9ZY11ZsH0g4esStDMKB\nUWMmA8tTNEIphapTYRB6iYdeuYT7nziNP3nYS8apBtNWj9oeIrb79VqgmWxFo4t7x948AOAxj5hl\nI3532Ww4z/XbjCTcGc48glNjGuul4PEy5ksq+mxv3MF0FDMF99pwweakpOqIWR6CIYJoNwh/9/QE\n/uWlSXz7qOEZv+3qQa7vO55L4OxsqckDd/J8GXuGjMql09PO7xFVp67ndgKdeVcdzvdPzwEAXrvC\nvzO3PATbLp+5t9MepaONOkaMoXQUO/MJPH9unvs+AENym2fUH2OhbNzf67YwD4HPIDg1prGfY8kj\nZFStUiyUlDqDMJyOglJg2odnJuh9SrJmVaKlYyGEgwSf/d4ENL0KSikeOz6NO/cO4g/vPYAbt/fh\nmhbVRXa29cdRkLWmGc1Oni+jLx5GNBxwbVBj1XUih9BDsAX45QuL3Atr2cEgMPe2VbwRMBbPlEsT\nzY58Emc5duzsDbPfHDPoJ9y1aL4hhjNRDKUlnJ3jCxk57aSYBIVXDmGpoqJKUdfiP2T+vi5zyAMI\nNg8lRbfURgkhGEobmkJPn5rFxYUyLsyX8eY9Ofz0rWP4yi/d5joUp5HhtOFtNPYitPIQCCEYbSFh\nUTMInbn0duZddThTyzJyyQi0KuVu1GJSuvayUyf31oklFw8BMNzaE1MFz+8RIAQ/e/sYfuPtewD4\nCxstmvH+dDRktOcv8o2ydGpMk0JBRMOBlmWBQL1sBWPIfINeWRIeQi9SUrQVzbsoKVrdKMwv/+Jt\nAIDT0wUcvmjE8l+3hc8rsMM2bI2lp5ZBCDrLT4xmY64idyz/IEJGPUJFNaZ23bnXiEMemWxdYmY/\nD0BdYjgdCyESCniGQJYrWlNTGmP7QByUAm/440daX9+cbWyppPrxEMrGdKlQMIDRTAyTC3w7dMu1\nbmjTT0fDniEjllexVxkNZ5hBEB5CL3L3nz2J6/7gW77Ps5edAoYnmYgEMTFbwtHJRQQIsG847fv7\nsqFMjc8b2+i4LeojmajrponlEISH0COwhfTG7X1IRILcFT5OOQRCCPJJCdMeO96KqruO3BsbSFif\n6y7T0/SqUdkQDQVrcxR8eAgLZcXyUEYyUUwulLlKXWs7qfrHLBMLeyaVmTGx9230xyMIB4moNOpR\nWImnnymAml6FrFXrRCMJIRjLJXBmpojvnZrFrsGkFVLyQy2k6xwycsohAIaHMLUsW4N7nM4VjWk9\nAgvNDKUljGRj3LvVitYcMgKMXoKvvnART5+acT3XrQkGAG7bOYC37jO8FTcZi5oxClgGwU9i1h6y\nGs3GIGtVK6TTClnTEQwQhBoe/nQs3JSoa8QpThsIEAymosJD6HH8GPySypRO6xf88VwCj782jefO\nzuMDt2xf0X1Ew0FkYmH3kJHLe/INY/2gFNj/Xx7Gq5fry09FDqHHYLuFwVTUFGrj22mzRblxp/+r\nbzUaZF695O5pyA3zW+0QQvDjB41xE1MunoY9oR2LBJGIBH3mEOwGwXCjJzlkfp0UIY3vEcMFDwEw\ntzfdYFoSBqEHYWEYwJ88CpO+bvQAfu1tu/Frb92F/+dHrsFP3nTViu9ryOF5c9LosvOG8T7r86dO\n1G/02HMtqox6BOYh5FOSIR3BKS7nFDICgLeYvQStkmlOuil2rHZ9l12/3UMAgFxK8p1DYAZhRz4J\ngK/kVtaaNeMBYGwgjgvzJevN4YTbTmo4HRVVRj2I/W86wdn4CNTmc9uTyoDRD/Ab79iLD9yyvclD\n9cNQOtoUMmpVdmq8HsR/f991AJrzD9ZzLZLKvcGF+TLCQYJcUsJAkn/6V+OizAgFA0hKIdeYOqWm\nbkqLh5oJcbnlImoVToYxMqaW+TMILLm7I5eAFAq4dmLacZoqBRh5jyoFLjjoxDDcqjGG0lFXT0jQ\nvdgLFfz095RcPIS1Ip+Smp63Vp3KjB+9cSv2DqVwpsG4WY1pImTUG0zMFLGtP45ggGAgIWGhpHJJ\nObNOZafkcCYWdpVy0KoUlLYuU+P3EJgipORrjsJCqeYhhIIB7BtJc1VXOU2VAoCxnJEIbxUakF2S\nb8OZKJZlTYjc9Rhsg5KKhrgr94CaQWj0ENYKw0Oo1CW6ZfP95FZ2ytg+EG96xns6h0AIeR8h5Agh\npEoIOehyzDZCyKOEkKPmsf9xNdfcaCZmixg3K3tYCSePlHOrDsVUNISnTk47Nph5JbAAY6EfSks4\ndsl51y43COsNJPlDRhVVh6xV68Z3Xr81g5fOezflyZpzddSuwSQCBC1n1qouHsKIWXrKM7tW0D2w\n0M/N4wM4OrnEXWnEKnnWy0MYSklQdWoNawJsHoLH1LOdg0bTqD00qvR4p/JhAO8F8ESLYzQAv0kp\n3Q/gFgC/QgjZv8rrbgiUUpydLWG7aRByVk2/t0GQzUohJ8nddCyMK0sy7vjEY01f4y1Te9PuPJ48\nMeOo88MWXqajtLUvhtmiwjXKkuU27P0Ab96TR1nV8ezEXMtz3ZLKmVgY12/L4vHjUy3PBZp/7i1Z\nQ8OeJ6kt6B7YnIsbrsqiqOi44tG9z7A8BGmdDIJDMyTve3L/SBqqTnFiqhYCU7Ue9hAopccopS0V\n3iillyilz5ufL8OYq7xlNdfdKOZLKsqqbg3WYIPueeLxquY+Ns8ei2xUPuXtbHzT7hwWyypea+ic\n1vQq/ujrRwHUPITbdvJNdgJq/QoDNgmJW3cOIBYO4gs/ON/yXLekMmAYlZcvLrrmTtySbyOmQeD1\nEPQqxZSoSup42NjLq/rjAOBZllw7zzmpvFYMMoNgM1BeVUYMpnz6rSNXLI9ciNvZIISMAbgBwPfb\ned21goVIWFckWyR5SjgVXXd9COxdu8+eqd9184SMAEOIC0BTh6Q9WceE4q7dmkU2HsbTJ2c975t5\nEf0JyXotHgnhF+7YgW+8cqmllyC7eAgAcN3WLCgFjrkkp912YUMpCQECV/GwRj79+Cnc9LHvtExg\nCzaeoqIhGg7UwrCcEu2spHrdQkasOc22qfBqTGOMDSSQiYXxZ985gQ/+7Q+Mc3XjfrvWQyCEPEII\nOezwca+fCxFCkgC+AuDXKaWuJSqEkPsIIYcIIYemp/3JOq83jYsz8xB44vGqRl3jhvZKpdMNSqJe\nJW6MYRedH5bU+qN7D+CqAcNoBAMEewZTOD3jrcPEvJ/+RL2W0i+8eSeG01F86rvu09cMD8H5jeql\nG+9Wrx0KBjCcjuICp0F44dwCgOZ6cEFnsVzRkJTCyMYMg7DI7SGsb1KZFWxMNYSMAgSe5ayBAMFX\nf/k2vO/GrXj18hJKigZVY9IVXZpDoJTeRSm9xuHjQd6LEELCMIzB5ymlX/W43v2U0oOU0oP5fJ73\nEm1BaagQSEcNqV2e0tNW3cZsMQeaR1TyxivzKQmENHd5MoPwzgPDda8bFRDeu2YnDwEwdmRvGO/H\nuVaVQqruet+D6ShySQlHXRLhim6E2JxyLtv64zjHce9A7Q39vVPe3pBg4yjKGpJSsDYjhFPkrqxo\nIKS5nHutkEJB9MXDdSGjVu/lRnbmk3jHgWHDG760LMTtiPGO/lsAxyiln1zv660njR4CIUbpKc9c\nY6XF2Ly/+sDr8ekPvB7Xbs1OqeJ0AAAgAElEQVQ0lanxPkDhYAADCakpXj4xU0IsHLQWRsZYLoHp\nZdlzWttcUUGAAFkHtdV8Umpq2mm891aVGLsGEzg97eyltGrGG88luLtZWQWYn1nQgvZTlDUkpJAV\n1uQNGRUVHfFwsOV85NUylI7Wed5yC20xJ/ab3vDRS0u16rluDRm1ghDyHkLIBQC3AvgGIeRh8/VR\nQshD5mG3A/gggLcSQl40P+5e1V1vEE6L80CST77CreIGMGQw3nXNiDGyr9EgcOYQAGA4IzV5CItl\nFf2JSNMbZpz1AngslLNFY0iNk4Z8PiWhpOhWYq8RWXWX3ACMGOtZl51+q5zL9oEEZgoKVy/CnLmw\niKqkzmZZ1pCUQoiGA4iEAtwho5KiWdPS1ovBdLQ+h+DDQwCA0UwUkWAAF+fLvd2HQCl9gFK6lVIq\nUUqHKKXvNF+fpJTebX7+FKWUUEqvpZReb3481Po7dyZO4Zv+RAQznH0IXg/R7sEkzs+V63btftQR\nh1LNsg6K7rwoj3BKSc8V6uca27E6pF28BLdrM8ZyCcwWFcdKo1Y5lzEzF+JmTOywkNdsUfE1JU7Q\nXoqmQSCEIMshfsgoKfq6lZwyhlL1G61WxRJOEEKQT0mYXpaF/HUv4dQslUtyhow4HiKWaP3cM2fx\nsYeO4fxcyapK4NmRDGUcdFdU5532QIKVzLY2ZksV9+E8VsLNxSB4udZMutttkLnbz7xrkF9Paa6o\nWDMoRDNb51KUNSTNmR998Qh/yEjW66Sv14ORbAzTy7LV4NmqnNqNXErC1HIFZUVHgHRxUllQw2m3\nzqt4qrbIITAOmLNeP/7NV3H/E6fxj8+c9RcySkcxV1TqlCPdKn36zfI+r+a0oqIjITm/4ZhefCsP\nodV9b+s3egqclE9b5VzGOfWU9KrRYXrNFsPQ8paqCtpPwcwhAMZz5TZxrJGyqtUNx1kPxnNxVClw\nfs64J57NXSODpofAfs71zHmsBmEQfKA4dBkOJCWUVd1xGEbjuV6L+lBaqpuodnRyibvslJ0P1JfI\nGfIRzecmIkFEQgFvgyBrri55Puk+E5pS2lK2GzByJ4CzQWn1puPVU5ovKYYuvTlHmrf7VdB+irJe\nN/f7xJVCSzVc+3nrbRC2N3iyisdz7QQLGbHQWKciDIIP3JLKgHdzmqJTTw+BEIKP3nM1AODarRkc\nmVz0nN9qp9ZmX99E4/TwGhVS3t5NSdZcXfK+eAShAHFc0FXdFOVr8TP3JyIIEHeD0OpNd8O2LF44\nt9AyWcz6Q3aaISY3AUHBxkIpRVnVreTw/tE0FL2Kk1PefTJlZf0NAtMuYwUfrSYYupFPSpgrKVgo\nq64edycgDIIPnDoUc5wzihWXnXojP3Xzdjz/+2/Hj924FfMlFefmjMQpX5WRYRAaE2Bu1+1PRDDn\nIbtRVHTXHU0gYMiAOy7oHAJgwRbne4XYfu6N49CrFJ975qzrMUzRdUeOGQShkNqJWONSzWeFhU55\nVE+LirZuTWmMvkQEmVgYp6YNg1CQNaRcZpy7MZKJglLg+OVl4SH0Co2NaYAtOeux01Z196qZRvoT\nESvBzDptuZLKZgjGXmlkGATn3YxhEDw8BKV1jHYw7dyLILtMiGskn5IcZbu9Qmzb+uPY1h/H2Tn3\nSiNmpIczUcTCwZZDiAQbB6v+YuHS8VwCsXCQa+ZGSdERX+cqIwDYN5yy1ISXK7UEOC83jfcDAM7N\nlYRB6BVUhwSvFTLy2Gnz5BDs7BtOgxDghXPzTdd0IxsPIxoO1FXTyJp7PX8+aSTv3OY5yJoOVact\nXdy8h4fgdd+DZvWF0/le545koi0Txcwg5FMS0jH3IUSCjaXcYBCCAYKrR1I4ymUQ3EOaa8mB0Qxe\nvbwEvUpX5CGM5xKWcN96l8muBmEQfOC0yFlD61t07AJ8VUZ2ElII47kEigobxuF9LiEEo5lYncBd\nq1j8D107gpmCgi8/d8Hx6yW5XszPicG08w6/NhDIW3LDaQKaonn/vkYysZalpNPLMiLBANLRUMsh\nRL3I73/tMP78Oyc2+ja4cJqLfGA0g6OXlkCp+1wESikqatWSdV9PDoymUVGrOD1dwHJFRSrqXIrt\nBiGGkQOApOTv3HYiDIIPnATXouEgduQSeHZi3vNcv/ol2/ri5jUC3OeOZmN1Cqetcghv3TeILdkY\nvnfSWfitqHhLC+fNPozGOQy8EsFbsnFMLctNTWM8HsJoNoorSxXHGRCAMUEulzS6tNPR8KbyEP7x\nmbP45Ldf2+jb4KLsEF7cPZREQdZaSqNYFXguAopryY68kVg+MVWAqtMVhX1GMkaZdVJ4CL2B4jIP\n9Y69eTxzerZl6anXXGQnmLz2QIOwXCtGMtE6D0FW3RdWQggOjKZdXXM2fKRVjHZbv1Gj3SiUV0vA\nt374x3KG0TvXkAuQ1SqiHueOZmOoUuCKy6IxU1CQM5vn0rHwpskhsB13t8Bmfts9hFZNiwy5YVb4\nejJqzuE4ftlohkz7DBkZ38PI8fHNgtsYhEHwAauNb2wquefaUchaFZ958ozjeZRS3/onACzJiL4E\nv4s5ko1halm2FmS3MZaMA6MZnJktOuoRWcNHWuyG3CpCWHOc18/MNJXONLzxvZLZgG2cpkseYWZZ\ntkJ66ejmySHwCv91Co1JZcBmEFrO3WaexfovY7mkhFCAWN3xfkNGQG0eSSdXuwmD4AO3sM+N2/vw\npt05fO3Fi47n6VWjJt+vfgnrJo6H+XcjW/tioNQYlKPpVVRp6zfM3uEkKAVOTze/8Xi05ncPJREJ\nBpq8DN4hIqzp5y++e7Jujm6Ro76c7domXfIIM2bICMCmyiHYZ3O3isGvB5pexUOvXPJ1XebR2CWs\nR7NRhIOkpUR7pY0eQjBAMJyJ4rhpEFYSMmJzyZcqnfscCoPgAyMx7Fw6euP2PpyZcd5pr1QDfcBF\nVK4VtZ1ViSuOX1tUm3fZLIfQamEOBwPYOZhs0hXi7bDOxMKIhYN45eIiXr5oeBl6lULRqp7VI8xD\ncGpOq1YpZouK5SEMJCUsVVTPjvJe4OsvX7I+L7dZ0O9vnzqDX/788/jGK5e8DzZprDICjG70q/rj\nLZvTKm30EABgNBOzNk5+q4wAo5kSAN5349Y1va+1RBgEH7RKDB8YzYBS4NXLzfH42pQkvyEj/twB\ngymBTswUuRZlZhCcwi5s5+a1U9+Sba724U0qA8AX7rsFQK1/osRhiADDbU9JIcd7Xyir0KvUMgj7\nhlPWkJJeZrmi4usvX7Ji3LyqoWsFew4uLfDLhDCD0LjT3z+aaVl62s4cAlAbUwusLGQ0mI5i4uM/\nhHe/bmQtb2tNEQbBB63yAExA7cXzzd2Vsg/FUjsracnPpyQkIkGcmSnWYqwt3jADiQgioYBj+Sav\nsN5oNtokRlaL73r/DGynz8pXSw5liO7XjjmGjOw9CABwYIuR6zjK0f3azTAjfp25G223QQiaczMU\nl8ovJ2SXucj7R9K4uFDGgovyKfMQ1mtaWiO37RywPneThO92hEHwQSsFzpFMDDvyCTz+WvMcaNWq\nTvKncMia3tjEJR4IIdg+kMDZ2SJXHN/oXWhe0AFbqMvDsxnNxrBc0ermOPgR5RtISoam0RLzEPg8\nEwAYydZXVTFmzMoj5iGMZqLIxsM42uMeAvu9M+FAt8V0vWDendecDTtuHgLr1nfzEmq9Lu3xEO7Y\na4z03TWYtGRieo3O7aHuQLxkb+/cM4jPff9sU2jJj4S1nX3DaXzhvlvw+qv6fJ23pS+Gc5w5BMC9\nwYv3vu3VPruHUnXn8hiEYIAgEQnhz797EjvySeweMrSHeDpQh1JRxwVj2vIQDKNKCMFV/XFcmOeb\nxdytWAbBVL6db7OHwBo0/UyoKytm6KfhWWEG4cjkEm7blWs6r6K210PIJSV86RdvxR7zGe9FVjtC\n832EkCOEkCoh5KDHsUFCyAuEkK+v5pobiVdz2Z6hJBSt2tS5u5qxebfsGPBtSEYzUUwulrl3ULmU\n85Af3vveYuYhTtkqlfzkEABjhCIA/PVjp3x5CPmUhNmiAr1aX9Uy3eAhADC7uHtbApsZYvZzF+T2\nGgTWSHbRlkO4vFjBv/vUUzh80TlcV1Z1RIIBhBqes4GkhOF01FXkruIjLLlWvGGs33VgVC+wWtN6\nGMB7ATzBcex/BHBslddbUz79+Cl8/vvuapmNVFS9ZQLLbWCMnzGYa8GIGcKZMfWVvBblgUTEcXKa\n04Q4J67ZksFwOopPP37Keo23Ma2RHfmEb4OgV2mTSN9MQUE4SOrevCNZQ/uo3aWY7YSF+ViFWkFu\nb5URe/bPz5Ws3/M3XrmEVy4u4hf+8TnHcyqq7qqKe2A07SpyV0sqi8j3WrHamcrHKKXHvY4jhGwF\n8EMAPrOa660lsqbj4998Fb/3wGHuBaKi6nWlcY3kk0boZGqpea4xAITbVR5n7thfNhPcXuWr/YkI\nlita00AS9v9QoHXuIxoO4v03bcOL5xespCZvYxrjb37acDCXKirKVpWRd8jIba7zTEHGQEKqayIc\nzcRQVPSOrgNfLexvlo0bhtCpDHq9KMgaLi1WMJCIoCBr1ibjyRNGXu3iQv28cMZyRUPKpa5/12AS\nZ23GxU4tqdy5UhDdRrtM658C+B0AnqUHhJD7CCGHCCGHpqebE7RrxbNnatpDr17mSzSWPJqlLA+h\n4OwhSG3yEEbNmP7TpwyNojGzG9gNVjHROMdW0aljZ7YTO/JG3P/sXG2qVChArKoTL96+fwjvPDBk\nTpXy5yEAzVPbZgoycql6QzhiSgc4JaF7BWaIE1IIUijguACvF0we+oeuNcoqv/CDc3j01Sn826lZ\nbDfLodkxdhbLCrJx503LSCYKRas6erC8AooCfjx/k4SQRwghhx0+7uW5ACHkHgBTlFJnf7EBSun9\nlNKDlNKD+Xye55QVcfRSLS757MQc1zllDw+hPxEBIWhS71Tb7CEwmd3vn5lDLil5dlUyD6JxpoMf\nQb7xBu0Zr/GZTgymophallFS+Q2C2xhOo0u5vo+Djfxkg3N6EXt4MhUN1RmE164s45UL61d2e8TM\nEdxt1tn/ybdew8/+3bOQtSp+5c5ddcfYmS+plkfTyAhrnHRIUgsPYe3xfMdSSu+ilF7j8PEg5zVu\nB/DDhJAJAF8A8FZCyOdWcc9rwlxRRSQYQF88jCMXvXXXATOH4NG12x+PuHoI7coh5FOSJdkwnot7\nHF3zEBrj8K06sxthInWfe+acOU/ZfQ6DG/mUhIWSamm98PQhMK+sMVk8s6xYBoDBQlDt7t5tJ/bK\nsIQUqgsZffSBw/jFzz23bjmUiVlj+MuN2+ur4sJBgh++fhSZWBgnHDqPF0qKq0HYYhmE5mIAJl3R\nrvfVZmDdf5OU0t+llG6llI4BeD+A71JKP7De1/VivqigLxHGgdEMjlzi2zWVFB1xj93IaDaGMw26\nQKupMloJhBBrd8xTIuc25MfPDIdUNIykFMJTJ2fw/LkFcw6Dv50b82y+dfQKCOHLIUTDQQyno3Ui\naCenCpharmBrX70xZAaml+Ur7P0fiUgIBTNfUq1SHJlcxMWFMtes4pVQUjQkpCDCwUBd3mogISEa\nDjp2tAPAYlltGTICnMN8bNMR4AxLCrxZbdnpewghFwDcCuAbhJCHzddHCSEPrcUNrhdzJQV98Qj2\nj6bx2uVCnbCaE7VB4K0XuVt3DuDQ2bm6nZnfEsy14JYdRlflz79ph+exzHg0hrr8znBgEhSnpgpG\nyMhn9ce7XzeMLdkYXjq/gIFEhDv/MJaL18kk//3TE5BCQXzglqvqjmN/u8bZC72E3UNISrWQ0dm5\nkjVs6dHjU+ty7YpatUKq//4N26zXmXz6aDbaFPqhlGKhpCLrUsrZn4ggFg46ii/KbRqOs5lYbZXR\nA+buX6KUDlFK32m+Pkkpvdvh+Mcopfes5pprxXxRQX8igm19MSh61ZI6cEPWqqDUO4xxx548VJ3W\n5SVUlzkK68lH3r0PT/7OWzwTygCQjUeQT0k41qDD5HeGw77hFEIBggmzS9rvzyuFgrjhKkNyoTH+\n34rxXAJnbaqYlxYr2D4Qx0BjyMhcrLptXoAfZL1W7puMhiyBwuPm3zYeCeKx4+tTrFG2lWX/1jv2\n4jNm5djtO42mMmN4U71BKMgatCp1DRkRQnDLjn6rUsmOUa4q8gdryaY1r3MlBX2JiDXFyE1CmWGN\n+fN4ANlkJXvMc6WdyqshGg7WiXF54TQoR/U5w4EpVE7MFlfkIQC1mDGrHuJhbCCB2aKCeTMHMleU\nrTCYHStk1MMeAtMFYjkEFjKaNgsG3nVgGM9OzK2Ll1SxedCBAMFd+4fwT/fdgo/eczUAoyN+qUHi\nhGktuYWMAODOvYOYmC3hXIMUtqxVRQ/CGrNpf5vzRQX98UitFNGj1b7MWfniNGO5lkPo3FjngdE0\nTkwV6hYKnrnGjYzlEjg9bQjrrcQjYjFjr94HOyyJ+fSpWQBG1YqTUqwUCoAQoNLDHoKi13IISSlo\nNaYtmiXFt+wYgKpTXJhf+9LbstJchXfzjgErl8QmhtlzbMwgtOr+ZX/fly8u1L1eUVsPfxL4Z1Ma\nBL1KsVBW0ZeIWDvSX/r8800NZXZKCl+JWzgYQH8iUlcXvxEegl/2DKWgVynO20ZZqjr1bcSu25rF\n8SvLeO1KYUXy3cOmx6Z55HTsXL8ti3Q0hMfM2PhsQUa/QwiCEIJYOGj9LTsZvUrx2196CS+eX/A+\n2Ia9os3IIahWnD4eCeIqsx9gPXoxKlrrTv7bduaQiATx5989Yb3GuulbhQj3DKUQDpKmjmXhIaw9\nm/K3uVhWQSnQHw/X7UweeMF54hngPObPjXxSqvMQlDZXGa0ENrnMPsrSb1IZAN6yLw9KDQ+J7Qj9\nkDATkJrObxBCwQBuGu/HC+cXoOpVLFU0V2MUCwe7ouz01HQBX3ruAn7rSy/5Os/w6ggCAYLxXBIV\ntYqTUwWj1j8WtjZAfuYV8OLkIdjJpyT88lt24dtHr+DfTG9u2ixkGGwRIoyEAtg1mGoyCBVV95y7\nLfBH565Q6wirt+9LREAIwYffOA4AeOrkjOs5NX0dDjmFtGSJfAHt70NYCayxzJ6cbSX37cY1oxmw\naA+T0PDDG8b68a4Dw/ijHzng67z9oxmcni5YSct+hxwCYOQRuiGpzDp63apv3JBtyfw7Tbnmx45P\nW93AQ+koCHGekLdaKmrV04P+uTeOIxQgVpJ4umFuhRs3XJXF82fn6+RVWmkgCVbGpvxtMokG1pD1\n0Xv240O3bsehifkm1UyGNeYv4v0ra/QQVN2QcejkeulMPIxsPIwztnp+VfffbRwIEGuY+MgKNOOj\n4SA+/cEbsWvQn8TwgdE0qrSWR3DTb+oWD4Hthvt8DmJRtKpVeTOajWH7QBzPn5s3SjvjYURCAeSS\nki95al6MsuzWz0s0HMRQOmr1I0wvy0hFQ56G5M49eRRkDc+drUnOyFpVeAhrzKY0CJaHYKtsOLAl\ng7Kq1zU42WGCazGOgfcj2SiuLFUsXZmVhF42gt2DyTppgZUklYHaIjaUbt8QEaadz/IIbgYhHumO\nHAJb+FjXNi+N5b47cglMzJYwb+sG3pKNtS2p7IR9wt7UcqVluIhx264cAgR45vSs9ZrwENaeTfnb\nZOWJ9jF49mEcTlR8SO3uG05Dq1KcuGJ0hPrp+N1I3rQ7j5cvLlqzEVZ63z9963YAsATN2sGWbAyZ\nWBiPHDMMglu4KtoFHsJCScEL5wyD0Cgn4kWjZMhYzpieZ3gIxvM+NhCvCw2uBZRSz6QyYzQbw8mp\nAl44N4/LixWuEuOkFMKOfLLu/Sk8hLWn81epdWCu1Owh7B5MIRoO4LvHrjiewzOfmNE4+q/VLOZO\n4s69RkL4CTO+u1LP5oO3bMfhP3in1ePRDggh2D+Shl6lIMTdO4n7zCFUq7RJFny9ee7sPKrU8Nj8\nGgSlIcw3NmDMl5gt1rSdxnIJTC6W17QXgTVu8hiEfFLCXFHBe/7qaTx/bsESKPRi/0i6biZ2Ra2K\nxrQ1pvNXqXVgvqggFg7WdR1HQgH8zG3j+NqLk45jFv0khscGEkhKIbx0YcE8l3Z0QplxzWgGuWQE\nj75qGgSdrshDIIR4KqyuB9dsMQxxOhp2NWSxiD8P4SNffRl7PvrNNbk/XpjU8zVbMpgvKZ6yKnZk\ntd6I7xpMWp+//yZDTmJsIAFKgXNza+cl+KnCyzV4BLye5IHRNCYXK5aHL6u6kL5eYzblb3OuqNaF\nixhv3z8IAFaox44lGsYRMgoECG4e78cTJ6ZBKe0aDyEQIHjz7jyeNO9b1auIdHAzXSO37jT0mxZb\nxN1j4ZAvD+GLhy4AaO+gGZY3GBtIoEqBBR95hOWKhlS0Zoxv2TGAv/qp1+PhX3+z5bExOZO1DBvV\nii68DcKHbh3DP5m6V4Dxc/JwYDQDoBbWNfoQhIewlnT+KrUOzJcMpdNGxhxq8Rmyz9LRO/cN4vxc\nGROzJagav4z0RnPjWB/mSyouzJe7JhnOuHVH8yD2RhJS0NL38UPjvAUePvi338eXn7vg+7yligZC\ngK19xgLeysA1n6siHa0928EAwd2vG8He4VrVVr8ZKl0oNYej5ooK7vnUk3XVPDz4ybHFIkHcbIov\nAjXpdC9qeb5F6FVjoyUa09aWTfnbnCsqdfkDRn8igpQUcqw08msQbthmiLQdv7zUNR4CYN+FLXZN\nMpwRiwTxqZ+4AQ/+yu2uxzB9H78zAaZ8GoT5ooInT8z4biwDDA8hKYWsqiA/lUZLZRVpj96FpOlB\nOHk9z5yexeGLS/jRv37axx3za33Z6TN/vu2cHkJfIoLRTBRHLy3VcnoiqbymtD/Q2wHMlxRLe98O\nIcTS4mmElfPx9hKwuOjEbKmrFtZ9wykEAwSHJuahVam1eHQL/+660ZZfT0ohaFXqO9zg10NgjWUr\niXEvlVVkYrUuej8ewqJ5bitYN3jRIXT2qm3EZUXlqxoCaiEjP7/Tz334Zvzr4cueM7/t7B/N4Mjk\nkjU+U3gIa8um/G3OmdLXTtw83o+nT83geMOcZb/hk1Q0jFwygomZIpYrGhIcHc6dQDQcxFX9cTxn\nlj36ebN2AyzZzZMTsHsR08v+pB5YnHvLCrq1WdiH7fSXKnwGQdOrKCp6XcjICSkURCQYwHKl+Xdg\nL+u87KEAbMdPUplxYDSD33zHXq6Z3Yz9o2mcni5YzaXCQ1hbNp1BUPUqliuaY8gIAH7lLbuQiobx\nR18/Wvf6SkZCjg0kcGam2NIAdSIjmSgOmw1qbr+nbiVhGgSe4fNLtgXTb8iIhR1X0p2+VNaQjoWs\nhX2pzJfzYAt8Oua9+UhIQUejODFbtDwMP/IWFR9J5dXAOtJZBZ/wENaW1U5Mex8h5AghpEoIOdji\nuCwh5MuEkFcJIccIIbeu5rqroSZb4byL6ktE8At37MBTJ2esbkrAlATwaRB2DyVx7NISppflLjMI\nMWuoj9NcgW4m6cMg2IcmOY1+bAV7zpi8sx9Y2Ict7LweAjvOy0MA0DRvmbFQUq3yXT8CeCsJGa2E\nq4eNe3vlwlJbrrfZWK15PQzgvQCe8DjuzwD8K6V0H4DrABxb5XVXzHzReNO00ohh4yfrZBxWkBh+\n4648lioayqreVaGXLTaV0pVIWHcytZCRd+npbKFWhXPWRdLEDfacLZQU3wlsFjKKhYMIBwl3DoEd\n55VUBozfw3KDQahWKeZLCvaPmAbBh4ewkqTyShjOGOJ8Z2aM0nDRh7C2rHaE5jFK6fFWxxBCMgDe\nDOBvzXMUSqk/kfc1hHV+9rcIhewbToGQWscuYDb8+EwMv3F3rQzSTX2zExmxxb27ybPhwUqo+vAQ\nbrgqiwmfNfvMQ9Cq1DF524qlsopUNAxCCNLRMHeVEQsteSWVAcMgNP4OlisaqtTo8u5PRHx5RZU2\neQiRUAD5pIRTZuHHehugzUY7zOs4gGkAnyWEvEAI+QwhhK/ObB1gtdetRvbFIyHsHkzic8+cw9fM\nGQmK7n8kpP2N2U0eAqt/B4B0l1UZecGathp3x04wTacbr+rDXFHxVe0zX1LAcqVO9f5uVE0Dwu4z\nHQvX5TJawUJGKY6/mVPIaN4m6dIXD/sKd7E+hPXOIQDGhoV1WTfOzRasDs8VjhDyCCHksMPHvZzX\nCAF4PYC/ppTeAKAI4CMtrncfIeQQIeTQ9PTaDwNnuzUvaYW/+MnXY0s2ho9/81Wj23gFQ+MBIGd6\nBt0Uerl5vNY05KcCpBtI+Kgymi4Yi/rrzRGOvGEjSinmiyrGzY5gFj7igTXNseczHQ1xGyKm4spT\n0ZaMNoeMLIOQCKMvHsFCmd+QWTmENoRw7CFNHqVUAT+efz1K6V2U0mscPh7kvMYFABcopd83//9l\nGAbC7Xr3U0oPUkoP5vN5zkvwY8lYe+xk9gyl8OE3jePyUgWzRWVFVUaAoUcD1EIV3UAkFMAjv3EH\n/vfP37zRt7Lm+DEIswUZffGI1bPCO0OgqOhQ9Cr2md3Bl1uMZm0618xtsPtMx/hDRrzPNgAkI80e\ngn3gfTYe9mXIyqqOcJAg1IZ+GybBEQoQq3lPsDas+1+PUnoZwHlCyF7zpbcBONrilHXFj+YK03yZ\nmCmaVUb+F/VP/vj1+L27r7YSdd3CrsEkbtvpLQXRbbDds1MNfiMzBRm5ZG3u9kXOqhsmvsa6vv0k\nZwuysQgn60JGnAbBx7PNOrbt2ENGmVjEV4isrPA3sa2WnXlDsE+r0p7zYDea1ZadvocQcgHArQC+\nQQh52Hx9lBDykO3Q/wvA5wkhLwO4HsDHVnPd1VDyUQ1h1zaSV6jr05+I4OffvEM8uB1CMECQkkJc\ni+xsQcFAQkI2HkY0HMAlTg+BLay7BpMIBwkmfZRvFmQW0jSeTyOpzJdD8PNs98XDKCq6JQFh3Ldq\nfc3IIfCHjCoq33CctWIfYuoAABhUSURBVIBpGgnWnlVlDCmlDwB4wOH1SQB32/7/IgDXPoV2UlaN\n0E+Qo2Foa18MwQDB2dlS1wm9CdzJxMNY5EiYzpcU7BtOgxCC0UyMu1GLLay5ZATDmagvD4GFcZgn\nkzFDRpR674b9PNuDaSP2Pr0sY2ufERJbKCkIEMMIZU2DwfvcV1S9LQllAHVCfYK1pbdKSDioKDri\nnA9uOBjASCaKC/MlyCtoTBN0Jtl4mEtS2i4lPZqNce/0WcgoG49gJBPz1eDFQlm1HEIIil7l0l7i\nHWEJ1Iba2w2CMWYzgkCAIMMUUcsK1wCbchs9hGg4iLuuHsJtOwe8Dxb4YtOtcCUfbxrAqGKYLshN\nk6gE3UtfPMIVDinImlXts30gjpNTBai69/Q0qxs+HsFIJopLS/49BKvsNMqveFr2sdlhi7xdkmO+\nqFpJWqZEyuNJAUC5zdPLPvOhg/gPbxxv2/U2C5tuhfO7k8mnJEwtyeZ0pu6pFBK4k4l519hrehUl\nRUfKXJDftDuHgqxxzQmYLxrlqulYGLmkVNfx7AUrO7VXGQF8iqd+nm27h2Ddd6kmC8/+neUc4VlR\ndMSErlDXs+n+gmXFX6xzMBW1PASRQ+gNeEJGhYad+u27jIqrH5yZ8/z+cyUF2VgYwQBBLimhpOgo\ncQ7lYSEj5plkfCie+nm2BxIRENLgIZRUyzNgBsOu59SKita+kJFg/dh0K9xKPISFkmoM9BYGoSdg\nIaNWs4qthdk0CKloGKloiGvo/XxJtbSymDggr5dQlDUEA8R61linOE+lkZ9nOxQMYCARqfMQFswc\nAgDkzQ7gqSU+g1CUtbYllQXrx6Zb4Uq+PYRaJ+RKtO0FnUcmFkaVtpavsKSkbTIQ6ShfT8C8bSIf\nW1inOXfaRVlDIhK0Kor8zETw+2znkpJDyMi4XjYeRjhIuO/7ypLMlXwWdDabziD4rZfO2wwCa1QT\ndDdsF9xqt79s6QLVOmGNrmHvnbo99OLXQ1iqaHVqpX6mpq3k2WaDfyqqjopatX43hBDkkxKXh7BU\nUVGQNYxkhEHodjadQSir/JUYQK3bFKg1qgm6G9Y1/uyEez6A5RDsmlfpaIir2mfRFnrJJf3F4pca\nRmCmrJARn4fg59keTEUtD8HepczIp6NcHgIrqx0VHnTXs+kMgl+3eti26xlKCyGtXuDqkRSG0hIe\nP+4unshCRnbl0AynjMSCbVFn8uGzvAbBnIXAkEJBRMMBLsXTss/msLxZUs3E+Iz7rV073xBScoM1\n7I1mhYfQ7WzKxrRY2N+P/Wfvvx7HLy8L+YkegRCCW3cM4JnT7h6Ce8iotUFQzXJVZhCi4SBS0RBm\nOENGi+WaSqp1Xc6ZCH6f7cGUBFWnWCiplodgl4XPpyS8eN67zJZ5CEx0TtC9bCqDQClFyWfICADu\nvX7LOt2RYKM4MJrB116cxGxBdtTUX3LwEIykcuudOov128M+uaTkI2SkNQ24ycTCnjkERauioGhc\nsxAYVi9CQXYMGQ2mJMwWFWh6taWKKfvZ8kKKuuvZVCGjilqFXqVWKaFg88IE0o5MLjl+faGkIBYO\n1slFpGMhFGQNWotuZbZw22WZc8kIv0FoCBkZ1/UOVV1ZqoBSf5VwVn5jWa4TtmPkUxIo9W5OK8oa\npFAA4TZIXwvWl031F2SJwoTHcBxB77Pf0yCoTVr7bOfeSjrbaa4xb7cyCzc1zkQ2ktmtPRM2q2HE\nRxy/z8wXLJRVLBSbQ0aDDt3MTizL/jwTQeeyKQ1CsouG1QjWh2zcmHNw9JKzQZgvqU1jVi1doRa7\ndab9Uzc+ldNDYIamMWSU5ggZsfnHfuL42Zjx882XFMyXVCQiwbpufBYCmlpuLc5XlDWxyeoRNpVB\nKFoGQUxZEhhhoyOTi45fWywb8hN2rCaxFrt1txzCfEn1FMareRf1iytPddPFBf+VPswDWiipdV3K\nDCe9IyeMZjphEHqBTWUQatLCwkMQGInlMzNFq6LIjlPIiHUtt9qtOxkElrSe94jFs0qilNQcqloq\nq1A0d4NyebGCTCyMuI+FORo2SloXyyrmSooVQmLwGoTliibycj3CaiemvY8QcoQQUiWEuA7AIYT8\nJ/O4w4SQ/0MI2ZCC5aJDs5Fg83Lzjn5QCnzv5GzT1+adcghxjpCRg0GoCdS1zgMwpdPGxfV1WzKo\nUuDQWfcy2cWyWpcQ5iUbM3Sdzs+VsDUbr/uaFAoiEwvXCeC53bd4T/UGq/UQDgN4L4An3A4ghGwB\n8GsADlJKrwEQBPD+VV53RTRKCws2Nzdu70NKCuGx41N1r1NKjZCRWw6hhYewYMbi7RU3KfN5K7TQ\nTgIMtVIATWXRt+/KIRwkeKxFI91K4/jZeBizBQXn58rYnos3fX0w5d2cVpR1YRB6hFUZBErpMUrp\ncY5DQwBihJAQgDiAydVcd6VY3afi4RXAmIh341gfXjy/UPd6SdGh6tQ1h+AVMmpMCrMdf+NQ+0aK\nLgYhIYVw03h/k+Gys7wKg3D00hIUvYpxB2mWfEry9BCWKyKp3Cusew6BUnoRwJ8AOAfgEoBFSum3\n1ut6FVXH//vQMXzryOWmrxVF2amggQOjaZyYKqCi2ofNsxLM+oU9EQkiGCCeIaPGslGWcC3IrRPD\nZdODjTnkAe7cM4jXrhTwN0+cdjy3KGsr2uhkYxGrQslJvDHP5SFoonKvR/A0CISQR8zYf+PHvTwX\nIIT0AbgXwDiAUQAJQsgHWhx/HyHkECHk0PS0u4vshhQK4MEXJ/HACxebvlaUNRDSvAMTbF72j2Sg\nVylOXClYr7Fpao0hI0KIZ09AozgdUOt2btW/ABieCWAYnkbe/bphRMMB/PFDxxyntq00ZGTX6tqR\nbzYIgykJU8sVUOo8O0KvUpRVXVTu9QieBoFSehel9BqHjwc5r3EXgDOU0mlKqQrgqwBua3G9+yml\nBymlB/P5POclahBCcOfePJ46MdNU5rcsa0hGQkKTSGCxdzgFADg5vWy9ZnUbx5oXOa+u4cVyczI6\nyZlDYAbBSaBua18cz3307RhMSfjDrx9tGu5TkFdW6cOUXwE4zjPIpyRU1KrrvdeaPcUmqxdoR9np\nOQC3EELixFiJ3wbg2Hpe8M178liWNRy+WF9jLsrjBI0wBdvGQTFAs4cAGInlVjmEhbLS5CGwnbtX\nDqGkGNPSIi4SEAkphP/8rn146fwCvvZivQdckFdW6cM6tt1gRsItbMQS4SIM2xustuz0PYSQCwBu\nBfANQsjD5uujhJCHAIBS+n0AXwbwPIBXzGvev6q79uB1W4wZBo2yBE7NN4LNTVIKIRoO1A2CWXDQ\n9WFkYmFL98cJp6RyJBSAFApweQhx27Q0J95zwxZctzWD/+9fX7XmNGt6FRW1uqLmsN1DSQDAvdeP\nOn691q3sbBCYFy50jHqD1VYZPUAp3UoplSilQ5TSd5qvT1JK77Yd918ppfvMUNMHKaV8Sl8rZGtf\nDJlYuMkgzJfUOr13gYAQYgyKsUlLOOkRMYYzUVxZdJZykDVj6lijQQCMPEKrkZ2Asdv2ym8FAgS/\nf89+XFmS8ZXnDS+hKBu79JV4v1IoiO//32/Df/uxax2/7qVnpFgGQYRhe4GeNOuEEOwfSTfp1MwX\nhYcgaCafkho8hGalU8ZoJooryxVHGQqnpjRGUgpxlZ3ydBrfuL0PsXAQZ6aLAICCsjqNrqF0FFLI\n+VwvD0HTjVyGW5hL0F307F9x12ASp6cLddUR8yUF/cIgCBrIJ6U6D8GpS5kxko2BUkNuuhGmaOo0\nXyEZDXE0pmlcFXCEEIxmo7hkTiorVNavnDoTCyMSDLh6CMwwtpqXIOgeevavuH0gjuWKZsWD9SrF\nwgrb+wW9zWC6vtZ+wUHplMHmBl9yCBtZBiHRfC6Ph+BnJvJoNoZJ8x7WU9adENKyF0GEjHqLnjUI\nbAzhmVnDrV4qq6AU6HN4swo2N/mkhMWyajWnOSmdMkbNun02f8AOk7jOOUwOS0phzxxCUdEdm9Kc\nGMlE8dL5BfzzS5O16ybWZ2JZzuxFcIKFjERSuTfo2b8i67o8axqEOYcRgQIBYHgIQG1B9woZAcDk\nQvMC2WphTkVDXJ3KcYe8hRPBgPHW/bX/88KKhuP4oZWekagy6i169q+4rS+OcJDgNbMDdYEZBOEh\nCBpoTJy2ChklpRDS0ZAVv7czU1AQDpKmeQbsPFYN5EZZ1R2b0px4zw3GnG8pFMClxQoioYBjqGot\nECGjzUPPGoRIKIDdgymr9HSu6F5bLtjc5JO15qua0qn7czKajTmGjGYLMgYSkmMfQTLqnUOQ1Sqi\nYb635E3j/fi1t+6CrFXx4IsXMZqJrlsHfj4pYa6kOFZWiZBRb9HTf8X9o2kcnVwEpdTqPhUhI0Ej\nLGQ0tSyj6KJ0amckE3UNGeVS7p6Folcha+5egqxVXcs/He/DDF9dWZJbNsutlsG0BErhOBdahIx6\ni57+K75uSwYzBQUX5svWtCoRMhI0MpCIgBDDQ1jg2DiMmrOYGxf32aKCAZfEbopDAlvWdEgh/rfk\niE2Y7radA9zn+SWfdG9OU0XIqKfoaYNw+y7jTfLto1cwX1IRCQYclSQFm5tQ0Ii/GwbBbC5rETK6\nqt8YJPObX3yp7vWZZRk5hx4EwFvgjlJqegj8b8ktpofww9eN4r//+HXc5/llMG2G1ArNXpEqQkY9\nRU8rUu3MJzGSieIPv34U2/pjyMbDQulU4EguKWF6uVKTvm4RMvrALdvxwAsX8djxaShaFZFQAJRS\nzBQV5JLuISPAXQJb1SkoBSTOKiMA2D2Uwld+6TZcvy2LYGD9nmuWdL+y1MpDEAahF+jpvyIhBJ/4\nMWPndH6ujH4RLhK4wCppFsruSqeMhBTCb7x9Dwqyhr989CTe8T8ex6npAhSt6u4hRFt7CBUz/OTH\nQwAMGYv1NAYAMJSSEA4SnJsrNX2t1qksNlq9QE8bBAB44+6cJdDVqnJEsLkZTEUxtSxjxoyTD7js\n9Blv3pPHlmwMf/adE3jtSgHffOVyy/OYh/D7XzuML/zgXNPXZdVYWP14CO0iFAxgW38cEzPFpq+J\nkFFvsSn+iqxJTXgIAjfyKQkzBRlXlmUEA8RT8yoaDuLPf+IGvGWvMcSJzWX2yiGcmCrgI199penr\n8go9hHYxNpDAxKy7hyDE7XqDTfFXZA/t9duyG3wngk5lMCVB1SlOThWQS0YQ4AjD3Li9D5/92ZuQ\nS0p4dmIOgLuHwDSQGI27bVkzPYQONghnZ4tNozRVTYSMeonOfPrWmF+8Yydu3zWAn751bKNvRdCh\nsMTp0cklx1GSrRgbiGOpoiERCVoVSI1Ew0H85U++3mqMfKVhmp8VMvLRh9BOxnJxlBS9SQZbNUd5\nhtY5jyFoD6udmPYJQsirhJCXCSEPEEIct+CEkHcRQo4TQk4SQj6ymmuuhHceGMbnP3yLo769QADU\ndvAXF8qWceCF9QD87t1XIxV1z1P90LUjeOy33wIATdIXVsiIs1O53YwNGGHXRs9G1asIB4mo3usR\nVvv0fRvANZTSawG8BuB3Gw8ghAQB/CWAdwPYD+AnCCH7V3ldgWBN2TecAlvTBn0ahN94x16c/tjd\n+MAt2z2PTUdDSEqhpk7nitr5ISMAmJitNwiaXhUJ5R5itSM0v0UpZXV0zwDY6nDYTQBOUkpPU0oV\nAF8AcO9qrisQrDUJKQS2x22M9/PAk3MAjFLokUzU3UPo0JDRaDaKcJDgzEx9YlnVqTAIPcRa/iX/\nA4BvOry+BcB52/8vmK8JBB3Fjdv7AADvv2nbul5nJBtr8hA6PanMSk9PXFmue10xQ0aC3sCzU5kQ\n8giAYYcv/R6l9EHzmN8DoAH4/GpviBByH4D7AOCqq65a7bcTCLj56w/ciIWS4jup7Jct2SiONCaV\nTYPAq3a6EdyyYwAPvnDR6s4GRMio1/D8S1JK76KUXuPwwYzBzwC4B8BP0caaNIOLAOxbrq3ma27X\nu59SepBSejCfz/v6YQSC1ZBLStg1mFr362zrj2O2qNR1LctqZ4eMAODOPXkUFR3Pn5u3XlN1KkpO\ne4jVVhm9C8DvAPhhSmlz14rBswB2E0LGCSERAO8H8M+rua5A0M2MO1TsWCGjDvYQDmzJAABOT9fu\nWxEeQk+x2r/kXwBIAfg2IeRFQsinAYAQMkoIeQgAzKTzrwJ4GMAxAF+klB5Z5XUFgq5lu0PFTqUL\nPIShlIQAqS+Z1fSq6FLuIValdkop3eXy+iSAu23/fwjAQ6u5lkDQK4zljOY1Rw+hQ5PKgJFYHkrX\nDwcSIaPeonOfPoGgR4lHQtg+ELf0jwBgqWzM6+hkgwCgqWRWFSGjnkL8JQWCDeDNu/N4+tSs1X8w\nuVjB8DrORV4rRhrmSZcUHTGhANAzCIMgEGwAb9mXR0nR8ewZo2Ln0kIZo9n1LXddC3bmkzg3V8LJ\nKaMfYXpZ9i31IehchEEQCDaAW3fkEAkF8NjxKQDA5EIZoxn/HdLt5kO3bkc0HMT/+t4EKKWGQXCR\n/BZ0H8IgCAQbQCwSxM3j/Xj0+BT0KsWVZXlFkhntZiAp4fptWRy+uIiCrKGs6hhMC4PQKwiDIBBs\nEHfuHcSp6SIefdUwCt1gEADgwGgar15exqVFo9pIhIx6B2EQBIINgk1b+/A/HEJSCuEdB4Y2+I74\nODCagaJV8eirRrhrvaU+BO1DGASBYIMYzyWwd8iQyvjVt+5yHb/Zady5N49MLIxPfvs1AMJD6CX+\n//buJ0SLOo7j+PuDpqZF5moiKWySEB5qraVW9GBCYRKdPGSBIoKXDgZBKEHQsUtWFJFQdAmL/pF4\nMVPP2m7+W9vMFawUayvUbpH17TDfXR7EQ+4zu7PPzOcFwzO/38wDv8/s7H53fs/MblsPppnZ+Eli\n96aH+HTgAltWdVc9nP9t7uwZvPXMCvYc/YmuOTNZmv+z3Dqfbvz36KaG3t7e6O/vr3oYZmYdQ9JA\nRPSO572eMjIzM8AFwczMkguCmZkBLghmZpZcEMzMDHBBMDOz5IJgZmaAC4KZmaUp/WCapN+AH8f5\n9vnA7yUOp5M0OTs0O3+Ts4PzzwfmRMSC8bx5SheEdkjqH+/Tep2uydmh2fmbnB2cv938njIyMzPA\nBcHMzFKdC8LuqgdQoSZnh2bnb3J2cP628tf2MwQzM7s5db5CMDOzm1C7giBpnaQzkoYl7ah6PBNB\n0vuSRiQNtvTNk3RA0tl8vTP7JenNPB4nJT1Y3cjbJ2mJpMOSvpN0WtL27G9K/lmSjko6kflfyf57\nJB3JnB9LmpH9M7M9nNu7qxx/GSRNk3RM0r5sNyn7eUmnJB2X1J99pZ37tSoIkqYBbwNPAMuBjZKW\nVzuqCfEBsO66vh3AwYhYBhzMNhTHYlku24B3JmmME+Ua8EJELAf6gOfya9yU/H8BayPiAaAHWCep\nD3gV2BUR9wKXga25/1bgcvbvyv063XZgqKXdpOwAj0ZET8vtpeWd+xFRmwVYCexvae8EdlY9rgnK\n2g0MtrTPAItyfRFwJtffBTbeaL86LMCXwGNNzA/MBr4FHqF4GGt69o99HwD7gZW5Pj33U9VjbyPz\n4vyhtxbYB6gp2TPHeWD+dX2lnfu1ukIA7gZ+bmlfyL4mWBgRl3L9F2Bhrtf2mOQUwArgCA3Kn1Mm\nx4ER4ABwDrgSEddyl9aMY/lz+1Wga3JHXKrXgReBf7PdRXOyAwTwlaQBSduyr7Rzf3qZI7WpISJC\nUq1vH5N0G/AZ8HxE/ClpbFvd80fEP0CPpLnAF8B9FQ9pUkh6EhiJiAFJa6oeT0VWR8RFSXcBByR9\n37qx3XO/blcIF4ElLe3F2dcEv0paBJCvI9lfu2Mi6RaKYvBhRHye3Y3JPyoirgCHKaZJ5koa/QWv\nNeNY/tx+B/DHJA+1LKuApySdBz6imDZ6g2ZkByAiLubrCMUvAw9T4rlft4LwDbAs7zqYATwN7K14\nTJNlL7A51zdTzK2P9m/KOw76gKstl5cdR8WlwHvAUES81rKpKfkX5JUBkm6l+PxkiKIwbMjdrs8/\nelw2AIciJ5Q7TUTsjIjFEdFN8b19KCKepQHZASTNkXT76DrwODBImed+1R+STMCHLuuBHyjmVV+q\nejwTlHEPcAn4m2JecCvF3OhB4CzwNTAv9xXFnVfngFNAb9XjbzP7aop51JPA8VzWNyj//cCxzD8I\nvJz9S4GjwDDwCTAz+2dlezi3L606Q0nHYQ2wr0nZM+eJXE6P/nwr89z3k8pmZgbUb8rIzMzGyQXB\nzMwAFwQzM0suCGZmBrggmJlZckEwMzPABcHMzJILgpmZAfAfMscAd9j6No8AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2EgSuVqHXxtm",
        "colab_type": "text"
      },
      "source": [
        "## 6. 베이스라인 성능 측정\n",
        "\n",
        "이후에 우리가 학습시킨 모델의 성능의 좋고 나쁨을 판단할 기준(베이스라인)이 필요합니다. 적당한 수준의 기준 성능을 정해놓고 테스트 결과에서 우리의 모델이 그것보다 더 좋은 성능을 보이면 만족할만한 모델을 학습시킨 것으로 간주하면 되고, 반대로 그보다 성능이 좋지 않다면 모델을 더 개선시키는 방식으로 네트워크 구조를 변경해 나아가면 됩니다.\n",
        "\n",
        "과거의 기후 정보를 활용하는 우리의 딥러닝 모델이 과연 의미있는 성능이라는 걸 보이기 위해서는 어떠한 예측 방식이 기준이 될 수 있을까요? 아마 기온 예측 전문가가 아닌 대부분의 여러분이 지금으로부터 24시간 후의 기온을 예측하라는 질문을 받았다고 생각해 봅시다. 아마 가장 무난하면서도 안정적으로 예측하는 방식은 지금의 기온과 같다고 답하는 것일 겁니다. 이러한 예측 방식은 과거의 기후 정보를 복잡하게 고려할 필요도 없이 간단하지만, 많은 경우에 실제로 꽤나 정확한 예측을 할 수 있는 방식입니다. 내일 낮 12시의 기온은 특이한 경우가 아니라면, 오늘 낮 12시의 기온과 거의 비슷할 것입니다. \n",
        "\n",
        "아래의 **eval_baseline**은 이런 예측 방식의 성능, 즉 평균적인 loss를 측정하는 메소드입니다. 다음을 읽고 코드를 완성해보세요.\n",
        "- 우리의 베이스라인은 현재 기온을 24시간 후의 기온으로 예측하는 방식입니다. 현재 기후 정보는 **data_loader**에서 받은 **sequence**의 가장 마지막 타임스텝에 담겨있습니다. 입력으로 주는 **sequence**는 기온 뿐만아니라 총 9가지 속성이 포함된 것임에 유의하세요. 기온은 9가지 속성중 가장 첫번째에 위치하고 있음을 염두에 두고 **sequence**로 부터 현재 기온을 가져와 **pred** 변수에 저장하세요.\n",
        "- **pred**의 shape은 (batch_size, 1)이 되어야 합니다. 만약 여러분이 구한 **pred**의 shape이 (batch_size)라면, [torch.unsqueeze](https://pytorch.org/docs/stable/torch.html#torch.unsqueeze)를 활용하여 크기 1인 차원을 텐서에 삽입해보세요. \n",
        "- **criterion**이 우리가 선언한 loss function일 때, loss를 계산하여 **loss** 변수에 저장하세요. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HepQBWPEXxtp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def eval_baseline(data_loader, criterion):\n",
        "    total_loss = 0\n",
        "    cnt = 0\n",
        "    for step, (sequence, target) in enumerate(data_loader):\n",
        "        # 코드 시작\n",
        "        loss = criterion(pred, target)\n",
        "        # 코드 종료\n",
        "        total_loss += loss\n",
        "        cnt += 1\n",
        "    avrg_loss = total_loss / cnt\n",
        "    print('Baseline Average Loss: {:.4f}'.format(avrg_loss))\n",
        "    return avrg_loss.item()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KDYqMghdXxtw",
        "colab_type": "text"
      },
      "source": [
        "베이스라인의 성능을 측정합니다. 다음과 같은 결과가 출력된다면 성공적으로 구현한 것입니다.\n",
        "\n",
        "Baseline Average Loss: 0.0945"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9Wh8j0pXXxty",
        "colab_type": "code",
        "outputId": "269ad369-84ac-4a9f-b1fb-0919da42b785",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        }
      },
      "source": [
        "baseline_loss = eval_baseline(test_loader, nn.MSELoss())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Baseline Average Loss: 0.0945\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oHzx6qLTXxt-",
        "colab_type": "text"
      },
      "source": [
        "## 7. 네트워크 설계\n",
        "\n",
        "우리는 LSTM 구조를 통해 기온 예측을 모델을 학습시킬 것입니다. 설계할 네트워크의 대략적인 개요는 아래 그림과 같습니다.\n",
        "\n",
        "<img src=\"./img/lstm.png\" width=\"80%\" height=\"60%\">\n",
        "\n",
        "LSTM의 매 타임스텝의 입력은 매 시간마다 기록된 9가지 속성값이 들어가게 됩니다. 그리고 마지막 타입스텝의 출력을 마무리로 Fully Connected 레이어에 넣어 최종 예측 기온값을 출력하는 구조입니다.  \n",
        "\n",
        "다음을 읽고 코드를 완성해보세요.\n",
        "- 생성자의 파라미터 **hidden_size**는 우리가 설계할 LSTM 레이어의 hidden state의 크기입니다. 이 크기를 얼마를 할지는 역시 설계자의 몫입니다. 이번 예제에서 기본값은 100으로 하겠습니다. \n",
        "- **num_layers**는 LSTM 레이어의 총 레이어 수입니다. 기본값은 1로 하겠습니다.\n",
        "- [nn.LSTM](https://pytorch.org/docs/stable/nn.html#torch.nn.LSTM)을 활용하여 생성자에 LSTM 레이어를 선언하고 이를 **self.lstm** 변수에 저장하세요. LSTM 레이어 선언에 필요한 파라미터인 **input_size**에 적절한 값을 넣어보세요. 또 다른 파라미터인 **hidden_size**, **num_layers**에는 각각 **self.hidden_size**, **self.num_layers**를 넣어주세요. 그리고 **batch_first=True**로 하시기 바랍니다. **batch_first=True**이면 LSTM 레이어의 입력의 shape이 (**batch_size**, 전체 시퀀스 길이, **input_size**)가 됩니다. 기본값은 **batch_first=False**인데, 이 경우 (전체 시퀀스 길이, **batch_size**, **input_size**)가 됩니다. \n",
        "- LSTM의 마지막 타임스텝의 출력을 입력으로 받아 최종 기온을 예측하는 FC 레이어를 선언하고 이를 **self.fc** 변수에 저장하세요. \n",
        "- **forward** 함수를 구현할 차례입니다. **forward**의 파라미터 **x**, **h**, **c**는 각각 LSTM의 입력, LSTM의 초기 hidden state, 초기 cell state를 의미합니다. Pytorch의 LSTM 레이어는 **x, (h, c)**를 입력으로 주면 **LSTM의 모든 타입스텝의 출력, (마지막 타임스텝의 hidden state, 마지막 타임스텝의 cell state)**를 반환합니다. (hidden state와 cell state을 입출력으로 주고 받을 때에는 둘을 괄호로 묶어 tuple 형태여야 함에 유의하시기 바랍니다.)\n",
        "- LSTM의 출력의 마지막 타임스텝의 출력을 FC 레이어에 입력으로 주어 얻은 결과를 **final_output** 변수에 저장하세요.\n",
        "- 마지막으로 **init_hidden** 함수를 구현할 차례입니다. **init_hidden**은 초기 hidden state, cell state를 만들어 반환하는 함수입니다. 초기 hidden state, cell state는 일반적으로 0으로 채운 값을 사용합니다. 0으로 채운 텐서를 생성하기 위해서는 [torch.zeros](https://pytorch.org/docs/stable/torch.html?highlight=zeros#torch.zeros)를 활용하시기 바랍니다. hidden state와 cell state의 shape은 (**num_layers**, **batch_size**, **hidden_size**)가 되어야 합니다. 0으로 채운 hidden state와 cell state를 선언하여 각각 **hidden**, **cell** 변수에 저장하세요. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kRZORrcbXxuC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class SimpleLSTM(nn.Module):\n",
        "    def __init__(self, hidden_size=100, num_layers=1):\n",
        "        super().__init__()\n",
        "        self.hidden_size = hidden_size\n",
        "        self.num_layers = num_layers\n",
        "        # 코드 시작\n",
        "        self.lstm = nn.LSTM(input_size=9, hidden_size=hidden_size, num_layers=num_layers, batch_first=True).to(device)\n",
        "        self.fc = nn.Linear(in_features=hidden_size, out_features=1).to(device)\n",
        "        # 코드 종료\n",
        "    \n",
        "    def forward(self, x, h, c):\n",
        "        # 코드 시작\n",
        "        out, (h, c) = self.lstm(x, (h, c))\n",
        "        last_out = out[:, -1, :]\n",
        "        final_output = self.fc(last_out)\n",
        "        # 코드 종료\n",
        "        return final_output\n",
        "    \n",
        "    def init_hidden(self):\n",
        "        # 코드 시작\n",
        "        hidden = torch.zeros(self.num_layers, batch_size, self.hidden_size).to(device)\n",
        "        cell = torch.zeros(self.num_layers, batch_size, self.hidden_size).to(device)\n",
        "        # 코드 종료\n",
        "        return hidden, cell"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L1hQ83ZRXxuW",
        "colab_type": "text"
      },
      "source": [
        "아래의 코드를 실행해 코드를 성공적으로 완성했는지 확인해보세요. \n",
        "\n",
        "별다른 문제가 없다면 이어서 진행하면 됩니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3soyr0gLXxuY",
        "colab_type": "code",
        "outputId": "185081bd-9fb7-46fc-fd4a-bd3bf90b0676",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        }
      },
      "source": [
        "checker.model_check(SimpleLSTM(), batch_size)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "체크 함수를 실행하는 도중에 문제가 발생했습니다. 코드 구현을 완료했는지 다시 검토하시기 바랍니다.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3L-gd52yXxus",
        "colab_type": "text"
      },
      "source": [
        "## 8. train, validation, test 함수 정의\n",
        "\n",
        "이번에는 훈련, 검증, 테스트를 진행하는 함수를 정의하겠습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gi0b2jcsXxuu",
        "colab_type": "text"
      },
      "source": [
        "먼저 훈련 함수입니다. 다음을 읽고 코드를 완성해 보세요.\n",
        "- 먼저 **sequence**, **target**의 데이터 형식을 **torch.float32**로 변환해주었습니다. 이렇게 해주는 이유는 현재 구현한 DataLoader에서는 **torch.float64** 형식으로 데이터를 반환하는데, 모델의 파라미터는 기본적으로 **torch.float32**로 이루어져 있기 때문입니다. 모델의 파라미터와 입력 텐서가 같은 데이터 형식을 갖도록 해주어야 합니다. Torch.Tensor의 다양한 데이터 형식에 대해 알아보려면 [여기](https://pytorch.org/docs/stable/tensor_attributes.html#torch.torch.dtype)를 참고하시기 바랍니다. \n",
        "- 위의 모델에서 구현했던 **init_hidden** 함수를 이용해 초기 hidden state와 cell state를 받아 각각 **h0**, **c0** 변수에 저장합니다.\n",
        "- **train** 함수에 여러 인자들이 보입니다. **model**이 우리가 선언한 모델일 때, 모델에 입력 시퀀스와, 초기 hidden state, cell state를 주고 얻은 결과를 **outputs**에 저장합니다.\n",
        "- **criterion**이 우리가 선언한 loss function일 때, **outputs**와 **target**을 통해 loss를 계산하고 그 결과를 **loss**에 저장합니다.\n",
        "- **optim**이 우리가 선언한 optimizer일 때, 이전에 계산한 기울기를 모두 clear하고, backpropagation을 통해 기울기를 계산하고, optimizer를 통해 파라미터를 업데이트합니다. \n",
        "\n",
        "**tarin**에서는 일정한 에폭마다 다음에 구현할 **validation**함수를 통해 검증을 수행합니다. 모델 검증을 수행했을 때, 만약 검증 과정의 평균 loss가 현재까지 가장 낮다면 가장 잘 훈련된 모델로 가정하고 그때까지 학습한 모델을 저장합니다. 저장은 추후에 구현할 **save_model** 함수가 수행합니다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "RX5TsbfZXxuw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def train(num_epochs, model, data_loader, criterion, optim, saved_dir, val_every):\n",
        "    print('Start training..')\n",
        "    best_loss = 9999999\n",
        "    for epoch in range(num_epochs):\n",
        "        for step, (sequence, target) in enumerate(data_loader):\n",
        "            sequence = sequence.to(device, torch.float32)\n",
        "            target = target.to(device, torch.float32)\n",
        "            # 코드 시작\n",
        "            h0, c0 = model.init_hidden()\n",
        "            h0.to(device)\n",
        "            c0.to(device)\n",
        "            outputs = model(sequence, h0, c0).to(device)\n",
        "            loss = criterion(outputs, target).to(device)\n",
        "            \n",
        "            optim.zero_grad()\n",
        "            loss.backward()\n",
        "            optimizer.step()\n",
        "            # 코드 종료\n",
        "            \n",
        "            if (step + 1) % 1 == 0:\n",
        "                print('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'.format(\n",
        "                    epoch+1, num_epochs, step+1, len(train_loader), loss.item()))\n",
        "                \n",
        "        if (epoch + 1) % val_every == 0:\n",
        "            avrg_loss = validation(epoch + 1, model, val_loader, criterion)\n",
        "            if avrg_loss < best_loss:\n",
        "                print('Best performance at epoch: {}'.format(epoch + 1))\n",
        "                print('Save model in', saved_dir)\n",
        "                best_loss = avrg_loss\n",
        "                save_model(model, saved_dir)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xrez9axsXxu6",
        "colab_type": "text"
      },
      "source": [
        "검증 함수입니다. 다음을 읽고 코드를 완성해보세요.\n",
        "- 검증 과정에서는 파라미터 업데이트를 하지 않기 때문에 기울기를 계산할 필요는 없습니다. 하지만 검증 과정에서의 평균 loss를 계산하기 위해 loss는 계산해야 합니다. \n",
        "- 먼저 **train** 함수와 마찬가지로 초기 hidden state, cell state를 선언해 **h0**, **c0** 변수에 저장합니다. 그리고 **model**에 입력 시퀀스와 초기 hidden state, cell state를 주어 얻은 결과를 **outputs**에 저장하고, **criterion**을 통해 loss를 계산한 뒤, 그 결과를 **loss**에 저장합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iSB0k-i3Xxu8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def validation(epoch, model, data_loader, criterion):\n",
        "    print('Start validation #{}'.format(epoch))\n",
        "    model.eval()\n",
        "    with torch.no_grad():\n",
        "        total_loss = 0\n",
        "        cnt = 0\n",
        "        for step, (sequence, target) in enumerate(data_loader):\n",
        "            sequence = sequence.to(device, torch.float32)\n",
        "            target = target.to(device, torch.float32)\n",
        "            # 코드 시작\n",
        "            h0, c0 = model.init_hidden()\n",
        "            h0.to(device)\n",
        "            c0.to(device)\n",
        "            outputs = model(sequence, h0, c0).to(device)\n",
        "            loss = criterion(outputs, target).to(device)\n",
        "            # 코드 종료\n",
        "            total_loss += loss\n",
        "            cnt += 1\n",
        "        avrg_loss = total_loss / cnt\n",
        "        print('Validation #{}  Average Loss: {:.4f}'.format(epoch, avrg_loss))\n",
        "    model.train()\n",
        "    return avrg_loss"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "njmh8yyrXxvH",
        "colab_type": "text"
      },
      "source": [
        "테스트 함수입니다. 다음을 읽고 코드를 완성해보세요.\n",
        "\n",
        "- **validation** 함수와 마찬가지로 초기 hidden state, cell state를 선언해 **h0**, **c0** 변수에 저장합니다. 그리고 **model**에 입력 시퀀스와 초기 hidden state, cell state를 주어 얻은 결과를 **outputs**에 저장하고, **criterion**을 통해 loss를 계산한 뒤, 그 결과를 **loss**에 저장합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-aEh-F35XxvP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def test(model, data_loader, criterion, baseline_loss):\n",
        "    print('Start test..')\n",
        "    model.eval()\n",
        "    with torch.no_grad():\n",
        "        total_loss = 0\n",
        "        cnt = 0\n",
        "        for step, (sequence, target) in enumerate(data_loader):\n",
        "            sequence = sequence.to(device, torch.float32)\n",
        "            target = target.to(device, torch.float32)\n",
        "            # 코드 시작\n",
        "            h0, c0 = model.init_hidden()\n",
        "            h0.to(device)\n",
        "            c0.to(device)\n",
        "            outputs = model(sequence, h0, c0).to(device)\n",
        "            loss = criterion(outputs, target).to(device)\n",
        "            # 코드 종료\n",
        "            total_loss += loss\n",
        "            cnt += 1\n",
        "        avrg_loss = total_loss / cnt\n",
        "        print('Test  Average Loss: {:.4f}  Baseline Loss: {:.4f}'.format(avrg_loss, baseline_loss))\n",
        "        \n",
        "    if avrg_loss < baseline_loss:\n",
        "        print('베이스라인 성능을 뛰어 넘었습니다!')\n",
        "    else:\n",
        "        print('아쉽지만 베이스라인 성능을 넘지 못했습니다.')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AnzbiorQXxve",
        "colab_type": "text"
      },
      "source": [
        "## 9. 모델 저장 함수 정의\n",
        "\n",
        "모델을 저장하는 함수입니다. 모델 저장은 [torch.save](https://pytorch.org/docs/stable/torch.html?highlight=save#torch.save) 함수를 통해 할 수 있습니다. \n",
        "[nn.Module.state_dict](https://pytorch.org/docs/stable/nn.html?highlight=state_dict#torch.nn.Module.state_dict)를 통해 Module, 즉 우리 모델의 파라미터를 가져올 수 있습니다. 이렇게 불러온 파라미터를 **check_point** 딕셔너리에 저장합니다. 그리고 이 **check_point**를 정해준 경로에 저장하면 됩니다. \n",
        "\n",
        "torch.save는 단순히 모델의 파라미터만 저장하는 함수가 아닙니다. 어떤 파이썬 객체든 저장할 수 있습니다. 그래서 경우에 따라 **check_point** 딕셔너리에 모델의 파라미터 뿐만 아니라 다른 여러 가지 필요한 정보를 저장할 수도 있습니다. 예를 들어 총 몇 에폭동안 학습한 모델인지 그 정보도 저장할 수 있겠죠? \n",
        "\n",
        "다음을 읽고 코드를 완성해보세요.\n",
        "- torch.save를 통해 **output_path** 경로에 **check_point**를 저장하세요."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vo27H1YEXxvj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def save_model(model, saved_dir, file_name='best_model.pt'):\n",
        "    os.makedirs(saved_dir, exist_ok=True)\n",
        "    check_point = {\n",
        "        'net': model.state_dict()\n",
        "    }\n",
        "    output_path = os.path.join(saved_dir, file_name)\n",
        "    # 코드 시작\n",
        "    torch.save(check_point, output_path)\n",
        "    # 코드 종료"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7ouCIv45Xxvq",
        "colab_type": "text"
      },
      "source": [
        "## 10. 모델 생성 및 Loss function, Optimizer 정의\n",
        "\n",
        "이제 학습할 모델을 생성하고 loss function과 optimizer를 정의할 차례입니다. 다음을 읽고 코드를 완성해보세요.\n",
        "- 위에서 정의한 SimpleLSTM class를 통해 모델을 생성하고 이를 **model** 변수에 저장합니다.\n",
        "- 연속된 값을 예측하는 회기(regression) 문제에서는 보통 Mean Squared Error(MSE) loss function을 사용합니다. [nn.MSELoss](https://pytorch.org/docs/stable/nn.html?highlight=mseloss#torch.nn.MSELoss)를 통해 MSE loss function을 선언하고 이를 **criterion** 변수에 저장합니다.\n",
        "- Adam optimizer를 통해 파라미터를 업데이트 하겠습니다. Adam optimizer([torch.optim.Adam](https://pytorch.org/docs/stable/optim.html?highlight=adam#torch.optim.Adam))를 **optimizer** 변수에 저장합니다.\n",
        "\n",
        "**val_every**는 검증을 몇 에폭마다 진행할지 정하는 변수입니다. **saved_dir**은 모델이 저장될 디렉토리의 경로입니다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E9LSaps0Xxvt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 코드 시작\n",
        "model = SimpleLSTM().to(device)\n",
        "criterion = nn.MSELoss().to(device)\n",
        "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)\n",
        "# 코드 종료\n",
        "val_every = 1\n",
        "saved_dir = './saved/LSTM'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h_Q9CpZ9Xxv0",
        "colab_type": "text"
      },
      "source": [
        "아래의 코드를 실행해 코드를 성공적으로 완성했는지 확인해보세요. \n",
        "\n",
        "별다른 문제가 없다면 이어서 진행하면 됩니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ufO0U4xGXxv2",
        "colab_type": "code",
        "outputId": "5d4adba9-de44-4533-886a-4b44eeedb75f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "checker.loss_func_check(criterion)\n",
        "checker.optim_check(optimizer)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "MSE loss function을 잘 정의하셨습니다! 이어서 진행하셔도 좋습니다.\n",
            "Adam optimizer를 잘 정의하셨습니다! 이어서 진행하셔도 좋습니다.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ROUC5fj_XxwG",
        "colab_type": "text"
      },
      "source": [
        "## 11. Training\n",
        "\n",
        "**train** 함수를 통해 모델 학습을 진행합니다. LSTM과 같은 recurrent layer는 타임스텝을 따라 계산이 순차적으로 진행을 해야 되기 때문에 상대적으로 학습 시간이 많이 소요됩니다. 따라서 시간이 여유가 없는 분들은 모델 학습이 적당히 진행된다는 정도만 확인하고 다음 단계로 넘어가셔도 됩니다.\n",
        "\n",
        "만약 한 에폭 이상이 지나도록 loss가 전혀 감소하는 기미가 보이지 않는다면 이전의 구현에 문제가 있을 가능성이 높으니 코드를 다시 검토해주시기 바랍니다.\n",
        "\n",
        "또한, 모델 저장 코드를 제대로 구현했다면 첫 에폭 학습후에 ./saved/LSTM 경로에 best_model.pt 파일이 저장되어 있어야 합니다. 만약에 파일이 존재하지 않는다면 모델 저장 코드를 다시 확인하시기 바랍니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "SmKgOv4JXxwI",
        "colab_type": "code",
        "outputId": "9754270d-f2ee-4fd4-a33e-82b26a186f67",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 80079
        }
      },
      "source": [
        "train(num_epochs, model, train_loader, criterion, optimizer, saved_dir, val_every)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Start training..\n",
            "Epoch [1/30], Step [1/139], Loss: 1.0675\n",
            "Epoch [1/30], Step [2/139], Loss: 1.0593\n",
            "Epoch [1/30], Step [3/139], Loss: 0.8684\n",
            "Epoch [1/30], Step [4/139], Loss: 1.0954\n",
            "Epoch [1/30], Step [5/139], Loss: 1.1162\n",
            "Epoch [1/30], Step [6/139], Loss: 1.0491\n",
            "Epoch [1/30], Step [7/139], Loss: 1.2158\n",
            "Epoch [1/30], Step [8/139], Loss: 0.9393\n",
            "Epoch [1/30], Step [9/139], Loss: 0.8996\n",
            "Epoch [1/30], Step [10/139], Loss: 0.9082\n",
            "Epoch [1/30], Step [11/139], Loss: 0.8983\n",
            "Epoch [1/30], Step [12/139], Loss: 1.0149\n",
            "Epoch [1/30], Step [13/139], Loss: 1.0637\n",
            "Epoch [1/30], Step [14/139], Loss: 1.0560\n",
            "Epoch [1/30], Step [15/139], Loss: 0.9612\n",
            "Epoch [1/30], Step [16/139], Loss: 0.9994\n",
            "Epoch [1/30], Step [17/139], Loss: 0.9839\n",
            "Epoch [1/30], Step [18/139], Loss: 0.8016\n",
            "Epoch [1/30], Step [19/139], Loss: 1.0151\n",
            "Epoch [1/30], Step [20/139], Loss: 0.9264\n",
            "Epoch [1/30], Step [21/139], Loss: 0.9161\n",
            "Epoch [1/30], Step [22/139], Loss: 1.0179\n",
            "Epoch [1/30], Step [23/139], Loss: 1.1032\n",
            "Epoch [1/30], Step [24/139], Loss: 1.1390\n",
            "Epoch [1/30], Step [25/139], Loss: 0.9404\n",
            "Epoch [1/30], Step [26/139], Loss: 1.0309\n",
            "Epoch [1/30], Step [27/139], Loss: 0.9318\n",
            "Epoch [1/30], Step [28/139], Loss: 1.0304\n",
            "Epoch [1/30], Step [29/139], Loss: 0.8615\n",
            "Epoch [1/30], Step [30/139], Loss: 0.8735\n",
            "Epoch [1/30], Step [31/139], Loss: 0.8742\n",
            "Epoch [1/30], Step [32/139], Loss: 0.8829\n",
            "Epoch [1/30], Step [33/139], Loss: 0.9609\n",
            "Epoch [1/30], Step [34/139], Loss: 1.0867\n",
            "Epoch [1/30], Step [35/139], Loss: 1.0105\n",
            "Epoch [1/30], Step [36/139], Loss: 0.9160\n",
            "Epoch [1/30], Step [37/139], Loss: 0.8706\n",
            "Epoch [1/30], Step [38/139], Loss: 1.0215\n",
            "Epoch [1/30], Step [39/139], Loss: 0.9260\n",
            "Epoch [1/30], Step [40/139], Loss: 0.9594\n",
            "Epoch [1/30], Step [41/139], Loss: 0.9579\n",
            "Epoch [1/30], Step [42/139], Loss: 0.9554\n",
            "Epoch [1/30], Step [43/139], Loss: 1.0495\n",
            "Epoch [1/30], Step [44/139], Loss: 0.8392\n",
            "Epoch [1/30], Step [45/139], Loss: 0.9848\n",
            "Epoch [1/30], Step [46/139], Loss: 1.1816\n",
            "Epoch [1/30], Step [47/139], Loss: 1.1605\n",
            "Epoch [1/30], Step [48/139], Loss: 0.9110\n",
            "Epoch [1/30], Step [49/139], Loss: 0.8782\n",
            "Epoch [1/30], Step [50/139], Loss: 1.0487\n",
            "Epoch [1/30], Step [51/139], Loss: 0.9334\n",
            "Epoch [1/30], Step [52/139], Loss: 0.8271\n",
            "Epoch [1/30], Step [53/139], Loss: 1.0133\n",
            "Epoch [1/30], Step [54/139], Loss: 1.0698\n",
            "Epoch [1/30], Step [55/139], Loss: 0.8757\n",
            "Epoch [1/30], Step [56/139], Loss: 0.9028\n",
            "Epoch [1/30], Step [57/139], Loss: 0.9453\n",
            "Epoch [1/30], Step [58/139], Loss: 0.9920\n",
            "Epoch [1/30], Step [59/139], Loss: 0.8715\n",
            "Epoch [1/30], Step [60/139], Loss: 1.0558\n",
            "Epoch [1/30], Step [61/139], Loss: 0.8467\n",
            "Epoch [1/30], Step [62/139], Loss: 0.9431\n",
            "Epoch [1/30], Step [63/139], Loss: 0.7817\n",
            "Epoch [1/30], Step [64/139], Loss: 1.0331\n",
            "Epoch [1/30], Step [65/139], Loss: 0.7676\n",
            "Epoch [1/30], Step [66/139], Loss: 1.0086\n",
            "Epoch [1/30], Step [67/139], Loss: 0.8795\n",
            "Epoch [1/30], Step [68/139], Loss: 1.1845\n",
            "Epoch [1/30], Step [69/139], Loss: 0.9728\n",
            "Epoch [1/30], Step [70/139], Loss: 0.9732\n",
            "Epoch [1/30], Step [71/139], Loss: 1.0121\n",
            "Epoch [1/30], Step [72/139], Loss: 0.8147\n",
            "Epoch [1/30], Step [73/139], Loss: 0.8655\n",
            "Epoch [1/30], Step [74/139], Loss: 1.0440\n",
            "Epoch [1/30], Step [75/139], Loss: 0.8626\n",
            "Epoch [1/30], Step [76/139], Loss: 0.8815\n",
            "Epoch [1/30], Step [77/139], Loss: 0.9276\n",
            "Epoch [1/30], Step [78/139], Loss: 0.8992\n",
            "Epoch [1/30], Step [79/139], Loss: 0.9698\n",
            "Epoch [1/30], Step [80/139], Loss: 0.9360\n",
            "Epoch [1/30], Step [81/139], Loss: 0.7472\n",
            "Epoch [1/30], Step [82/139], Loss: 0.8598\n",
            "Epoch [1/30], Step [83/139], Loss: 0.8359\n",
            "Epoch [1/30], Step [84/139], Loss: 0.9037\n",
            "Epoch [1/30], Step [85/139], Loss: 0.9820\n",
            "Epoch [1/30], Step [86/139], Loss: 0.7861\n",
            "Epoch [1/30], Step [87/139], Loss: 0.8314\n",
            "Epoch [1/30], Step [88/139], Loss: 1.0491\n",
            "Epoch [1/30], Step [89/139], Loss: 0.8807\n",
            "Epoch [1/30], Step [90/139], Loss: 0.7767\n",
            "Epoch [1/30], Step [91/139], Loss: 0.8824\n",
            "Epoch [1/30], Step [92/139], Loss: 1.0335\n",
            "Epoch [1/30], Step [93/139], Loss: 0.7502\n",
            "Epoch [1/30], Step [94/139], Loss: 0.8404\n",
            "Epoch [1/30], Step [95/139], Loss: 0.7542\n",
            "Epoch [1/30], Step [96/139], Loss: 0.8885\n",
            "Epoch [1/30], Step [97/139], Loss: 0.8956\n",
            "Epoch [1/30], Step [98/139], Loss: 0.8989\n",
            "Epoch [1/30], Step [99/139], Loss: 0.8543\n",
            "Epoch [1/30], Step [100/139], Loss: 0.7471\n",
            "Epoch [1/30], Step [101/139], Loss: 0.8321\n",
            "Epoch [1/30], Step [102/139], Loss: 0.7748\n",
            "Epoch [1/30], Step [103/139], Loss: 0.8124\n",
            "Epoch [1/30], Step [104/139], Loss: 0.8081\n",
            "Epoch [1/30], Step [105/139], Loss: 0.8842\n",
            "Epoch [1/30], Step [106/139], Loss: 0.8554\n",
            "Epoch [1/30], Step [107/139], Loss: 0.8635\n",
            "Epoch [1/30], Step [108/139], Loss: 0.9143\n",
            "Epoch [1/30], Step [109/139], Loss: 0.7219\n",
            "Epoch [1/30], Step [110/139], Loss: 0.7591\n",
            "Epoch [1/30], Step [111/139], Loss: 0.8303\n",
            "Epoch [1/30], Step [112/139], Loss: 0.8664\n",
            "Epoch [1/30], Step [113/139], Loss: 0.9955\n",
            "Epoch [1/30], Step [114/139], Loss: 0.7593\n",
            "Epoch [1/30], Step [115/139], Loss: 0.7860\n",
            "Epoch [1/30], Step [116/139], Loss: 0.9295\n",
            "Epoch [1/30], Step [117/139], Loss: 0.8284\n",
            "Epoch [1/30], Step [118/139], Loss: 0.8916\n",
            "Epoch [1/30], Step [119/139], Loss: 0.8422\n",
            "Epoch [1/30], Step [120/139], Loss: 0.9326\n",
            "Epoch [1/30], Step [121/139], Loss: 0.8795\n",
            "Epoch [1/30], Step [122/139], Loss: 0.8210\n",
            "Epoch [1/30], Step [123/139], Loss: 0.7537\n",
            "Epoch [1/30], Step [124/139], Loss: 1.0025\n",
            "Epoch [1/30], Step [125/139], Loss: 0.7649\n",
            "Epoch [1/30], Step [126/139], Loss: 0.6577\n",
            "Epoch [1/30], Step [127/139], Loss: 0.8739\n",
            "Epoch [1/30], Step [128/139], Loss: 0.7769\n",
            "Epoch [1/30], Step [129/139], Loss: 0.7484\n",
            "Epoch [1/30], Step [130/139], Loss: 0.8323\n",
            "Epoch [1/30], Step [131/139], Loss: 0.8714\n",
            "Epoch [1/30], Step [132/139], Loss: 0.7984\n",
            "Epoch [1/30], Step [133/139], Loss: 0.9292\n",
            "Epoch [1/30], Step [134/139], Loss: 0.7595\n",
            "Epoch [1/30], Step [135/139], Loss: 0.8237\n",
            "Epoch [1/30], Step [136/139], Loss: 0.7469\n",
            "Epoch [1/30], Step [137/139], Loss: 0.7176\n",
            "Epoch [1/30], Step [138/139], Loss: 0.7574\n",
            "Epoch [1/30], Step [139/139], Loss: 0.8310\n",
            "Start validation #1\n",
            "Validation #1  Average Loss: 0.6992\n",
            "Best performance at epoch: 1\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [2/30], Step [1/139], Loss: 0.8335\n",
            "Epoch [2/30], Step [2/139], Loss: 0.5969\n",
            "Epoch [2/30], Step [3/139], Loss: 0.7911\n",
            "Epoch [2/30], Step [4/139], Loss: 0.8536\n",
            "Epoch [2/30], Step [5/139], Loss: 0.7994\n",
            "Epoch [2/30], Step [6/139], Loss: 0.8046\n",
            "Epoch [2/30], Step [7/139], Loss: 0.9191\n",
            "Epoch [2/30], Step [8/139], Loss: 0.7948\n",
            "Epoch [2/30], Step [9/139], Loss: 0.7575\n",
            "Epoch [2/30], Step [10/139], Loss: 0.7641\n",
            "Epoch [2/30], Step [11/139], Loss: 0.6736\n",
            "Epoch [2/30], Step [12/139], Loss: 0.6559\n",
            "Epoch [2/30], Step [13/139], Loss: 0.8776\n",
            "Epoch [2/30], Step [14/139], Loss: 0.7979\n",
            "Epoch [2/30], Step [15/139], Loss: 0.7687\n",
            "Epoch [2/30], Step [16/139], Loss: 0.7801\n",
            "Epoch [2/30], Step [17/139], Loss: 0.7867\n",
            "Epoch [2/30], Step [18/139], Loss: 0.7866\n",
            "Epoch [2/30], Step [19/139], Loss: 0.8808\n",
            "Epoch [2/30], Step [20/139], Loss: 0.5816\n",
            "Epoch [2/30], Step [21/139], Loss: 0.7514\n",
            "Epoch [2/30], Step [22/139], Loss: 0.7672\n",
            "Epoch [2/30], Step [23/139], Loss: 0.6817\n",
            "Epoch [2/30], Step [24/139], Loss: 0.7291\n",
            "Epoch [2/30], Step [25/139], Loss: 0.6205\n",
            "Epoch [2/30], Step [26/139], Loss: 0.7371\n",
            "Epoch [2/30], Step [27/139], Loss: 0.6468\n",
            "Epoch [2/30], Step [28/139], Loss: 0.8037\n",
            "Epoch [2/30], Step [29/139], Loss: 0.7396\n",
            "Epoch [2/30], Step [30/139], Loss: 0.6774\n",
            "Epoch [2/30], Step [31/139], Loss: 0.7698\n",
            "Epoch [2/30], Step [32/139], Loss: 0.9090\n",
            "Epoch [2/30], Step [33/139], Loss: 0.5895\n",
            "Epoch [2/30], Step [34/139], Loss: 0.7742\n",
            "Epoch [2/30], Step [35/139], Loss: 0.7262\n",
            "Epoch [2/30], Step [36/139], Loss: 0.7614\n",
            "Epoch [2/30], Step [37/139], Loss: 0.8049\n",
            "Epoch [2/30], Step [38/139], Loss: 0.7017\n",
            "Epoch [2/30], Step [39/139], Loss: 0.7619\n",
            "Epoch [2/30], Step [40/139], Loss: 0.7348\n",
            "Epoch [2/30], Step [41/139], Loss: 0.6329\n",
            "Epoch [2/30], Step [42/139], Loss: 0.7133\n",
            "Epoch [2/30], Step [43/139], Loss: 0.7250\n",
            "Epoch [2/30], Step [44/139], Loss: 0.8088\n",
            "Epoch [2/30], Step [45/139], Loss: 0.6958\n",
            "Epoch [2/30], Step [46/139], Loss: 0.5970\n",
            "Epoch [2/30], Step [47/139], Loss: 0.5779\n",
            "Epoch [2/30], Step [48/139], Loss: 0.7099\n",
            "Epoch [2/30], Step [49/139], Loss: 0.6420\n",
            "Epoch [2/30], Step [50/139], Loss: 0.5898\n",
            "Epoch [2/30], Step [51/139], Loss: 0.6925\n",
            "Epoch [2/30], Step [52/139], Loss: 0.6451\n",
            "Epoch [2/30], Step [53/139], Loss: 0.6384\n",
            "Epoch [2/30], Step [54/139], Loss: 0.5156\n",
            "Epoch [2/30], Step [55/139], Loss: 0.6871\n",
            "Epoch [2/30], Step [56/139], Loss: 0.8002\n",
            "Epoch [2/30], Step [57/139], Loss: 0.6903\n",
            "Epoch [2/30], Step [58/139], Loss: 0.5940\n",
            "Epoch [2/30], Step [59/139], Loss: 0.5465\n",
            "Epoch [2/30], Step [60/139], Loss: 0.6279\n",
            "Epoch [2/30], Step [61/139], Loss: 0.5305\n",
            "Epoch [2/30], Step [62/139], Loss: 0.7062\n",
            "Epoch [2/30], Step [63/139], Loss: 0.6522\n",
            "Epoch [2/30], Step [64/139], Loss: 0.6144\n",
            "Epoch [2/30], Step [65/139], Loss: 0.6490\n",
            "Epoch [2/30], Step [66/139], Loss: 0.7569\n",
            "Epoch [2/30], Step [67/139], Loss: 0.7014\n",
            "Epoch [2/30], Step [68/139], Loss: 0.7748\n",
            "Epoch [2/30], Step [69/139], Loss: 0.6365\n",
            "Epoch [2/30], Step [70/139], Loss: 0.5693\n",
            "Epoch [2/30], Step [71/139], Loss: 0.7040\n",
            "Epoch [2/30], Step [72/139], Loss: 0.6727\n",
            "Epoch [2/30], Step [73/139], Loss: 0.6051\n",
            "Epoch [2/30], Step [74/139], Loss: 0.5426\n",
            "Epoch [2/30], Step [75/139], Loss: 0.7044\n",
            "Epoch [2/30], Step [76/139], Loss: 0.6386\n",
            "Epoch [2/30], Step [77/139], Loss: 0.6767\n",
            "Epoch [2/30], Step [78/139], Loss: 0.7678\n",
            "Epoch [2/30], Step [79/139], Loss: 0.7271\n",
            "Epoch [2/30], Step [80/139], Loss: 0.6382\n",
            "Epoch [2/30], Step [81/139], Loss: 0.6428\n",
            "Epoch [2/30], Step [82/139], Loss: 0.6148\n",
            "Epoch [2/30], Step [83/139], Loss: 0.6663\n",
            "Epoch [2/30], Step [84/139], Loss: 0.6110\n",
            "Epoch [2/30], Step [85/139], Loss: 0.5905\n",
            "Epoch [2/30], Step [86/139], Loss: 0.6100\n",
            "Epoch [2/30], Step [87/139], Loss: 0.6083\n",
            "Epoch [2/30], Step [88/139], Loss: 0.6204\n",
            "Epoch [2/30], Step [89/139], Loss: 0.6219\n",
            "Epoch [2/30], Step [90/139], Loss: 0.4828\n",
            "Epoch [2/30], Step [91/139], Loss: 0.5753\n",
            "Epoch [2/30], Step [92/139], Loss: 0.5439\n",
            "Epoch [2/30], Step [93/139], Loss: 0.6296\n",
            "Epoch [2/30], Step [94/139], Loss: 0.5656\n",
            "Epoch [2/30], Step [95/139], Loss: 0.6687\n",
            "Epoch [2/30], Step [96/139], Loss: 0.5861\n",
            "Epoch [2/30], Step [97/139], Loss: 0.6728\n",
            "Epoch [2/30], Step [98/139], Loss: 0.6819\n",
            "Epoch [2/30], Step [99/139], Loss: 0.5917\n",
            "Epoch [2/30], Step [100/139], Loss: 0.5576\n",
            "Epoch [2/30], Step [101/139], Loss: 0.6230\n",
            "Epoch [2/30], Step [102/139], Loss: 0.4625\n",
            "Epoch [2/30], Step [103/139], Loss: 0.5717\n",
            "Epoch [2/30], Step [104/139], Loss: 0.6331\n",
            "Epoch [2/30], Step [105/139], Loss: 0.5921\n",
            "Epoch [2/30], Step [106/139], Loss: 0.5310\n",
            "Epoch [2/30], Step [107/139], Loss: 0.5536\n",
            "Epoch [2/30], Step [108/139], Loss: 0.5606\n",
            "Epoch [2/30], Step [109/139], Loss: 0.5215\n",
            "Epoch [2/30], Step [110/139], Loss: 0.4886\n",
            "Epoch [2/30], Step [111/139], Loss: 0.5293\n",
            "Epoch [2/30], Step [112/139], Loss: 0.5603\n",
            "Epoch [2/30], Step [113/139], Loss: 0.5897\n",
            "Epoch [2/30], Step [114/139], Loss: 0.5099\n",
            "Epoch [2/30], Step [115/139], Loss: 0.5747\n",
            "Epoch [2/30], Step [116/139], Loss: 0.5487\n",
            "Epoch [2/30], Step [117/139], Loss: 0.5314\n",
            "Epoch [2/30], Step [118/139], Loss: 0.6003\n",
            "Epoch [2/30], Step [119/139], Loss: 0.4146\n",
            "Epoch [2/30], Step [120/139], Loss: 0.4542\n",
            "Epoch [2/30], Step [121/139], Loss: 0.4410\n",
            "Epoch [2/30], Step [122/139], Loss: 0.6698\n",
            "Epoch [2/30], Step [123/139], Loss: 0.5609\n",
            "Epoch [2/30], Step [124/139], Loss: 0.6308\n",
            "Epoch [2/30], Step [125/139], Loss: 0.4897\n",
            "Epoch [2/30], Step [126/139], Loss: 0.5651\n",
            "Epoch [2/30], Step [127/139], Loss: 0.5626\n",
            "Epoch [2/30], Step [128/139], Loss: 0.5984\n",
            "Epoch [2/30], Step [129/139], Loss: 0.5477\n",
            "Epoch [2/30], Step [130/139], Loss: 0.3956\n",
            "Epoch [2/30], Step [131/139], Loss: 0.4892\n",
            "Epoch [2/30], Step [132/139], Loss: 0.5301\n",
            "Epoch [2/30], Step [133/139], Loss: 0.5101\n",
            "Epoch [2/30], Step [134/139], Loss: 0.5074\n",
            "Epoch [2/30], Step [135/139], Loss: 0.4313\n",
            "Epoch [2/30], Step [136/139], Loss: 0.5483\n",
            "Epoch [2/30], Step [137/139], Loss: 0.4434\n",
            "Epoch [2/30], Step [138/139], Loss: 0.4644\n",
            "Epoch [2/30], Step [139/139], Loss: 0.5420\n",
            "Start validation #2\n",
            "Validation #2  Average Loss: 0.4343\n",
            "Best performance at epoch: 2\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [3/30], Step [1/139], Loss: 0.4619\n",
            "Epoch [3/30], Step [2/139], Loss: 0.5168\n",
            "Epoch [3/30], Step [3/139], Loss: 0.4766\n",
            "Epoch [3/30], Step [4/139], Loss: 0.4206\n",
            "Epoch [3/30], Step [5/139], Loss: 0.5140\n",
            "Epoch [3/30], Step [6/139], Loss: 0.4202\n",
            "Epoch [3/30], Step [7/139], Loss: 0.5080\n",
            "Epoch [3/30], Step [8/139], Loss: 0.4851\n",
            "Epoch [3/30], Step [9/139], Loss: 0.3930\n",
            "Epoch [3/30], Step [10/139], Loss: 0.4619\n",
            "Epoch [3/30], Step [11/139], Loss: 0.4100\n",
            "Epoch [3/30], Step [12/139], Loss: 0.4663\n",
            "Epoch [3/30], Step [13/139], Loss: 0.4722\n",
            "Epoch [3/30], Step [14/139], Loss: 0.4982\n",
            "Epoch [3/30], Step [15/139], Loss: 0.4505\n",
            "Epoch [3/30], Step [16/139], Loss: 0.4299\n",
            "Epoch [3/30], Step [17/139], Loss: 0.4956\n",
            "Epoch [3/30], Step [18/139], Loss: 0.4084\n",
            "Epoch [3/30], Step [19/139], Loss: 0.4261\n",
            "Epoch [3/30], Step [20/139], Loss: 0.4126\n",
            "Epoch [3/30], Step [21/139], Loss: 0.4507\n",
            "Epoch [3/30], Step [22/139], Loss: 0.3856\n",
            "Epoch [3/30], Step [23/139], Loss: 0.3478\n",
            "Epoch [3/30], Step [24/139], Loss: 0.4422\n",
            "Epoch [3/30], Step [25/139], Loss: 0.3789\n",
            "Epoch [3/30], Step [26/139], Loss: 0.4283\n",
            "Epoch [3/30], Step [27/139], Loss: 0.3687\n",
            "Epoch [3/30], Step [28/139], Loss: 0.3920\n",
            "Epoch [3/30], Step [29/139], Loss: 0.3448\n",
            "Epoch [3/30], Step [30/139], Loss: 0.4033\n",
            "Epoch [3/30], Step [31/139], Loss: 0.3012\n",
            "Epoch [3/30], Step [32/139], Loss: 0.3913\n",
            "Epoch [3/30], Step [33/139], Loss: 0.4130\n",
            "Epoch [3/30], Step [34/139], Loss: 0.3484\n",
            "Epoch [3/30], Step [35/139], Loss: 0.3590\n",
            "Epoch [3/30], Step [36/139], Loss: 0.3939\n",
            "Epoch [3/30], Step [37/139], Loss: 0.4806\n",
            "Epoch [3/30], Step [38/139], Loss: 0.3170\n",
            "Epoch [3/30], Step [39/139], Loss: 0.3270\n",
            "Epoch [3/30], Step [40/139], Loss: 0.2992\n",
            "Epoch [3/30], Step [41/139], Loss: 0.4027\n",
            "Epoch [3/30], Step [42/139], Loss: 0.3125\n",
            "Epoch [3/30], Step [43/139], Loss: 0.2760\n",
            "Epoch [3/30], Step [44/139], Loss: 0.3102\n",
            "Epoch [3/30], Step [45/139], Loss: 0.3134\n",
            "Epoch [3/30], Step [46/139], Loss: 0.4181\n",
            "Epoch [3/30], Step [47/139], Loss: 0.3818\n",
            "Epoch [3/30], Step [48/139], Loss: 0.3275\n",
            "Epoch [3/30], Step [49/139], Loss: 0.3439\n",
            "Epoch [3/30], Step [50/139], Loss: 0.3684\n",
            "Epoch [3/30], Step [51/139], Loss: 0.3692\n",
            "Epoch [3/30], Step [52/139], Loss: 0.2698\n",
            "Epoch [3/30], Step [53/139], Loss: 0.2748\n",
            "Epoch [3/30], Step [54/139], Loss: 0.3232\n",
            "Epoch [3/30], Step [55/139], Loss: 0.3223\n",
            "Epoch [3/30], Step [56/139], Loss: 0.3548\n",
            "Epoch [3/30], Step [57/139], Loss: 0.3262\n",
            "Epoch [3/30], Step [58/139], Loss: 0.3790\n",
            "Epoch [3/30], Step [59/139], Loss: 0.3830\n",
            "Epoch [3/30], Step [60/139], Loss: 0.2929\n",
            "Epoch [3/30], Step [61/139], Loss: 0.3051\n",
            "Epoch [3/30], Step [62/139], Loss: 0.3042\n",
            "Epoch [3/30], Step [63/139], Loss: 0.3219\n",
            "Epoch [3/30], Step [64/139], Loss: 0.2649\n",
            "Epoch [3/30], Step [65/139], Loss: 0.2939\n",
            "Epoch [3/30], Step [66/139], Loss: 0.3366\n",
            "Epoch [3/30], Step [67/139], Loss: 0.3388\n",
            "Epoch [3/30], Step [68/139], Loss: 0.3037\n",
            "Epoch [3/30], Step [69/139], Loss: 0.2801\n",
            "Epoch [3/30], Step [70/139], Loss: 0.3012\n",
            "Epoch [3/30], Step [71/139], Loss: 0.2922\n",
            "Epoch [3/30], Step [72/139], Loss: 0.3656\n",
            "Epoch [3/30], Step [73/139], Loss: 0.3021\n",
            "Epoch [3/30], Step [74/139], Loss: 0.2530\n",
            "Epoch [3/30], Step [75/139], Loss: 0.3273\n",
            "Epoch [3/30], Step [76/139], Loss: 0.3547\n",
            "Epoch [3/30], Step [77/139], Loss: 0.3003\n",
            "Epoch [3/30], Step [78/139], Loss: 0.2456\n",
            "Epoch [3/30], Step [79/139], Loss: 0.3200\n",
            "Epoch [3/30], Step [80/139], Loss: 0.2594\n",
            "Epoch [3/30], Step [81/139], Loss: 0.2872\n",
            "Epoch [3/30], Step [82/139], Loss: 0.2609\n",
            "Epoch [3/30], Step [83/139], Loss: 0.3131\n",
            "Epoch [3/30], Step [84/139], Loss: 0.2554\n",
            "Epoch [3/30], Step [85/139], Loss: 0.2934\n",
            "Epoch [3/30], Step [86/139], Loss: 0.2369\n",
            "Epoch [3/30], Step [87/139], Loss: 0.2373\n",
            "Epoch [3/30], Step [88/139], Loss: 0.2530\n",
            "Epoch [3/30], Step [89/139], Loss: 0.2878\n",
            "Epoch [3/30], Step [90/139], Loss: 0.2507\n",
            "Epoch [3/30], Step [91/139], Loss: 0.3277\n",
            "Epoch [3/30], Step [92/139], Loss: 0.2358\n",
            "Epoch [3/30], Step [93/139], Loss: 0.2109\n",
            "Epoch [3/30], Step [94/139], Loss: 0.2619\n",
            "Epoch [3/30], Step [95/139], Loss: 0.2770\n",
            "Epoch [3/30], Step [96/139], Loss: 0.2471\n",
            "Epoch [3/30], Step [97/139], Loss: 0.2520\n",
            "Epoch [3/30], Step [98/139], Loss: 0.2339\n",
            "Epoch [3/30], Step [99/139], Loss: 0.2132\n",
            "Epoch [3/30], Step [100/139], Loss: 0.2161\n",
            "Epoch [3/30], Step [101/139], Loss: 0.2394\n",
            "Epoch [3/30], Step [102/139], Loss: 0.2399\n",
            "Epoch [3/30], Step [103/139], Loss: 0.2375\n",
            "Epoch [3/30], Step [104/139], Loss: 0.2394\n",
            "Epoch [3/30], Step [105/139], Loss: 0.1633\n",
            "Epoch [3/30], Step [106/139], Loss: 0.2264\n",
            "Epoch [3/30], Step [107/139], Loss: 0.2597\n",
            "Epoch [3/30], Step [108/139], Loss: 0.1641\n",
            "Epoch [3/30], Step [109/139], Loss: 0.2139\n",
            "Epoch [3/30], Step [110/139], Loss: 0.2513\n",
            "Epoch [3/30], Step [111/139], Loss: 0.1818\n",
            "Epoch [3/30], Step [112/139], Loss: 0.2133\n",
            "Epoch [3/30], Step [113/139], Loss: 0.2089\n",
            "Epoch [3/30], Step [114/139], Loss: 0.2459\n",
            "Epoch [3/30], Step [115/139], Loss: 0.2575\n",
            "Epoch [3/30], Step [116/139], Loss: 0.2408\n",
            "Epoch [3/30], Step [117/139], Loss: 0.2879\n",
            "Epoch [3/30], Step [118/139], Loss: 0.1950\n",
            "Epoch [3/30], Step [119/139], Loss: 0.2560\n",
            "Epoch [3/30], Step [120/139], Loss: 0.2347\n",
            "Epoch [3/30], Step [121/139], Loss: 0.2328\n",
            "Epoch [3/30], Step [122/139], Loss: 0.1878\n",
            "Epoch [3/30], Step [123/139], Loss: 0.2677\n",
            "Epoch [3/30], Step [124/139], Loss: 0.2087\n",
            "Epoch [3/30], Step [125/139], Loss: 0.1859\n",
            "Epoch [3/30], Step [126/139], Loss: 0.1371\n",
            "Epoch [3/30], Step [127/139], Loss: 0.1884\n",
            "Epoch [3/30], Step [128/139], Loss: 0.1620\n",
            "Epoch [3/30], Step [129/139], Loss: 0.2174\n",
            "Epoch [3/30], Step [130/139], Loss: 0.2153\n",
            "Epoch [3/30], Step [131/139], Loss: 0.2579\n",
            "Epoch [3/30], Step [132/139], Loss: 0.1631\n",
            "Epoch [3/30], Step [133/139], Loss: 0.2039\n",
            "Epoch [3/30], Step [134/139], Loss: 0.1845\n",
            "Epoch [3/30], Step [135/139], Loss: 0.2553\n",
            "Epoch [3/30], Step [136/139], Loss: 0.1992\n",
            "Epoch [3/30], Step [137/139], Loss: 0.1955\n",
            "Epoch [3/30], Step [138/139], Loss: 0.2195\n",
            "Epoch [3/30], Step [139/139], Loss: 0.2789\n",
            "Start validation #3\n",
            "Validation #3  Average Loss: 0.2187\n",
            "Best performance at epoch: 3\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [4/30], Step [1/139], Loss: 0.1652\n",
            "Epoch [4/30], Step [2/139], Loss: 0.2333\n",
            "Epoch [4/30], Step [3/139], Loss: 0.1781\n",
            "Epoch [4/30], Step [4/139], Loss: 0.1917\n",
            "Epoch [4/30], Step [5/139], Loss: 0.2568\n",
            "Epoch [4/30], Step [6/139], Loss: 0.2285\n",
            "Epoch [4/30], Step [7/139], Loss: 0.2047\n",
            "Epoch [4/30], Step [8/139], Loss: 0.1706\n",
            "Epoch [4/30], Step [9/139], Loss: 0.1767\n",
            "Epoch [4/30], Step [10/139], Loss: 0.1577\n",
            "Epoch [4/30], Step [11/139], Loss: 0.1871\n",
            "Epoch [4/30], Step [12/139], Loss: 0.1844\n",
            "Epoch [4/30], Step [13/139], Loss: 0.1533\n",
            "Epoch [4/30], Step [14/139], Loss: 0.1551\n",
            "Epoch [4/30], Step [15/139], Loss: 0.1783\n",
            "Epoch [4/30], Step [16/139], Loss: 0.1617\n",
            "Epoch [4/30], Step [17/139], Loss: 0.2410\n",
            "Epoch [4/30], Step [18/139], Loss: 0.1655\n",
            "Epoch [4/30], Step [19/139], Loss: 0.1996\n",
            "Epoch [4/30], Step [20/139], Loss: 0.2046\n",
            "Epoch [4/30], Step [21/139], Loss: 0.2167\n",
            "Epoch [4/30], Step [22/139], Loss: 0.1537\n",
            "Epoch [4/30], Step [23/139], Loss: 0.1827\n",
            "Epoch [4/30], Step [24/139], Loss: 0.1891\n",
            "Epoch [4/30], Step [25/139], Loss: 0.1593\n",
            "Epoch [4/30], Step [26/139], Loss: 0.1457\n",
            "Epoch [4/30], Step [27/139], Loss: 0.1886\n",
            "Epoch [4/30], Step [28/139], Loss: 0.1447\n",
            "Epoch [4/30], Step [29/139], Loss: 0.1594\n",
            "Epoch [4/30], Step [30/139], Loss: 0.2017\n",
            "Epoch [4/30], Step [31/139], Loss: 0.1366\n",
            "Epoch [4/30], Step [32/139], Loss: 0.1872\n",
            "Epoch [4/30], Step [33/139], Loss: 0.1691\n",
            "Epoch [4/30], Step [34/139], Loss: 0.1345\n",
            "Epoch [4/30], Step [35/139], Loss: 0.2246\n",
            "Epoch [4/30], Step [36/139], Loss: 0.1865\n",
            "Epoch [4/30], Step [37/139], Loss: 0.1391\n",
            "Epoch [4/30], Step [38/139], Loss: 0.1329\n",
            "Epoch [4/30], Step [39/139], Loss: 0.2189\n",
            "Epoch [4/30], Step [40/139], Loss: 0.1790\n",
            "Epoch [4/30], Step [41/139], Loss: 0.1930\n",
            "Epoch [4/30], Step [42/139], Loss: 0.1128\n",
            "Epoch [4/30], Step [43/139], Loss: 0.2098\n",
            "Epoch [4/30], Step [44/139], Loss: 0.1315\n",
            "Epoch [4/30], Step [45/139], Loss: 0.1752\n",
            "Epoch [4/30], Step [46/139], Loss: 0.1588\n",
            "Epoch [4/30], Step [47/139], Loss: 0.1614\n",
            "Epoch [4/30], Step [48/139], Loss: 0.1628\n",
            "Epoch [4/30], Step [49/139], Loss: 0.1505\n",
            "Epoch [4/30], Step [50/139], Loss: 0.1248\n",
            "Epoch [4/30], Step [51/139], Loss: 0.1878\n",
            "Epoch [4/30], Step [52/139], Loss: 0.1849\n",
            "Epoch [4/30], Step [53/139], Loss: 0.1191\n",
            "Epoch [4/30], Step [54/139], Loss: 0.1254\n",
            "Epoch [4/30], Step [55/139], Loss: 0.1540\n",
            "Epoch [4/30], Step [56/139], Loss: 0.2064\n",
            "Epoch [4/30], Step [57/139], Loss: 0.2641\n",
            "Epoch [4/30], Step [58/139], Loss: 0.1988\n",
            "Epoch [4/30], Step [59/139], Loss: 0.1553\n",
            "Epoch [4/30], Step [60/139], Loss: 0.1831\n",
            "Epoch [4/30], Step [61/139], Loss: 0.1294\n",
            "Epoch [4/30], Step [62/139], Loss: 0.1381\n",
            "Epoch [4/30], Step [63/139], Loss: 0.2132\n",
            "Epoch [4/30], Step [64/139], Loss: 0.1358\n",
            "Epoch [4/30], Step [65/139], Loss: 0.1584\n",
            "Epoch [4/30], Step [66/139], Loss: 0.1822\n",
            "Epoch [4/30], Step [67/139], Loss: 0.1650\n",
            "Epoch [4/30], Step [68/139], Loss: 0.1189\n",
            "Epoch [4/30], Step [69/139], Loss: 0.1384\n",
            "Epoch [4/30], Step [70/139], Loss: 0.1765\n",
            "Epoch [4/30], Step [71/139], Loss: 0.1442\n",
            "Epoch [4/30], Step [72/139], Loss: 0.1897\n",
            "Epoch [4/30], Step [73/139], Loss: 0.1695\n",
            "Epoch [4/30], Step [74/139], Loss: 0.1716\n",
            "Epoch [4/30], Step [75/139], Loss: 0.1677\n",
            "Epoch [4/30], Step [76/139], Loss: 0.1593\n",
            "Epoch [4/30], Step [77/139], Loss: 0.1455\n",
            "Epoch [4/30], Step [78/139], Loss: 0.1402\n",
            "Epoch [4/30], Step [79/139], Loss: 0.1355\n",
            "Epoch [4/30], Step [80/139], Loss: 0.1884\n",
            "Epoch [4/30], Step [81/139], Loss: 0.1509\n",
            "Epoch [4/30], Step [82/139], Loss: 0.1398\n",
            "Epoch [4/30], Step [83/139], Loss: 0.1415\n",
            "Epoch [4/30], Step [84/139], Loss: 0.1486\n",
            "Epoch [4/30], Step [85/139], Loss: 0.1659\n",
            "Epoch [4/30], Step [86/139], Loss: 0.1469\n",
            "Epoch [4/30], Step [87/139], Loss: 0.1751\n",
            "Epoch [4/30], Step [88/139], Loss: 0.1443\n",
            "Epoch [4/30], Step [89/139], Loss: 0.1830\n",
            "Epoch [4/30], Step [90/139], Loss: 0.1210\n",
            "Epoch [4/30], Step [91/139], Loss: 0.1856\n",
            "Epoch [4/30], Step [92/139], Loss: 0.1172\n",
            "Epoch [4/30], Step [93/139], Loss: 0.1134\n",
            "Epoch [4/30], Step [94/139], Loss: 0.1588\n",
            "Epoch [4/30], Step [95/139], Loss: 0.1475\n",
            "Epoch [4/30], Step [96/139], Loss: 0.1892\n",
            "Epoch [4/30], Step [97/139], Loss: 0.1864\n",
            "Epoch [4/30], Step [98/139], Loss: 0.1389\n",
            "Epoch [4/30], Step [99/139], Loss: 0.1713\n",
            "Epoch [4/30], Step [100/139], Loss: 0.1470\n",
            "Epoch [4/30], Step [101/139], Loss: 0.1300\n",
            "Epoch [4/30], Step [102/139], Loss: 0.1725\n",
            "Epoch [4/30], Step [103/139], Loss: 0.2026\n",
            "Epoch [4/30], Step [104/139], Loss: 0.1289\n",
            "Epoch [4/30], Step [105/139], Loss: 0.1287\n",
            "Epoch [4/30], Step [106/139], Loss: 0.1278\n",
            "Epoch [4/30], Step [107/139], Loss: 0.1376\n",
            "Epoch [4/30], Step [108/139], Loss: 0.1803\n",
            "Epoch [4/30], Step [109/139], Loss: 0.1545\n",
            "Epoch [4/30], Step [110/139], Loss: 0.1697\n",
            "Epoch [4/30], Step [111/139], Loss: 0.1700\n",
            "Epoch [4/30], Step [112/139], Loss: 0.1550\n",
            "Epoch [4/30], Step [113/139], Loss: 0.1528\n",
            "Epoch [4/30], Step [114/139], Loss: 0.1388\n",
            "Epoch [4/30], Step [115/139], Loss: 0.1727\n",
            "Epoch [4/30], Step [116/139], Loss: 0.1614\n",
            "Epoch [4/30], Step [117/139], Loss: 0.2056\n",
            "Epoch [4/30], Step [118/139], Loss: 0.1485\n",
            "Epoch [4/30], Step [119/139], Loss: 0.1342\n",
            "Epoch [4/30], Step [120/139], Loss: 0.1808\n",
            "Epoch [4/30], Step [121/139], Loss: 0.1519\n",
            "Epoch [4/30], Step [122/139], Loss: 0.1234\n",
            "Epoch [4/30], Step [123/139], Loss: 0.1768\n",
            "Epoch [4/30], Step [124/139], Loss: 0.1523\n",
            "Epoch [4/30], Step [125/139], Loss: 0.1584\n",
            "Epoch [4/30], Step [126/139], Loss: 0.1447\n",
            "Epoch [4/30], Step [127/139], Loss: 0.1640\n",
            "Epoch [4/30], Step [128/139], Loss: 0.1363\n",
            "Epoch [4/30], Step [129/139], Loss: 0.1492\n",
            "Epoch [4/30], Step [130/139], Loss: 0.1047\n",
            "Epoch [4/30], Step [131/139], Loss: 0.1482\n",
            "Epoch [4/30], Step [132/139], Loss: 0.2026\n",
            "Epoch [4/30], Step [133/139], Loss: 0.1315\n",
            "Epoch [4/30], Step [134/139], Loss: 0.1097\n",
            "Epoch [4/30], Step [135/139], Loss: 0.1313\n",
            "Epoch [4/30], Step [136/139], Loss: 0.1568\n",
            "Epoch [4/30], Step [137/139], Loss: 0.1110\n",
            "Epoch [4/30], Step [138/139], Loss: 0.0964\n",
            "Epoch [4/30], Step [139/139], Loss: 0.1414\n",
            "Start validation #4\n",
            "Validation #4  Average Loss: 0.1790\n",
            "Best performance at epoch: 4\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [5/30], Step [1/139], Loss: 0.1305\n",
            "Epoch [5/30], Step [2/139], Loss: 0.1056\n",
            "Epoch [5/30], Step [3/139], Loss: 0.1682\n",
            "Epoch [5/30], Step [4/139], Loss: 0.1164\n",
            "Epoch [5/30], Step [5/139], Loss: 0.1483\n",
            "Epoch [5/30], Step [6/139], Loss: 0.1480\n",
            "Epoch [5/30], Step [7/139], Loss: 0.1022\n",
            "Epoch [5/30], Step [8/139], Loss: 0.1352\n",
            "Epoch [5/30], Step [9/139], Loss: 0.1271\n",
            "Epoch [5/30], Step [10/139], Loss: 0.1230\n",
            "Epoch [5/30], Step [11/139], Loss: 0.1075\n",
            "Epoch [5/30], Step [12/139], Loss: 0.1711\n",
            "Epoch [5/30], Step [13/139], Loss: 0.1416\n",
            "Epoch [5/30], Step [14/139], Loss: 0.1406\n",
            "Epoch [5/30], Step [15/139], Loss: 0.1105\n",
            "Epoch [5/30], Step [16/139], Loss: 0.1455\n",
            "Epoch [5/30], Step [17/139], Loss: 0.1387\n",
            "Epoch [5/30], Step [18/139], Loss: 0.1620\n",
            "Epoch [5/30], Step [19/139], Loss: 0.1492\n",
            "Epoch [5/30], Step [20/139], Loss: 0.1220\n",
            "Epoch [5/30], Step [21/139], Loss: 0.1000\n",
            "Epoch [5/30], Step [22/139], Loss: 0.1344\n",
            "Epoch [5/30], Step [23/139], Loss: 0.1325\n",
            "Epoch [5/30], Step [24/139], Loss: 0.1221\n",
            "Epoch [5/30], Step [25/139], Loss: 0.1733\n",
            "Epoch [5/30], Step [26/139], Loss: 0.1261\n",
            "Epoch [5/30], Step [27/139], Loss: 0.1231\n",
            "Epoch [5/30], Step [28/139], Loss: 0.1075\n",
            "Epoch [5/30], Step [29/139], Loss: 0.1621\n",
            "Epoch [5/30], Step [30/139], Loss: 0.1107\n",
            "Epoch [5/30], Step [31/139], Loss: 0.1579\n",
            "Epoch [5/30], Step [32/139], Loss: 0.1639\n",
            "Epoch [5/30], Step [33/139], Loss: 0.1585\n",
            "Epoch [5/30], Step [34/139], Loss: 0.1411\n",
            "Epoch [5/30], Step [35/139], Loss: 0.1380\n",
            "Epoch [5/30], Step [36/139], Loss: 0.1849\n",
            "Epoch [5/30], Step [37/139], Loss: 0.1359\n",
            "Epoch [5/30], Step [38/139], Loss: 0.1498\n",
            "Epoch [5/30], Step [39/139], Loss: 0.1487\n",
            "Epoch [5/30], Step [40/139], Loss: 0.1565\n",
            "Epoch [5/30], Step [41/139], Loss: 0.1124\n",
            "Epoch [5/30], Step [42/139], Loss: 0.1515\n",
            "Epoch [5/30], Step [43/139], Loss: 0.1334\n",
            "Epoch [5/30], Step [44/139], Loss: 0.1309\n",
            "Epoch [5/30], Step [45/139], Loss: 0.1311\n",
            "Epoch [5/30], Step [46/139], Loss: 0.1740\n",
            "Epoch [5/30], Step [47/139], Loss: 0.1246\n",
            "Epoch [5/30], Step [48/139], Loss: 0.1263\n",
            "Epoch [5/30], Step [49/139], Loss: 0.1475\n",
            "Epoch [5/30], Step [50/139], Loss: 0.1515\n",
            "Epoch [5/30], Step [51/139], Loss: 0.1360\n",
            "Epoch [5/30], Step [52/139], Loss: 0.1571\n",
            "Epoch [5/30], Step [53/139], Loss: 0.1121\n",
            "Epoch [5/30], Step [54/139], Loss: 0.1392\n",
            "Epoch [5/30], Step [55/139], Loss: 0.1527\n",
            "Epoch [5/30], Step [56/139], Loss: 0.1334\n",
            "Epoch [5/30], Step [57/139], Loss: 0.1490\n",
            "Epoch [5/30], Step [58/139], Loss: 0.1098\n",
            "Epoch [5/30], Step [59/139], Loss: 0.1203\n",
            "Epoch [5/30], Step [60/139], Loss: 0.1518\n",
            "Epoch [5/30], Step [61/139], Loss: 0.1166\n",
            "Epoch [5/30], Step [62/139], Loss: 0.1444\n",
            "Epoch [5/30], Step [63/139], Loss: 0.1573\n",
            "Epoch [5/30], Step [64/139], Loss: 0.1179\n",
            "Epoch [5/30], Step [65/139], Loss: 0.2121\n",
            "Epoch [5/30], Step [66/139], Loss: 0.1168\n",
            "Epoch [5/30], Step [67/139], Loss: 0.1616\n",
            "Epoch [5/30], Step [68/139], Loss: 0.1322\n",
            "Epoch [5/30], Step [69/139], Loss: 0.1712\n",
            "Epoch [5/30], Step [70/139], Loss: 0.1411\n",
            "Epoch [5/30], Step [71/139], Loss: 0.1373\n",
            "Epoch [5/30], Step [72/139], Loss: 0.1168\n",
            "Epoch [5/30], Step [73/139], Loss: 0.1290\n",
            "Epoch [5/30], Step [74/139], Loss: 0.1084\n",
            "Epoch [5/30], Step [75/139], Loss: 0.1543\n",
            "Epoch [5/30], Step [76/139], Loss: 0.1538\n",
            "Epoch [5/30], Step [77/139], Loss: 0.1454\n",
            "Epoch [5/30], Step [78/139], Loss: 0.1071\n",
            "Epoch [5/30], Step [79/139], Loss: 0.1447\n",
            "Epoch [5/30], Step [80/139], Loss: 0.1219\n",
            "Epoch [5/30], Step [81/139], Loss: 0.1310\n",
            "Epoch [5/30], Step [82/139], Loss: 0.1662\n",
            "Epoch [5/30], Step [83/139], Loss: 0.1058\n",
            "Epoch [5/30], Step [84/139], Loss: 0.1695\n",
            "Epoch [5/30], Step [85/139], Loss: 0.1361\n",
            "Epoch [5/30], Step [86/139], Loss: 0.0990\n",
            "Epoch [5/30], Step [87/139], Loss: 0.1161\n",
            "Epoch [5/30], Step [88/139], Loss: 0.1342\n",
            "Epoch [5/30], Step [89/139], Loss: 0.1537\n",
            "Epoch [5/30], Step [90/139], Loss: 0.1300\n",
            "Epoch [5/30], Step [91/139], Loss: 0.1571\n",
            "Epoch [5/30], Step [92/139], Loss: 0.1216\n",
            "Epoch [5/30], Step [93/139], Loss: 0.1518\n",
            "Epoch [5/30], Step [94/139], Loss: 0.1240\n",
            "Epoch [5/30], Step [95/139], Loss: 0.1497\n",
            "Epoch [5/30], Step [96/139], Loss: 0.1758\n",
            "Epoch [5/30], Step [97/139], Loss: 0.1581\n",
            "Epoch [5/30], Step [98/139], Loss: 0.1185\n",
            "Epoch [5/30], Step [99/139], Loss: 0.1249\n",
            "Epoch [5/30], Step [100/139], Loss: 0.1098\n",
            "Epoch [5/30], Step [101/139], Loss: 0.1434\n",
            "Epoch [5/30], Step [102/139], Loss: 0.1418\n",
            "Epoch [5/30], Step [103/139], Loss: 0.1266\n",
            "Epoch [5/30], Step [104/139], Loss: 0.1336\n",
            "Epoch [5/30], Step [105/139], Loss: 0.1558\n",
            "Epoch [5/30], Step [106/139], Loss: 0.1050\n",
            "Epoch [5/30], Step [107/139], Loss: 0.1721\n",
            "Epoch [5/30], Step [108/139], Loss: 0.1160\n",
            "Epoch [5/30], Step [109/139], Loss: 0.1185\n",
            "Epoch [5/30], Step [110/139], Loss: 0.1084\n",
            "Epoch [5/30], Step [111/139], Loss: 0.1175\n",
            "Epoch [5/30], Step [112/139], Loss: 0.1551\n",
            "Epoch [5/30], Step [113/139], Loss: 0.1434\n",
            "Epoch [5/30], Step [114/139], Loss: 0.1321\n",
            "Epoch [5/30], Step [115/139], Loss: 0.1647\n",
            "Epoch [5/30], Step [116/139], Loss: 0.1443\n",
            "Epoch [5/30], Step [117/139], Loss: 0.1490\n",
            "Epoch [5/30], Step [118/139], Loss: 0.1203\n",
            "Epoch [5/30], Step [119/139], Loss: 0.1267\n",
            "Epoch [5/30], Step [120/139], Loss: 0.1309\n",
            "Epoch [5/30], Step [121/139], Loss: 0.1318\n",
            "Epoch [5/30], Step [122/139], Loss: 0.1218\n",
            "Epoch [5/30], Step [123/139], Loss: 0.1161\n",
            "Epoch [5/30], Step [124/139], Loss: 0.1552\n",
            "Epoch [5/30], Step [125/139], Loss: 0.1466\n",
            "Epoch [5/30], Step [126/139], Loss: 0.1560\n",
            "Epoch [5/30], Step [127/139], Loss: 0.1249\n",
            "Epoch [5/30], Step [128/139], Loss: 0.1192\n",
            "Epoch [5/30], Step [129/139], Loss: 0.1611\n",
            "Epoch [5/30], Step [130/139], Loss: 0.1252\n",
            "Epoch [5/30], Step [131/139], Loss: 0.1501\n",
            "Epoch [5/30], Step [132/139], Loss: 0.1237\n",
            "Epoch [5/30], Step [133/139], Loss: 0.1748\n",
            "Epoch [5/30], Step [134/139], Loss: 0.1135\n",
            "Epoch [5/30], Step [135/139], Loss: 0.1207\n",
            "Epoch [5/30], Step [136/139], Loss: 0.1296\n",
            "Epoch [5/30], Step [137/139], Loss: 0.1268\n",
            "Epoch [5/30], Step [138/139], Loss: 0.0748\n",
            "Epoch [5/30], Step [139/139], Loss: 0.1640\n",
            "Start validation #5\n",
            "Validation #5  Average Loss: 0.1536\n",
            "Best performance at epoch: 5\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [6/30], Step [1/139], Loss: 0.1290\n",
            "Epoch [6/30], Step [2/139], Loss: 0.0881\n",
            "Epoch [6/30], Step [3/139], Loss: 0.1314\n",
            "Epoch [6/30], Step [4/139], Loss: 0.1201\n",
            "Epoch [6/30], Step [5/139], Loss: 0.1099\n",
            "Epoch [6/30], Step [6/139], Loss: 0.0992\n",
            "Epoch [6/30], Step [7/139], Loss: 0.1355\n",
            "Epoch [6/30], Step [8/139], Loss: 0.1311\n",
            "Epoch [6/30], Step [9/139], Loss: 0.1392\n",
            "Epoch [6/30], Step [10/139], Loss: 0.0887\n",
            "Epoch [6/30], Step [11/139], Loss: 0.1293\n",
            "Epoch [6/30], Step [12/139], Loss: 0.1206\n",
            "Epoch [6/30], Step [13/139], Loss: 0.1307\n",
            "Epoch [6/30], Step [14/139], Loss: 0.1781\n",
            "Epoch [6/30], Step [15/139], Loss: 0.1098\n",
            "Epoch [6/30], Step [16/139], Loss: 0.1577\n",
            "Epoch [6/30], Step [17/139], Loss: 0.1021\n",
            "Epoch [6/30], Step [18/139], Loss: 0.1295\n",
            "Epoch [6/30], Step [19/139], Loss: 0.1208\n",
            "Epoch [6/30], Step [20/139], Loss: 0.1085\n",
            "Epoch [6/30], Step [21/139], Loss: 0.1224\n",
            "Epoch [6/30], Step [22/139], Loss: 0.1257\n",
            "Epoch [6/30], Step [23/139], Loss: 0.1165\n",
            "Epoch [6/30], Step [24/139], Loss: 0.1391\n",
            "Epoch [6/30], Step [25/139], Loss: 0.1316\n",
            "Epoch [6/30], Step [26/139], Loss: 0.1525\n",
            "Epoch [6/30], Step [27/139], Loss: 0.1104\n",
            "Epoch [6/30], Step [28/139], Loss: 0.1040\n",
            "Epoch [6/30], Step [29/139], Loss: 0.0856\n",
            "Epoch [6/30], Step [30/139], Loss: 0.1088\n",
            "Epoch [6/30], Step [31/139], Loss: 0.0908\n",
            "Epoch [6/30], Step [32/139], Loss: 0.1727\n",
            "Epoch [6/30], Step [33/139], Loss: 0.0817\n",
            "Epoch [6/30], Step [34/139], Loss: 0.1243\n",
            "Epoch [6/30], Step [35/139], Loss: 0.1625\n",
            "Epoch [6/30], Step [36/139], Loss: 0.0993\n",
            "Epoch [6/30], Step [37/139], Loss: 0.1444\n",
            "Epoch [6/30], Step [38/139], Loss: 0.1064\n",
            "Epoch [6/30], Step [39/139], Loss: 0.1385\n",
            "Epoch [6/30], Step [40/139], Loss: 0.1438\n",
            "Epoch [6/30], Step [41/139], Loss: 0.1003\n",
            "Epoch [6/30], Step [42/139], Loss: 0.1213\n",
            "Epoch [6/30], Step [43/139], Loss: 0.1249\n",
            "Epoch [6/30], Step [44/139], Loss: 0.1259\n",
            "Epoch [6/30], Step [45/139], Loss: 0.1159\n",
            "Epoch [6/30], Step [46/139], Loss: 0.1362\n",
            "Epoch [6/30], Step [47/139], Loss: 0.1500\n",
            "Epoch [6/30], Step [48/139], Loss: 0.1343\n",
            "Epoch [6/30], Step [49/139], Loss: 0.1009\n",
            "Epoch [6/30], Step [50/139], Loss: 0.1204\n",
            "Epoch [6/30], Step [51/139], Loss: 0.1464\n",
            "Epoch [6/30], Step [52/139], Loss: 0.1277\n",
            "Epoch [6/30], Step [53/139], Loss: 0.1214\n",
            "Epoch [6/30], Step [54/139], Loss: 0.1287\n",
            "Epoch [6/30], Step [55/139], Loss: 0.1298\n",
            "Epoch [6/30], Step [56/139], Loss: 0.1203\n",
            "Epoch [6/30], Step [57/139], Loss: 0.1467\n",
            "Epoch [6/30], Step [58/139], Loss: 0.1657\n",
            "Epoch [6/30], Step [59/139], Loss: 0.0900\n",
            "Epoch [6/30], Step [60/139], Loss: 0.1523\n",
            "Epoch [6/30], Step [61/139], Loss: 0.0955\n",
            "Epoch [6/30], Step [62/139], Loss: 0.1082\n",
            "Epoch [6/30], Step [63/139], Loss: 0.1503\n",
            "Epoch [6/30], Step [64/139], Loss: 0.1029\n",
            "Epoch [6/30], Step [65/139], Loss: 0.1513\n",
            "Epoch [6/30], Step [66/139], Loss: 0.1318\n",
            "Epoch [6/30], Step [67/139], Loss: 0.1014\n",
            "Epoch [6/30], Step [68/139], Loss: 0.1038\n",
            "Epoch [6/30], Step [69/139], Loss: 0.1060\n",
            "Epoch [6/30], Step [70/139], Loss: 0.1402\n",
            "Epoch [6/30], Step [71/139], Loss: 0.1120\n",
            "Epoch [6/30], Step [72/139], Loss: 0.1373\n",
            "Epoch [6/30], Step [73/139], Loss: 0.1212\n",
            "Epoch [6/30], Step [74/139], Loss: 0.0994\n",
            "Epoch [6/30], Step [75/139], Loss: 0.1098\n",
            "Epoch [6/30], Step [76/139], Loss: 0.1351\n",
            "Epoch [6/30], Step [77/139], Loss: 0.1055\n",
            "Epoch [6/30], Step [78/139], Loss: 0.1528\n",
            "Epoch [6/30], Step [79/139], Loss: 0.1166\n",
            "Epoch [6/30], Step [80/139], Loss: 0.1661\n",
            "Epoch [6/30], Step [81/139], Loss: 0.1266\n",
            "Epoch [6/30], Step [82/139], Loss: 0.1054\n",
            "Epoch [6/30], Step [83/139], Loss: 0.1250\n",
            "Epoch [6/30], Step [84/139], Loss: 0.1487\n",
            "Epoch [6/30], Step [85/139], Loss: 0.1228\n",
            "Epoch [6/30], Step [86/139], Loss: 0.1467\n",
            "Epoch [6/30], Step [87/139], Loss: 0.1233\n",
            "Epoch [6/30], Step [88/139], Loss: 0.1269\n",
            "Epoch [6/30], Step [89/139], Loss: 0.0973\n",
            "Epoch [6/30], Step [90/139], Loss: 0.1259\n",
            "Epoch [6/30], Step [91/139], Loss: 0.0930\n",
            "Epoch [6/30], Step [92/139], Loss: 0.0947\n",
            "Epoch [6/30], Step [93/139], Loss: 0.0960\n",
            "Epoch [6/30], Step [94/139], Loss: 0.1042\n",
            "Epoch [6/30], Step [95/139], Loss: 0.1259\n",
            "Epoch [6/30], Step [96/139], Loss: 0.1015\n",
            "Epoch [6/30], Step [97/139], Loss: 0.1074\n",
            "Epoch [6/30], Step [98/139], Loss: 0.1448\n",
            "Epoch [6/30], Step [99/139], Loss: 0.1321\n",
            "Epoch [6/30], Step [100/139], Loss: 0.1343\n",
            "Epoch [6/30], Step [101/139], Loss: 0.0970\n",
            "Epoch [6/30], Step [102/139], Loss: 0.1482\n",
            "Epoch [6/30], Step [103/139], Loss: 0.0840\n",
            "Epoch [6/30], Step [104/139], Loss: 0.1439\n",
            "Epoch [6/30], Step [105/139], Loss: 0.0989\n",
            "Epoch [6/30], Step [106/139], Loss: 0.1494\n",
            "Epoch [6/30], Step [107/139], Loss: 0.0781\n",
            "Epoch [6/30], Step [108/139], Loss: 0.1373\n",
            "Epoch [6/30], Step [109/139], Loss: 0.1288\n",
            "Epoch [6/30], Step [110/139], Loss: 0.1145\n",
            "Epoch [6/30], Step [111/139], Loss: 0.0933\n",
            "Epoch [6/30], Step [112/139], Loss: 0.0984\n",
            "Epoch [6/30], Step [113/139], Loss: 0.1270\n",
            "Epoch [6/30], Step [114/139], Loss: 0.0966\n",
            "Epoch [6/30], Step [115/139], Loss: 0.0953\n",
            "Epoch [6/30], Step [116/139], Loss: 0.1225\n",
            "Epoch [6/30], Step [117/139], Loss: 0.1361\n",
            "Epoch [6/30], Step [118/139], Loss: 0.1061\n",
            "Epoch [6/30], Step [119/139], Loss: 0.1267\n",
            "Epoch [6/30], Step [120/139], Loss: 0.1180\n",
            "Epoch [6/30], Step [121/139], Loss: 0.1035\n",
            "Epoch [6/30], Step [122/139], Loss: 0.1067\n",
            "Epoch [6/30], Step [123/139], Loss: 0.1657\n",
            "Epoch [6/30], Step [124/139], Loss: 0.1195\n",
            "Epoch [6/30], Step [125/139], Loss: 0.1121\n",
            "Epoch [6/30], Step [126/139], Loss: 0.1301\n",
            "Epoch [6/30], Step [127/139], Loss: 0.1239\n",
            "Epoch [6/30], Step [128/139], Loss: 0.1527\n",
            "Epoch [6/30], Step [129/139], Loss: 0.1038\n",
            "Epoch [6/30], Step [130/139], Loss: 0.1150\n",
            "Epoch [6/30], Step [131/139], Loss: 0.1292\n",
            "Epoch [6/30], Step [132/139], Loss: 0.1226\n",
            "Epoch [6/30], Step [133/139], Loss: 0.1148\n",
            "Epoch [6/30], Step [134/139], Loss: 0.1242\n",
            "Epoch [6/30], Step [135/139], Loss: 0.1336\n",
            "Epoch [6/30], Step [136/139], Loss: 0.1132\n",
            "Epoch [6/30], Step [137/139], Loss: 0.0802\n",
            "Epoch [6/30], Step [138/139], Loss: 0.0942\n",
            "Epoch [6/30], Step [139/139], Loss: 0.1090\n",
            "Start validation #6\n",
            "Validation #6  Average Loss: 0.1350\n",
            "Best performance at epoch: 6\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [7/30], Step [1/139], Loss: 0.1209\n",
            "Epoch [7/30], Step [2/139], Loss: 0.1089\n",
            "Epoch [7/30], Step [3/139], Loss: 0.1620\n",
            "Epoch [7/30], Step [4/139], Loss: 0.1760\n",
            "Epoch [7/30], Step [5/139], Loss: 0.1045\n",
            "Epoch [7/30], Step [6/139], Loss: 0.0957\n",
            "Epoch [7/30], Step [7/139], Loss: 0.0875\n",
            "Epoch [7/30], Step [8/139], Loss: 0.0753\n",
            "Epoch [7/30], Step [9/139], Loss: 0.1011\n",
            "Epoch [7/30], Step [10/139], Loss: 0.1393\n",
            "Epoch [7/30], Step [11/139], Loss: 0.1070\n",
            "Epoch [7/30], Step [12/139], Loss: 0.1405\n",
            "Epoch [7/30], Step [13/139], Loss: 0.1582\n",
            "Epoch [7/30], Step [14/139], Loss: 0.1052\n",
            "Epoch [7/30], Step [15/139], Loss: 0.1109\n",
            "Epoch [7/30], Step [16/139], Loss: 0.1133\n",
            "Epoch [7/30], Step [17/139], Loss: 0.1244\n",
            "Epoch [7/30], Step [18/139], Loss: 0.1475\n",
            "Epoch [7/30], Step [19/139], Loss: 0.1300\n",
            "Epoch [7/30], Step [20/139], Loss: 0.1220\n",
            "Epoch [7/30], Step [21/139], Loss: 0.1072\n",
            "Epoch [7/30], Step [22/139], Loss: 0.1211\n",
            "Epoch [7/30], Step [23/139], Loss: 0.0997\n",
            "Epoch [7/30], Step [24/139], Loss: 0.1131\n",
            "Epoch [7/30], Step [25/139], Loss: 0.1262\n",
            "Epoch [7/30], Step [26/139], Loss: 0.1212\n",
            "Epoch [7/30], Step [27/139], Loss: 0.1239\n",
            "Epoch [7/30], Step [28/139], Loss: 0.1190\n",
            "Epoch [7/30], Step [29/139], Loss: 0.1220\n",
            "Epoch [7/30], Step [30/139], Loss: 0.0959\n",
            "Epoch [7/30], Step [31/139], Loss: 0.1013\n",
            "Epoch [7/30], Step [32/139], Loss: 0.1160\n",
            "Epoch [7/30], Step [33/139], Loss: 0.1238\n",
            "Epoch [7/30], Step [34/139], Loss: 0.1097\n",
            "Epoch [7/30], Step [35/139], Loss: 0.0864\n",
            "Epoch [7/30], Step [36/139], Loss: 0.1215\n",
            "Epoch [7/30], Step [37/139], Loss: 0.0852\n",
            "Epoch [7/30], Step [38/139], Loss: 0.1066\n",
            "Epoch [7/30], Step [39/139], Loss: 0.1349\n",
            "Epoch [7/30], Step [40/139], Loss: 0.0954\n",
            "Epoch [7/30], Step [41/139], Loss: 0.1022\n",
            "Epoch [7/30], Step [42/139], Loss: 0.1012\n",
            "Epoch [7/30], Step [43/139], Loss: 0.0981\n",
            "Epoch [7/30], Step [44/139], Loss: 0.1172\n",
            "Epoch [7/30], Step [45/139], Loss: 0.1046\n",
            "Epoch [7/30], Step [46/139], Loss: 0.1031\n",
            "Epoch [7/30], Step [47/139], Loss: 0.1019\n",
            "Epoch [7/30], Step [48/139], Loss: 0.1072\n",
            "Epoch [7/30], Step [49/139], Loss: 0.1184\n",
            "Epoch [7/30], Step [50/139], Loss: 0.1170\n",
            "Epoch [7/30], Step [51/139], Loss: 0.1289\n",
            "Epoch [7/30], Step [52/139], Loss: 0.0962\n",
            "Epoch [7/30], Step [53/139], Loss: 0.1090\n",
            "Epoch [7/30], Step [54/139], Loss: 0.1184\n",
            "Epoch [7/30], Step [55/139], Loss: 0.1337\n",
            "Epoch [7/30], Step [56/139], Loss: 0.1133\n",
            "Epoch [7/30], Step [57/139], Loss: 0.0854\n",
            "Epoch [7/30], Step [58/139], Loss: 0.0948\n",
            "Epoch [7/30], Step [59/139], Loss: 0.1493\n",
            "Epoch [7/30], Step [60/139], Loss: 0.1203\n",
            "Epoch [7/30], Step [61/139], Loss: 0.1225\n",
            "Epoch [7/30], Step [62/139], Loss: 0.1236\n",
            "Epoch [7/30], Step [63/139], Loss: 0.0774\n",
            "Epoch [7/30], Step [64/139], Loss: 0.0924\n",
            "Epoch [7/30], Step [65/139], Loss: 0.1191\n",
            "Epoch [7/30], Step [66/139], Loss: 0.1107\n",
            "Epoch [7/30], Step [67/139], Loss: 0.1288\n",
            "Epoch [7/30], Step [68/139], Loss: 0.1241\n",
            "Epoch [7/30], Step [69/139], Loss: 0.1104\n",
            "Epoch [7/30], Step [70/139], Loss: 0.1033\n",
            "Epoch [7/30], Step [71/139], Loss: 0.1198\n",
            "Epoch [7/30], Step [72/139], Loss: 0.1262\n",
            "Epoch [7/30], Step [73/139], Loss: 0.1074\n",
            "Epoch [7/30], Step [74/139], Loss: 0.0921\n",
            "Epoch [7/30], Step [75/139], Loss: 0.1018\n",
            "Epoch [7/30], Step [76/139], Loss: 0.0932\n",
            "Epoch [7/30], Step [77/139], Loss: 0.1053\n",
            "Epoch [7/30], Step [78/139], Loss: 0.1037\n",
            "Epoch [7/30], Step [79/139], Loss: 0.0997\n",
            "Epoch [7/30], Step [80/139], Loss: 0.1093\n",
            "Epoch [7/30], Step [81/139], Loss: 0.1412\n",
            "Epoch [7/30], Step [82/139], Loss: 0.1127\n",
            "Epoch [7/30], Step [83/139], Loss: 0.1125\n",
            "Epoch [7/30], Step [84/139], Loss: 0.1083\n",
            "Epoch [7/30], Step [85/139], Loss: 0.1581\n",
            "Epoch [7/30], Step [86/139], Loss: 0.1057\n",
            "Epoch [7/30], Step [87/139], Loss: 0.1257\n",
            "Epoch [7/30], Step [88/139], Loss: 0.1285\n",
            "Epoch [7/30], Step [89/139], Loss: 0.0792\n",
            "Epoch [7/30], Step [90/139], Loss: 0.1112\n",
            "Epoch [7/30], Step [91/139], Loss: 0.1005\n",
            "Epoch [7/30], Step [92/139], Loss: 0.1089\n",
            "Epoch [7/30], Step [93/139], Loss: 0.0875\n",
            "Epoch [7/30], Step [94/139], Loss: 0.1081\n",
            "Epoch [7/30], Step [95/139], Loss: 0.1076\n",
            "Epoch [7/30], Step [96/139], Loss: 0.1178\n",
            "Epoch [7/30], Step [97/139], Loss: 0.1036\n",
            "Epoch [7/30], Step [98/139], Loss: 0.0934\n",
            "Epoch [7/30], Step [99/139], Loss: 0.0947\n",
            "Epoch [7/30], Step [100/139], Loss: 0.0828\n",
            "Epoch [7/30], Step [101/139], Loss: 0.0909\n",
            "Epoch [7/30], Step [102/139], Loss: 0.0951\n",
            "Epoch [7/30], Step [103/139], Loss: 0.1031\n",
            "Epoch [7/30], Step [104/139], Loss: 0.0849\n",
            "Epoch [7/30], Step [105/139], Loss: 0.1320\n",
            "Epoch [7/30], Step [106/139], Loss: 0.1236\n",
            "Epoch [7/30], Step [107/139], Loss: 0.0890\n",
            "Epoch [7/30], Step [108/139], Loss: 0.1084\n",
            "Epoch [7/30], Step [109/139], Loss: 0.1374\n",
            "Epoch [7/30], Step [110/139], Loss: 0.1047\n",
            "Epoch [7/30], Step [111/139], Loss: 0.0900\n",
            "Epoch [7/30], Step [112/139], Loss: 0.1148\n",
            "Epoch [7/30], Step [113/139], Loss: 0.1218\n",
            "Epoch [7/30], Step [114/139], Loss: 0.1077\n",
            "Epoch [7/30], Step [115/139], Loss: 0.1003\n",
            "Epoch [7/30], Step [116/139], Loss: 0.1509\n",
            "Epoch [7/30], Step [117/139], Loss: 0.0925\n",
            "Epoch [7/30], Step [118/139], Loss: 0.1309\n",
            "Epoch [7/30], Step [119/139], Loss: 0.1114\n",
            "Epoch [7/30], Step [120/139], Loss: 0.1148\n",
            "Epoch [7/30], Step [121/139], Loss: 0.1013\n",
            "Epoch [7/30], Step [122/139], Loss: 0.1019\n",
            "Epoch [7/30], Step [123/139], Loss: 0.1220\n",
            "Epoch [7/30], Step [124/139], Loss: 0.1034\n",
            "Epoch [7/30], Step [125/139], Loss: 0.1030\n",
            "Epoch [7/30], Step [126/139], Loss: 0.0997\n",
            "Epoch [7/30], Step [127/139], Loss: 0.1094\n",
            "Epoch [7/30], Step [128/139], Loss: 0.1275\n",
            "Epoch [7/30], Step [129/139], Loss: 0.0930\n",
            "Epoch [7/30], Step [130/139], Loss: 0.1075\n",
            "Epoch [7/30], Step [131/139], Loss: 0.1103\n",
            "Epoch [7/30], Step [132/139], Loss: 0.1259\n",
            "Epoch [7/30], Step [133/139], Loss: 0.1466\n",
            "Epoch [7/30], Step [134/139], Loss: 0.1063\n",
            "Epoch [7/30], Step [135/139], Loss: 0.0818\n",
            "Epoch [7/30], Step [136/139], Loss: 0.1009\n",
            "Epoch [7/30], Step [137/139], Loss: 0.1054\n",
            "Epoch [7/30], Step [138/139], Loss: 0.0991\n",
            "Epoch [7/30], Step [139/139], Loss: 0.1233\n",
            "Start validation #7\n",
            "Validation #7  Average Loss: 0.1247\n",
            "Best performance at epoch: 7\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [8/30], Step [1/139], Loss: 0.0956\n",
            "Epoch [8/30], Step [2/139], Loss: 0.0933\n",
            "Epoch [8/30], Step [3/139], Loss: 0.0995\n",
            "Epoch [8/30], Step [4/139], Loss: 0.0990\n",
            "Epoch [8/30], Step [5/139], Loss: 0.1081\n",
            "Epoch [8/30], Step [6/139], Loss: 0.1193\n",
            "Epoch [8/30], Step [7/139], Loss: 0.1064\n",
            "Epoch [8/30], Step [8/139], Loss: 0.1184\n",
            "Epoch [8/30], Step [9/139], Loss: 0.1324\n",
            "Epoch [8/30], Step [10/139], Loss: 0.1134\n",
            "Epoch [8/30], Step [11/139], Loss: 0.0907\n",
            "Epoch [8/30], Step [12/139], Loss: 0.1319\n",
            "Epoch [8/30], Step [13/139], Loss: 0.1295\n",
            "Epoch [8/30], Step [14/139], Loss: 0.0903\n",
            "Epoch [8/30], Step [15/139], Loss: 0.1067\n",
            "Epoch [8/30], Step [16/139], Loss: 0.0924\n",
            "Epoch [8/30], Step [17/139], Loss: 0.0970\n",
            "Epoch [8/30], Step [18/139], Loss: 0.1241\n",
            "Epoch [8/30], Step [19/139], Loss: 0.1202\n",
            "Epoch [8/30], Step [20/139], Loss: 0.1322\n",
            "Epoch [8/30], Step [21/139], Loss: 0.0806\n",
            "Epoch [8/30], Step [22/139], Loss: 0.1026\n",
            "Epoch [8/30], Step [23/139], Loss: 0.1023\n",
            "Epoch [8/30], Step [24/139], Loss: 0.1410\n",
            "Epoch [8/30], Step [25/139], Loss: 0.1158\n",
            "Epoch [8/30], Step [26/139], Loss: 0.0837\n",
            "Epoch [8/30], Step [27/139], Loss: 0.1497\n",
            "Epoch [8/30], Step [28/139], Loss: 0.0991\n",
            "Epoch [8/30], Step [29/139], Loss: 0.0904\n",
            "Epoch [8/30], Step [30/139], Loss: 0.1272\n",
            "Epoch [8/30], Step [31/139], Loss: 0.1112\n",
            "Epoch [8/30], Step [32/139], Loss: 0.0957\n",
            "Epoch [8/30], Step [33/139], Loss: 0.1443\n",
            "Epoch [8/30], Step [34/139], Loss: 0.1053\n",
            "Epoch [8/30], Step [35/139], Loss: 0.0902\n",
            "Epoch [8/30], Step [36/139], Loss: 0.0706\n",
            "Epoch [8/30], Step [37/139], Loss: 0.0894\n",
            "Epoch [8/30], Step [38/139], Loss: 0.1069\n",
            "Epoch [8/30], Step [39/139], Loss: 0.1032\n",
            "Epoch [8/30], Step [40/139], Loss: 0.0900\n",
            "Epoch [8/30], Step [41/139], Loss: 0.1039\n",
            "Epoch [8/30], Step [42/139], Loss: 0.0979\n",
            "Epoch [8/30], Step [43/139], Loss: 0.0800\n",
            "Epoch [8/30], Step [44/139], Loss: 0.0782\n",
            "Epoch [8/30], Step [45/139], Loss: 0.1006\n",
            "Epoch [8/30], Step [46/139], Loss: 0.1156\n",
            "Epoch [8/30], Step [47/139], Loss: 0.1240\n",
            "Epoch [8/30], Step [48/139], Loss: 0.0910\n",
            "Epoch [8/30], Step [49/139], Loss: 0.1057\n",
            "Epoch [8/30], Step [50/139], Loss: 0.1168\n",
            "Epoch [8/30], Step [51/139], Loss: 0.1258\n",
            "Epoch [8/30], Step [52/139], Loss: 0.1331\n",
            "Epoch [8/30], Step [53/139], Loss: 0.0880\n",
            "Epoch [8/30], Step [54/139], Loss: 0.1102\n",
            "Epoch [8/30], Step [55/139], Loss: 0.1098\n",
            "Epoch [8/30], Step [56/139], Loss: 0.1164\n",
            "Epoch [8/30], Step [57/139], Loss: 0.0987\n",
            "Epoch [8/30], Step [58/139], Loss: 0.0965\n",
            "Epoch [8/30], Step [59/139], Loss: 0.1015\n",
            "Epoch [8/30], Step [60/139], Loss: 0.0997\n",
            "Epoch [8/30], Step [61/139], Loss: 0.1051\n",
            "Epoch [8/30], Step [62/139], Loss: 0.1192\n",
            "Epoch [8/30], Step [63/139], Loss: 0.1296\n",
            "Epoch [8/30], Step [64/139], Loss: 0.1142\n",
            "Epoch [8/30], Step [65/139], Loss: 0.1018\n",
            "Epoch [8/30], Step [66/139], Loss: 0.1120\n",
            "Epoch [8/30], Step [67/139], Loss: 0.0939\n",
            "Epoch [8/30], Step [68/139], Loss: 0.0830\n",
            "Epoch [8/30], Step [69/139], Loss: 0.1089\n",
            "Epoch [8/30], Step [70/139], Loss: 0.1080\n",
            "Epoch [8/30], Step [71/139], Loss: 0.1085\n",
            "Epoch [8/30], Step [72/139], Loss: 0.1262\n",
            "Epoch [8/30], Step [73/139], Loss: 0.1000\n",
            "Epoch [8/30], Step [74/139], Loss: 0.1161\n",
            "Epoch [8/30], Step [75/139], Loss: 0.1211\n",
            "Epoch [8/30], Step [76/139], Loss: 0.1013\n",
            "Epoch [8/30], Step [77/139], Loss: 0.1164\n",
            "Epoch [8/30], Step [78/139], Loss: 0.0957\n",
            "Epoch [8/30], Step [79/139], Loss: 0.0837\n",
            "Epoch [8/30], Step [80/139], Loss: 0.1005\n",
            "Epoch [8/30], Step [81/139], Loss: 0.1003\n",
            "Epoch [8/30], Step [82/139], Loss: 0.0920\n",
            "Epoch [8/30], Step [83/139], Loss: 0.1050\n",
            "Epoch [8/30], Step [84/139], Loss: 0.1152\n",
            "Epoch [8/30], Step [85/139], Loss: 0.0917\n",
            "Epoch [8/30], Step [86/139], Loss: 0.0930\n",
            "Epoch [8/30], Step [87/139], Loss: 0.0941\n",
            "Epoch [8/30], Step [88/139], Loss: 0.0990\n",
            "Epoch [8/30], Step [89/139], Loss: 0.0836\n",
            "Epoch [8/30], Step [90/139], Loss: 0.1020\n",
            "Epoch [8/30], Step [91/139], Loss: 0.1305\n",
            "Epoch [8/30], Step [92/139], Loss: 0.0988\n",
            "Epoch [8/30], Step [93/139], Loss: 0.1015\n",
            "Epoch [8/30], Step [94/139], Loss: 0.0847\n",
            "Epoch [8/30], Step [95/139], Loss: 0.1142\n",
            "Epoch [8/30], Step [96/139], Loss: 0.1119\n",
            "Epoch [8/30], Step [97/139], Loss: 0.1164\n",
            "Epoch [8/30], Step [98/139], Loss: 0.1067\n",
            "Epoch [8/30], Step [99/139], Loss: 0.0968\n",
            "Epoch [8/30], Step [100/139], Loss: 0.1101\n",
            "Epoch [8/30], Step [101/139], Loss: 0.1005\n",
            "Epoch [8/30], Step [102/139], Loss: 0.1097\n",
            "Epoch [8/30], Step [103/139], Loss: 0.0733\n",
            "Epoch [8/30], Step [104/139], Loss: 0.0995\n",
            "Epoch [8/30], Step [105/139], Loss: 0.1035\n",
            "Epoch [8/30], Step [106/139], Loss: 0.1077\n",
            "Epoch [8/30], Step [107/139], Loss: 0.1153\n",
            "Epoch [8/30], Step [108/139], Loss: 0.1011\n",
            "Epoch [8/30], Step [109/139], Loss: 0.1023\n",
            "Epoch [8/30], Step [110/139], Loss: 0.0810\n",
            "Epoch [8/30], Step [111/139], Loss: 0.1170\n",
            "Epoch [8/30], Step [112/139], Loss: 0.1012\n",
            "Epoch [8/30], Step [113/139], Loss: 0.0981\n",
            "Epoch [8/30], Step [114/139], Loss: 0.1166\n",
            "Epoch [8/30], Step [115/139], Loss: 0.0881\n",
            "Epoch [8/30], Step [116/139], Loss: 0.1093\n",
            "Epoch [8/30], Step [117/139], Loss: 0.0720\n",
            "Epoch [8/30], Step [118/139], Loss: 0.0941\n",
            "Epoch [8/30], Step [119/139], Loss: 0.1097\n",
            "Epoch [8/30], Step [120/139], Loss: 0.0881\n",
            "Epoch [8/30], Step [121/139], Loss: 0.1396\n",
            "Epoch [8/30], Step [122/139], Loss: 0.1161\n",
            "Epoch [8/30], Step [123/139], Loss: 0.1020\n",
            "Epoch [8/30], Step [124/139], Loss: 0.0881\n",
            "Epoch [8/30], Step [125/139], Loss: 0.0962\n",
            "Epoch [8/30], Step [126/139], Loss: 0.1219\n",
            "Epoch [8/30], Step [127/139], Loss: 0.1049\n",
            "Epoch [8/30], Step [128/139], Loss: 0.1023\n",
            "Epoch [8/30], Step [129/139], Loss: 0.0981\n",
            "Epoch [8/30], Step [130/139], Loss: 0.1109\n",
            "Epoch [8/30], Step [131/139], Loss: 0.1130\n",
            "Epoch [8/30], Step [132/139], Loss: 0.1168\n",
            "Epoch [8/30], Step [133/139], Loss: 0.1130\n",
            "Epoch [8/30], Step [134/139], Loss: 0.0989\n",
            "Epoch [8/30], Step [135/139], Loss: 0.1211\n",
            "Epoch [8/30], Step [136/139], Loss: 0.1022\n",
            "Epoch [8/30], Step [137/139], Loss: 0.1084\n",
            "Epoch [8/30], Step [138/139], Loss: 0.0844\n",
            "Epoch [8/30], Step [139/139], Loss: 0.1156\n",
            "Start validation #8\n",
            "Validation #8  Average Loss: 0.1179\n",
            "Best performance at epoch: 8\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [9/30], Step [1/139], Loss: 0.1106\n",
            "Epoch [9/30], Step [2/139], Loss: 0.0920\n",
            "Epoch [9/30], Step [3/139], Loss: 0.1202\n",
            "Epoch [9/30], Step [4/139], Loss: 0.0966\n",
            "Epoch [9/30], Step [5/139], Loss: 0.0800\n",
            "Epoch [9/30], Step [6/139], Loss: 0.0891\n",
            "Epoch [9/30], Step [7/139], Loss: 0.1199\n",
            "Epoch [9/30], Step [8/139], Loss: 0.1141\n",
            "Epoch [9/30], Step [9/139], Loss: 0.1049\n",
            "Epoch [9/30], Step [10/139], Loss: 0.1027\n",
            "Epoch [9/30], Step [11/139], Loss: 0.0757\n",
            "Epoch [9/30], Step [12/139], Loss: 0.1015\n",
            "Epoch [9/30], Step [13/139], Loss: 0.0970\n",
            "Epoch [9/30], Step [14/139], Loss: 0.1071\n",
            "Epoch [9/30], Step [15/139], Loss: 0.1109\n",
            "Epoch [9/30], Step [16/139], Loss: 0.1134\n",
            "Epoch [9/30], Step [17/139], Loss: 0.1078\n",
            "Epoch [9/30], Step [18/139], Loss: 0.0921\n",
            "Epoch [9/30], Step [19/139], Loss: 0.1024\n",
            "Epoch [9/30], Step [20/139], Loss: 0.1023\n",
            "Epoch [9/30], Step [21/139], Loss: 0.0955\n",
            "Epoch [9/30], Step [22/139], Loss: 0.0909\n",
            "Epoch [9/30], Step [23/139], Loss: 0.1066\n",
            "Epoch [9/30], Step [24/139], Loss: 0.0990\n",
            "Epoch [9/30], Step [25/139], Loss: 0.1154\n",
            "Epoch [9/30], Step [26/139], Loss: 0.0960\n",
            "Epoch [9/30], Step [27/139], Loss: 0.0692\n",
            "Epoch [9/30], Step [28/139], Loss: 0.1117\n",
            "Epoch [9/30], Step [29/139], Loss: 0.0944\n",
            "Epoch [9/30], Step [30/139], Loss: 0.1016\n",
            "Epoch [9/30], Step [31/139], Loss: 0.1287\n",
            "Epoch [9/30], Step [32/139], Loss: 0.1435\n",
            "Epoch [9/30], Step [33/139], Loss: 0.0897\n",
            "Epoch [9/30], Step [34/139], Loss: 0.1200\n",
            "Epoch [9/30], Step [35/139], Loss: 0.0965\n",
            "Epoch [9/30], Step [36/139], Loss: 0.1090\n",
            "Epoch [9/30], Step [37/139], Loss: 0.0934\n",
            "Epoch [9/30], Step [38/139], Loss: 0.1168\n",
            "Epoch [9/30], Step [39/139], Loss: 0.1045\n",
            "Epoch [9/30], Step [40/139], Loss: 0.0937\n",
            "Epoch [9/30], Step [41/139], Loss: 0.1078\n",
            "Epoch [9/30], Step [42/139], Loss: 0.0853\n",
            "Epoch [9/30], Step [43/139], Loss: 0.0730\n",
            "Epoch [9/30], Step [44/139], Loss: 0.1130\n",
            "Epoch [9/30], Step [45/139], Loss: 0.1214\n",
            "Epoch [9/30], Step [46/139], Loss: 0.0907\n",
            "Epoch [9/30], Step [47/139], Loss: 0.0811\n",
            "Epoch [9/30], Step [48/139], Loss: 0.0906\n",
            "Epoch [9/30], Step [49/139], Loss: 0.1091\n",
            "Epoch [9/30], Step [50/139], Loss: 0.1344\n",
            "Epoch [9/30], Step [51/139], Loss: 0.1183\n",
            "Epoch [9/30], Step [52/139], Loss: 0.0862\n",
            "Epoch [9/30], Step [53/139], Loss: 0.0881\n",
            "Epoch [9/30], Step [54/139], Loss: 0.0750\n",
            "Epoch [9/30], Step [55/139], Loss: 0.1157\n",
            "Epoch [9/30], Step [56/139], Loss: 0.0991\n",
            "Epoch [9/30], Step [57/139], Loss: 0.0978\n",
            "Epoch [9/30], Step [58/139], Loss: 0.0877\n",
            "Epoch [9/30], Step [59/139], Loss: 0.0908\n",
            "Epoch [9/30], Step [60/139], Loss: 0.0859\n",
            "Epoch [9/30], Step [61/139], Loss: 0.0764\n",
            "Epoch [9/30], Step [62/139], Loss: 0.1132\n",
            "Epoch [9/30], Step [63/139], Loss: 0.0988\n",
            "Epoch [9/30], Step [64/139], Loss: 0.0955\n",
            "Epoch [9/30], Step [65/139], Loss: 0.0805\n",
            "Epoch [9/30], Step [66/139], Loss: 0.0834\n",
            "Epoch [9/30], Step [67/139], Loss: 0.1319\n",
            "Epoch [9/30], Step [68/139], Loss: 0.0863\n",
            "Epoch [9/30], Step [69/139], Loss: 0.1007\n",
            "Epoch [9/30], Step [70/139], Loss: 0.0805\n",
            "Epoch [9/30], Step [71/139], Loss: 0.1265\n",
            "Epoch [9/30], Step [72/139], Loss: 0.0713\n",
            "Epoch [9/30], Step [73/139], Loss: 0.1004\n",
            "Epoch [9/30], Step [74/139], Loss: 0.1036\n",
            "Epoch [9/30], Step [75/139], Loss: 0.1070\n",
            "Epoch [9/30], Step [76/139], Loss: 0.1000\n",
            "Epoch [9/30], Step [77/139], Loss: 0.0871\n",
            "Epoch [9/30], Step [78/139], Loss: 0.0884\n",
            "Epoch [9/30], Step [79/139], Loss: 0.1034\n",
            "Epoch [9/30], Step [80/139], Loss: 0.1104\n",
            "Epoch [9/30], Step [81/139], Loss: 0.1004\n",
            "Epoch [9/30], Step [82/139], Loss: 0.1244\n",
            "Epoch [9/30], Step [83/139], Loss: 0.0838\n",
            "Epoch [9/30], Step [84/139], Loss: 0.1010\n",
            "Epoch [9/30], Step [85/139], Loss: 0.1168\n",
            "Epoch [9/30], Step [86/139], Loss: 0.0954\n",
            "Epoch [9/30], Step [87/139], Loss: 0.0900\n",
            "Epoch [9/30], Step [88/139], Loss: 0.1077\n",
            "Epoch [9/30], Step [89/139], Loss: 0.0965\n",
            "Epoch [9/30], Step [90/139], Loss: 0.1027\n",
            "Epoch [9/30], Step [91/139], Loss: 0.1065\n",
            "Epoch [9/30], Step [92/139], Loss: 0.0937\n",
            "Epoch [9/30], Step [93/139], Loss: 0.1265\n",
            "Epoch [9/30], Step [94/139], Loss: 0.0885\n",
            "Epoch [9/30], Step [95/139], Loss: 0.1019\n",
            "Epoch [9/30], Step [96/139], Loss: 0.1122\n",
            "Epoch [9/30], Step [97/139], Loss: 0.1138\n",
            "Epoch [9/30], Step [98/139], Loss: 0.1195\n",
            "Epoch [9/30], Step [99/139], Loss: 0.1114\n",
            "Epoch [9/30], Step [100/139], Loss: 0.1027\n",
            "Epoch [9/30], Step [101/139], Loss: 0.1272\n",
            "Epoch [9/30], Step [102/139], Loss: 0.1225\n",
            "Epoch [9/30], Step [103/139], Loss: 0.1053\n",
            "Epoch [9/30], Step [104/139], Loss: 0.1011\n",
            "Epoch [9/30], Step [105/139], Loss: 0.1100\n",
            "Epoch [9/30], Step [106/139], Loss: 0.0891\n",
            "Epoch [9/30], Step [107/139], Loss: 0.0943\n",
            "Epoch [9/30], Step [108/139], Loss: 0.1245\n",
            "Epoch [9/30], Step [109/139], Loss: 0.1076\n",
            "Epoch [9/30], Step [110/139], Loss: 0.1021\n",
            "Epoch [9/30], Step [111/139], Loss: 0.0940\n",
            "Epoch [9/30], Step [112/139], Loss: 0.1354\n",
            "Epoch [9/30], Step [113/139], Loss: 0.0726\n",
            "Epoch [9/30], Step [114/139], Loss: 0.0885\n",
            "Epoch [9/30], Step [115/139], Loss: 0.1159\n",
            "Epoch [9/30], Step [116/139], Loss: 0.1115\n",
            "Epoch [9/30], Step [117/139], Loss: 0.0840\n",
            "Epoch [9/30], Step [118/139], Loss: 0.0776\n",
            "Epoch [9/30], Step [119/139], Loss: 0.0855\n",
            "Epoch [9/30], Step [120/139], Loss: 0.0848\n",
            "Epoch [9/30], Step [121/139], Loss: 0.1175\n",
            "Epoch [9/30], Step [122/139], Loss: 0.1029\n",
            "Epoch [9/30], Step [123/139], Loss: 0.0969\n",
            "Epoch [9/30], Step [124/139], Loss: 0.0707\n",
            "Epoch [9/30], Step [125/139], Loss: 0.1110\n",
            "Epoch [9/30], Step [126/139], Loss: 0.0863\n",
            "Epoch [9/30], Step [127/139], Loss: 0.1016\n",
            "Epoch [9/30], Step [128/139], Loss: 0.0965\n",
            "Epoch [9/30], Step [129/139], Loss: 0.1028\n",
            "Epoch [9/30], Step [130/139], Loss: 0.0820\n",
            "Epoch [9/30], Step [131/139], Loss: 0.1056\n",
            "Epoch [9/30], Step [132/139], Loss: 0.0784\n",
            "Epoch [9/30], Step [133/139], Loss: 0.0983\n",
            "Epoch [9/30], Step [134/139], Loss: 0.1111\n",
            "Epoch [9/30], Step [135/139], Loss: 0.0890\n",
            "Epoch [9/30], Step [136/139], Loss: 0.1109\n",
            "Epoch [9/30], Step [137/139], Loss: 0.0968\n",
            "Epoch [9/30], Step [138/139], Loss: 0.1304\n",
            "Epoch [9/30], Step [139/139], Loss: 0.0982\n",
            "Start validation #9\n",
            "Validation #9  Average Loss: 0.1127\n",
            "Best performance at epoch: 9\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [10/30], Step [1/139], Loss: 0.1240\n",
            "Epoch [10/30], Step [2/139], Loss: 0.0723\n",
            "Epoch [10/30], Step [3/139], Loss: 0.0919\n",
            "Epoch [10/30], Step [4/139], Loss: 0.0971\n",
            "Epoch [10/30], Step [5/139], Loss: 0.1214\n",
            "Epoch [10/30], Step [6/139], Loss: 0.0741\n",
            "Epoch [10/30], Step [7/139], Loss: 0.0787\n",
            "Epoch [10/30], Step [8/139], Loss: 0.1026\n",
            "Epoch [10/30], Step [9/139], Loss: 0.0999\n",
            "Epoch [10/30], Step [10/139], Loss: 0.1308\n",
            "Epoch [10/30], Step [11/139], Loss: 0.0788\n",
            "Epoch [10/30], Step [12/139], Loss: 0.1113\n",
            "Epoch [10/30], Step [13/139], Loss: 0.1069\n",
            "Epoch [10/30], Step [14/139], Loss: 0.0998\n",
            "Epoch [10/30], Step [15/139], Loss: 0.0850\n",
            "Epoch [10/30], Step [16/139], Loss: 0.1021\n",
            "Epoch [10/30], Step [17/139], Loss: 0.1131\n",
            "Epoch [10/30], Step [18/139], Loss: 0.0944\n",
            "Epoch [10/30], Step [19/139], Loss: 0.0911\n",
            "Epoch [10/30], Step [20/139], Loss: 0.1095\n",
            "Epoch [10/30], Step [21/139], Loss: 0.1206\n",
            "Epoch [10/30], Step [22/139], Loss: 0.1049\n",
            "Epoch [10/30], Step [23/139], Loss: 0.0770\n",
            "Epoch [10/30], Step [24/139], Loss: 0.0684\n",
            "Epoch [10/30], Step [25/139], Loss: 0.1135\n",
            "Epoch [10/30], Step [26/139], Loss: 0.0786\n",
            "Epoch [10/30], Step [27/139], Loss: 0.1069\n",
            "Epoch [10/30], Step [28/139], Loss: 0.1194\n",
            "Epoch [10/30], Step [29/139], Loss: 0.1019\n",
            "Epoch [10/30], Step [30/139], Loss: 0.1204\n",
            "Epoch [10/30], Step [31/139], Loss: 0.1226\n",
            "Epoch [10/30], Step [32/139], Loss: 0.0853\n",
            "Epoch [10/30], Step [33/139], Loss: 0.0881\n",
            "Epoch [10/30], Step [34/139], Loss: 0.0932\n",
            "Epoch [10/30], Step [35/139], Loss: 0.0930\n",
            "Epoch [10/30], Step [36/139], Loss: 0.1170\n",
            "Epoch [10/30], Step [37/139], Loss: 0.1173\n",
            "Epoch [10/30], Step [38/139], Loss: 0.0824\n",
            "Epoch [10/30], Step [39/139], Loss: 0.0591\n",
            "Epoch [10/30], Step [40/139], Loss: 0.1046\n",
            "Epoch [10/30], Step [41/139], Loss: 0.1129\n",
            "Epoch [10/30], Step [42/139], Loss: 0.1014\n",
            "Epoch [10/30], Step [43/139], Loss: 0.1008\n",
            "Epoch [10/30], Step [44/139], Loss: 0.0917\n",
            "Epoch [10/30], Step [45/139], Loss: 0.0957\n",
            "Epoch [10/30], Step [46/139], Loss: 0.1010\n",
            "Epoch [10/30], Step [47/139], Loss: 0.0898\n",
            "Epoch [10/30], Step [48/139], Loss: 0.0700\n",
            "Epoch [10/30], Step [49/139], Loss: 0.0731\n",
            "Epoch [10/30], Step [50/139], Loss: 0.0901\n",
            "Epoch [10/30], Step [51/139], Loss: 0.1037\n",
            "Epoch [10/30], Step [52/139], Loss: 0.1063\n",
            "Epoch [10/30], Step [53/139], Loss: 0.1153\n",
            "Epoch [10/30], Step [54/139], Loss: 0.1119\n",
            "Epoch [10/30], Step [55/139], Loss: 0.0856\n",
            "Epoch [10/30], Step [56/139], Loss: 0.0938\n",
            "Epoch [10/30], Step [57/139], Loss: 0.0976\n",
            "Epoch [10/30], Step [58/139], Loss: 0.1057\n",
            "Epoch [10/30], Step [59/139], Loss: 0.0900\n",
            "Epoch [10/30], Step [60/139], Loss: 0.1246\n",
            "Epoch [10/30], Step [61/139], Loss: 0.0874\n",
            "Epoch [10/30], Step [62/139], Loss: 0.0959\n",
            "Epoch [10/30], Step [63/139], Loss: 0.0871\n",
            "Epoch [10/30], Step [64/139], Loss: 0.0976\n",
            "Epoch [10/30], Step [65/139], Loss: 0.0993\n",
            "Epoch [10/30], Step [66/139], Loss: 0.1074\n",
            "Epoch [10/30], Step [67/139], Loss: 0.1221\n",
            "Epoch [10/30], Step [68/139], Loss: 0.0908\n",
            "Epoch [10/30], Step [69/139], Loss: 0.1444\n",
            "Epoch [10/30], Step [70/139], Loss: 0.1219\n",
            "Epoch [10/30], Step [71/139], Loss: 0.0845\n",
            "Epoch [10/30], Step [72/139], Loss: 0.0700\n",
            "Epoch [10/30], Step [73/139], Loss: 0.0873\n",
            "Epoch [10/30], Step [74/139], Loss: 0.0829\n",
            "Epoch [10/30], Step [75/139], Loss: 0.1210\n",
            "Epoch [10/30], Step [76/139], Loss: 0.0975\n",
            "Epoch [10/30], Step [77/139], Loss: 0.1216\n",
            "Epoch [10/30], Step [78/139], Loss: 0.0858\n",
            "Epoch [10/30], Step [79/139], Loss: 0.0890\n",
            "Epoch [10/30], Step [80/139], Loss: 0.0968\n",
            "Epoch [10/30], Step [81/139], Loss: 0.1017\n",
            "Epoch [10/30], Step [82/139], Loss: 0.0777\n",
            "Epoch [10/30], Step [83/139], Loss: 0.1030\n",
            "Epoch [10/30], Step [84/139], Loss: 0.0993\n",
            "Epoch [10/30], Step [85/139], Loss: 0.1018\n",
            "Epoch [10/30], Step [86/139], Loss: 0.0852\n",
            "Epoch [10/30], Step [87/139], Loss: 0.0739\n",
            "Epoch [10/30], Step [88/139], Loss: 0.1172\n",
            "Epoch [10/30], Step [89/139], Loss: 0.0742\n",
            "Epoch [10/30], Step [90/139], Loss: 0.0975\n",
            "Epoch [10/30], Step [91/139], Loss: 0.0769\n",
            "Epoch [10/30], Step [92/139], Loss: 0.1000\n",
            "Epoch [10/30], Step [93/139], Loss: 0.0862\n",
            "Epoch [10/30], Step [94/139], Loss: 0.0841\n",
            "Epoch [10/30], Step [95/139], Loss: 0.1123\n",
            "Epoch [10/30], Step [96/139], Loss: 0.0831\n",
            "Epoch [10/30], Step [97/139], Loss: 0.1005\n",
            "Epoch [10/30], Step [98/139], Loss: 0.1170\n",
            "Epoch [10/30], Step [99/139], Loss: 0.1108\n",
            "Epoch [10/30], Step [100/139], Loss: 0.0859\n",
            "Epoch [10/30], Step [101/139], Loss: 0.0809\n",
            "Epoch [10/30], Step [102/139], Loss: 0.0860\n",
            "Epoch [10/30], Step [103/139], Loss: 0.0946\n",
            "Epoch [10/30], Step [104/139], Loss: 0.0881\n",
            "Epoch [10/30], Step [105/139], Loss: 0.0835\n",
            "Epoch [10/30], Step [106/139], Loss: 0.1161\n",
            "Epoch [10/30], Step [107/139], Loss: 0.1036\n",
            "Epoch [10/30], Step [108/139], Loss: 0.1010\n",
            "Epoch [10/30], Step [109/139], Loss: 0.0936\n",
            "Epoch [10/30], Step [110/139], Loss: 0.0948\n",
            "Epoch [10/30], Step [111/139], Loss: 0.0818\n",
            "Epoch [10/30], Step [112/139], Loss: 0.0965\n",
            "Epoch [10/30], Step [113/139], Loss: 0.0796\n",
            "Epoch [10/30], Step [114/139], Loss: 0.0800\n",
            "Epoch [10/30], Step [115/139], Loss: 0.1263\n",
            "Epoch [10/30], Step [116/139], Loss: 0.0752\n",
            "Epoch [10/30], Step [117/139], Loss: 0.0768\n",
            "Epoch [10/30], Step [118/139], Loss: 0.0964\n",
            "Epoch [10/30], Step [119/139], Loss: 0.1363\n",
            "Epoch [10/30], Step [120/139], Loss: 0.1267\n",
            "Epoch [10/30], Step [121/139], Loss: 0.0983\n",
            "Epoch [10/30], Step [122/139], Loss: 0.0767\n",
            "Epoch [10/30], Step [123/139], Loss: 0.0709\n",
            "Epoch [10/30], Step [124/139], Loss: 0.0748\n",
            "Epoch [10/30], Step [125/139], Loss: 0.0977\n",
            "Epoch [10/30], Step [126/139], Loss: 0.0891\n",
            "Epoch [10/30], Step [127/139], Loss: 0.1272\n",
            "Epoch [10/30], Step [128/139], Loss: 0.0698\n",
            "Epoch [10/30], Step [129/139], Loss: 0.0899\n",
            "Epoch [10/30], Step [130/139], Loss: 0.1176\n",
            "Epoch [10/30], Step [131/139], Loss: 0.0858\n",
            "Epoch [10/30], Step [132/139], Loss: 0.0855\n",
            "Epoch [10/30], Step [133/139], Loss: 0.1180\n",
            "Epoch [10/30], Step [134/139], Loss: 0.1036\n",
            "Epoch [10/30], Step [135/139], Loss: 0.0970\n",
            "Epoch [10/30], Step [136/139], Loss: 0.0805\n",
            "Epoch [10/30], Step [137/139], Loss: 0.1247\n",
            "Epoch [10/30], Step [138/139], Loss: 0.0977\n",
            "Epoch [10/30], Step [139/139], Loss: 0.1181\n",
            "Start validation #10\n",
            "Validation #10  Average Loss: 0.1088\n",
            "Best performance at epoch: 10\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [11/30], Step [1/139], Loss: 0.1048\n",
            "Epoch [11/30], Step [2/139], Loss: 0.0855\n",
            "Epoch [11/30], Step [3/139], Loss: 0.0912\n",
            "Epoch [11/30], Step [4/139], Loss: 0.0965\n",
            "Epoch [11/30], Step [5/139], Loss: 0.0737\n",
            "Epoch [11/30], Step [6/139], Loss: 0.0738\n",
            "Epoch [11/30], Step [7/139], Loss: 0.0840\n",
            "Epoch [11/30], Step [8/139], Loss: 0.0954\n",
            "Epoch [11/30], Step [9/139], Loss: 0.1065\n",
            "Epoch [11/30], Step [10/139], Loss: 0.1027\n",
            "Epoch [11/30], Step [11/139], Loss: 0.1024\n",
            "Epoch [11/30], Step [12/139], Loss: 0.1027\n",
            "Epoch [11/30], Step [13/139], Loss: 0.0894\n",
            "Epoch [11/30], Step [14/139], Loss: 0.1108\n",
            "Epoch [11/30], Step [15/139], Loss: 0.0707\n",
            "Epoch [11/30], Step [16/139], Loss: 0.0756\n",
            "Epoch [11/30], Step [17/139], Loss: 0.1036\n",
            "Epoch [11/30], Step [18/139], Loss: 0.0835\n",
            "Epoch [11/30], Step [19/139], Loss: 0.1093\n",
            "Epoch [11/30], Step [20/139], Loss: 0.0742\n",
            "Epoch [11/30], Step [21/139], Loss: 0.0792\n",
            "Epoch [11/30], Step [22/139], Loss: 0.1051\n",
            "Epoch [11/30], Step [23/139], Loss: 0.1016\n",
            "Epoch [11/30], Step [24/139], Loss: 0.0976\n",
            "Epoch [11/30], Step [25/139], Loss: 0.0936\n",
            "Epoch [11/30], Step [26/139], Loss: 0.1122\n",
            "Epoch [11/30], Step [27/139], Loss: 0.1530\n",
            "Epoch [11/30], Step [28/139], Loss: 0.1414\n",
            "Epoch [11/30], Step [29/139], Loss: 0.1218\n",
            "Epoch [11/30], Step [30/139], Loss: 0.1017\n",
            "Epoch [11/30], Step [31/139], Loss: 0.0832\n",
            "Epoch [11/30], Step [32/139], Loss: 0.0982\n",
            "Epoch [11/30], Step [33/139], Loss: 0.0923\n",
            "Epoch [11/30], Step [34/139], Loss: 0.1245\n",
            "Epoch [11/30], Step [35/139], Loss: 0.0853\n",
            "Epoch [11/30], Step [36/139], Loss: 0.0733\n",
            "Epoch [11/30], Step [37/139], Loss: 0.1007\n",
            "Epoch [11/30], Step [38/139], Loss: 0.1418\n",
            "Epoch [11/30], Step [39/139], Loss: 0.1212\n",
            "Epoch [11/30], Step [40/139], Loss: 0.0945\n",
            "Epoch [11/30], Step [41/139], Loss: 0.1023\n",
            "Epoch [11/30], Step [42/139], Loss: 0.0690\n",
            "Epoch [11/30], Step [43/139], Loss: 0.1131\n",
            "Epoch [11/30], Step [44/139], Loss: 0.0575\n",
            "Epoch [11/30], Step [45/139], Loss: 0.1023\n",
            "Epoch [11/30], Step [46/139], Loss: 0.0996\n",
            "Epoch [11/30], Step [47/139], Loss: 0.1134\n",
            "Epoch [11/30], Step [48/139], Loss: 0.0988\n",
            "Epoch [11/30], Step [49/139], Loss: 0.0803\n",
            "Epoch [11/30], Step [50/139], Loss: 0.1152\n",
            "Epoch [11/30], Step [51/139], Loss: 0.0883\n",
            "Epoch [11/30], Step [52/139], Loss: 0.0848\n",
            "Epoch [11/30], Step [53/139], Loss: 0.0952\n",
            "Epoch [11/30], Step [54/139], Loss: 0.0964\n",
            "Epoch [11/30], Step [55/139], Loss: 0.0923\n",
            "Epoch [11/30], Step [56/139], Loss: 0.0635\n",
            "Epoch [11/30], Step [57/139], Loss: 0.0680\n",
            "Epoch [11/30], Step [58/139], Loss: 0.0926\n",
            "Epoch [11/30], Step [59/139], Loss: 0.0917\n",
            "Epoch [11/30], Step [60/139], Loss: 0.1050\n",
            "Epoch [11/30], Step [61/139], Loss: 0.1125\n",
            "Epoch [11/30], Step [62/139], Loss: 0.0984\n",
            "Epoch [11/30], Step [63/139], Loss: 0.1041\n",
            "Epoch [11/30], Step [64/139], Loss: 0.0883\n",
            "Epoch [11/30], Step [65/139], Loss: 0.0762\n",
            "Epoch [11/30], Step [66/139], Loss: 0.0862\n",
            "Epoch [11/30], Step [67/139], Loss: 0.0898\n",
            "Epoch [11/30], Step [68/139], Loss: 0.1164\n",
            "Epoch [11/30], Step [69/139], Loss: 0.0970\n",
            "Epoch [11/30], Step [70/139], Loss: 0.1167\n",
            "Epoch [11/30], Step [71/139], Loss: 0.0795\n",
            "Epoch [11/30], Step [72/139], Loss: 0.0777\n",
            "Epoch [11/30], Step [73/139], Loss: 0.1038\n",
            "Epoch [11/30], Step [74/139], Loss: 0.0822\n",
            "Epoch [11/30], Step [75/139], Loss: 0.0975\n",
            "Epoch [11/30], Step [76/139], Loss: 0.1070\n",
            "Epoch [11/30], Step [77/139], Loss: 0.1195\n",
            "Epoch [11/30], Step [78/139], Loss: 0.0952\n",
            "Epoch [11/30], Step [79/139], Loss: 0.1170\n",
            "Epoch [11/30], Step [80/139], Loss: 0.0876\n",
            "Epoch [11/30], Step [81/139], Loss: 0.0889\n",
            "Epoch [11/30], Step [82/139], Loss: 0.1199\n",
            "Epoch [11/30], Step [83/139], Loss: 0.0707\n",
            "Epoch [11/30], Step [84/139], Loss: 0.0747\n",
            "Epoch [11/30], Step [85/139], Loss: 0.0809\n",
            "Epoch [11/30], Step [86/139], Loss: 0.1262\n",
            "Epoch [11/30], Step [87/139], Loss: 0.1386\n",
            "Epoch [11/30], Step [88/139], Loss: 0.0838\n",
            "Epoch [11/30], Step [89/139], Loss: 0.0895\n",
            "Epoch [11/30], Step [90/139], Loss: 0.1085\n",
            "Epoch [11/30], Step [91/139], Loss: 0.0912\n",
            "Epoch [11/30], Step [92/139], Loss: 0.0988\n",
            "Epoch [11/30], Step [93/139], Loss: 0.0818\n",
            "Epoch [11/30], Step [94/139], Loss: 0.0660\n",
            "Epoch [11/30], Step [95/139], Loss: 0.0904\n",
            "Epoch [11/30], Step [96/139], Loss: 0.0918\n",
            "Epoch [11/30], Step [97/139], Loss: 0.0852\n",
            "Epoch [11/30], Step [98/139], Loss: 0.0821\n",
            "Epoch [11/30], Step [99/139], Loss: 0.1128\n",
            "Epoch [11/30], Step [100/139], Loss: 0.0950\n",
            "Epoch [11/30], Step [101/139], Loss: 0.1130\n",
            "Epoch [11/30], Step [102/139], Loss: 0.0573\n",
            "Epoch [11/30], Step [103/139], Loss: 0.0749\n",
            "Epoch [11/30], Step [104/139], Loss: 0.1138\n",
            "Epoch [11/30], Step [105/139], Loss: 0.1007\n",
            "Epoch [11/30], Step [106/139], Loss: 0.0921\n",
            "Epoch [11/30], Step [107/139], Loss: 0.0814\n",
            "Epoch [11/30], Step [108/139], Loss: 0.0968\n",
            "Epoch [11/30], Step [109/139], Loss: 0.0851\n",
            "Epoch [11/30], Step [110/139], Loss: 0.1038\n",
            "Epoch [11/30], Step [111/139], Loss: 0.0783\n",
            "Epoch [11/30], Step [112/139], Loss: 0.0964\n",
            "Epoch [11/30], Step [113/139], Loss: 0.0855\n",
            "Epoch [11/30], Step [114/139], Loss: 0.1079\n",
            "Epoch [11/30], Step [115/139], Loss: 0.1275\n",
            "Epoch [11/30], Step [116/139], Loss: 0.0896\n",
            "Epoch [11/30], Step [117/139], Loss: 0.1072\n",
            "Epoch [11/30], Step [118/139], Loss: 0.0909\n",
            "Epoch [11/30], Step [119/139], Loss: 0.1054\n",
            "Epoch [11/30], Step [120/139], Loss: 0.0865\n",
            "Epoch [11/30], Step [121/139], Loss: 0.0892\n",
            "Epoch [11/30], Step [122/139], Loss: 0.0883\n",
            "Epoch [11/30], Step [123/139], Loss: 0.1019\n",
            "Epoch [11/30], Step [124/139], Loss: 0.1131\n",
            "Epoch [11/30], Step [125/139], Loss: 0.0976\n",
            "Epoch [11/30], Step [126/139], Loss: 0.0800\n",
            "Epoch [11/30], Step [127/139], Loss: 0.0904\n",
            "Epoch [11/30], Step [128/139], Loss: 0.0886\n",
            "Epoch [11/30], Step [129/139], Loss: 0.0563\n",
            "Epoch [11/30], Step [130/139], Loss: 0.0728\n",
            "Epoch [11/30], Step [131/139], Loss: 0.0718\n",
            "Epoch [11/30], Step [132/139], Loss: 0.0800\n",
            "Epoch [11/30], Step [133/139], Loss: 0.0978\n",
            "Epoch [11/30], Step [134/139], Loss: 0.0879\n",
            "Epoch [11/30], Step [135/139], Loss: 0.1013\n",
            "Epoch [11/30], Step [136/139], Loss: 0.0774\n",
            "Epoch [11/30], Step [137/139], Loss: 0.0658\n",
            "Epoch [11/30], Step [138/139], Loss: 0.0753\n",
            "Epoch [11/30], Step [139/139], Loss: 0.0610\n",
            "Start validation #11\n",
            "Validation #11  Average Loss: 0.1050\n",
            "Best performance at epoch: 11\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [12/30], Step [1/139], Loss: 0.0810\n",
            "Epoch [12/30], Step [2/139], Loss: 0.0697\n",
            "Epoch [12/30], Step [3/139], Loss: 0.1092\n",
            "Epoch [12/30], Step [4/139], Loss: 0.0944\n",
            "Epoch [12/30], Step [5/139], Loss: 0.1106\n",
            "Epoch [12/30], Step [6/139], Loss: 0.0604\n",
            "Epoch [12/30], Step [7/139], Loss: 0.1124\n",
            "Epoch [12/30], Step [8/139], Loss: 0.1004\n",
            "Epoch [12/30], Step [9/139], Loss: 0.0757\n",
            "Epoch [12/30], Step [10/139], Loss: 0.0777\n",
            "Epoch [12/30], Step [11/139], Loss: 0.0829\n",
            "Epoch [12/30], Step [12/139], Loss: 0.0980\n",
            "Epoch [12/30], Step [13/139], Loss: 0.1161\n",
            "Epoch [12/30], Step [14/139], Loss: 0.0733\n",
            "Epoch [12/30], Step [15/139], Loss: 0.0907\n",
            "Epoch [12/30], Step [16/139], Loss: 0.0886\n",
            "Epoch [12/30], Step [17/139], Loss: 0.0713\n",
            "Epoch [12/30], Step [18/139], Loss: 0.0786\n",
            "Epoch [12/30], Step [19/139], Loss: 0.0867\n",
            "Epoch [12/30], Step [20/139], Loss: 0.0966\n",
            "Epoch [12/30], Step [21/139], Loss: 0.0966\n",
            "Epoch [12/30], Step [22/139], Loss: 0.0685\n",
            "Epoch [12/30], Step [23/139], Loss: 0.0969\n",
            "Epoch [12/30], Step [24/139], Loss: 0.1030\n",
            "Epoch [12/30], Step [25/139], Loss: 0.0830\n",
            "Epoch [12/30], Step [26/139], Loss: 0.0784\n",
            "Epoch [12/30], Step [27/139], Loss: 0.0973\n",
            "Epoch [12/30], Step [28/139], Loss: 0.1319\n",
            "Epoch [12/30], Step [29/139], Loss: 0.0733\n",
            "Epoch [12/30], Step [30/139], Loss: 0.0719\n",
            "Epoch [12/30], Step [31/139], Loss: 0.0782\n",
            "Epoch [12/30], Step [32/139], Loss: 0.0883\n",
            "Epoch [12/30], Step [33/139], Loss: 0.1040\n",
            "Epoch [12/30], Step [34/139], Loss: 0.1111\n",
            "Epoch [12/30], Step [35/139], Loss: 0.0838\n",
            "Epoch [12/30], Step [36/139], Loss: 0.0998\n",
            "Epoch [12/30], Step [37/139], Loss: 0.0819\n",
            "Epoch [12/30], Step [38/139], Loss: 0.0763\n",
            "Epoch [12/30], Step [39/139], Loss: 0.1089\n",
            "Epoch [12/30], Step [40/139], Loss: 0.0817\n",
            "Epoch [12/30], Step [41/139], Loss: 0.1021\n",
            "Epoch [12/30], Step [42/139], Loss: 0.1051\n",
            "Epoch [12/30], Step [43/139], Loss: 0.0832\n",
            "Epoch [12/30], Step [44/139], Loss: 0.0695\n",
            "Epoch [12/30], Step [45/139], Loss: 0.0853\n",
            "Epoch [12/30], Step [46/139], Loss: 0.1294\n",
            "Epoch [12/30], Step [47/139], Loss: 0.0920\n",
            "Epoch [12/30], Step [48/139], Loss: 0.0813\n",
            "Epoch [12/30], Step [49/139], Loss: 0.1052\n",
            "Epoch [12/30], Step [50/139], Loss: 0.0958\n",
            "Epoch [12/30], Step [51/139], Loss: 0.1138\n",
            "Epoch [12/30], Step [52/139], Loss: 0.1021\n",
            "Epoch [12/30], Step [53/139], Loss: 0.0891\n",
            "Epoch [12/30], Step [54/139], Loss: 0.0839\n",
            "Epoch [12/30], Step [55/139], Loss: 0.0988\n",
            "Epoch [12/30], Step [56/139], Loss: 0.0926\n",
            "Epoch [12/30], Step [57/139], Loss: 0.0913\n",
            "Epoch [12/30], Step [58/139], Loss: 0.1005\n",
            "Epoch [12/30], Step [59/139], Loss: 0.1134\n",
            "Epoch [12/30], Step [60/139], Loss: 0.0967\n",
            "Epoch [12/30], Step [61/139], Loss: 0.1033\n",
            "Epoch [12/30], Step [62/139], Loss: 0.0715\n",
            "Epoch [12/30], Step [63/139], Loss: 0.1082\n",
            "Epoch [12/30], Step [64/139], Loss: 0.0825\n",
            "Epoch [12/30], Step [65/139], Loss: 0.0652\n",
            "Epoch [12/30], Step [66/139], Loss: 0.0734\n",
            "Epoch [12/30], Step [67/139], Loss: 0.1086\n",
            "Epoch [12/30], Step [68/139], Loss: 0.0693\n",
            "Epoch [12/30], Step [69/139], Loss: 0.0701\n",
            "Epoch [12/30], Step [70/139], Loss: 0.0869\n",
            "Epoch [12/30], Step [71/139], Loss: 0.0898\n",
            "Epoch [12/30], Step [72/139], Loss: 0.0971\n",
            "Epoch [12/30], Step [73/139], Loss: 0.0880\n",
            "Epoch [12/30], Step [74/139], Loss: 0.0987\n",
            "Epoch [12/30], Step [75/139], Loss: 0.0838\n",
            "Epoch [12/30], Step [76/139], Loss: 0.0880\n",
            "Epoch [12/30], Step [77/139], Loss: 0.0979\n",
            "Epoch [12/30], Step [78/139], Loss: 0.0997\n",
            "Epoch [12/30], Step [79/139], Loss: 0.1304\n",
            "Epoch [12/30], Step [80/139], Loss: 0.0993\n",
            "Epoch [12/30], Step [81/139], Loss: 0.1102\n",
            "Epoch [12/30], Step [82/139], Loss: 0.0908\n",
            "Epoch [12/30], Step [83/139], Loss: 0.1040\n",
            "Epoch [12/30], Step [84/139], Loss: 0.0855\n",
            "Epoch [12/30], Step [85/139], Loss: 0.0691\n",
            "Epoch [12/30], Step [86/139], Loss: 0.0910\n",
            "Epoch [12/30], Step [87/139], Loss: 0.0941\n",
            "Epoch [12/30], Step [88/139], Loss: 0.0751\n",
            "Epoch [12/30], Step [89/139], Loss: 0.1119\n",
            "Epoch [12/30], Step [90/139], Loss: 0.0965\n",
            "Epoch [12/30], Step [91/139], Loss: 0.1155\n",
            "Epoch [12/30], Step [92/139], Loss: 0.0995\n",
            "Epoch [12/30], Step [93/139], Loss: 0.1219\n",
            "Epoch [12/30], Step [94/139], Loss: 0.1023\n",
            "Epoch [12/30], Step [95/139], Loss: 0.0833\n",
            "Epoch [12/30], Step [96/139], Loss: 0.0853\n",
            "Epoch [12/30], Step [97/139], Loss: 0.0826\n",
            "Epoch [12/30], Step [98/139], Loss: 0.0564\n",
            "Epoch [12/30], Step [99/139], Loss: 0.1122\n",
            "Epoch [12/30], Step [100/139], Loss: 0.1112\n",
            "Epoch [12/30], Step [101/139], Loss: 0.0958\n",
            "Epoch [12/30], Step [102/139], Loss: 0.0850\n",
            "Epoch [12/30], Step [103/139], Loss: 0.0847\n",
            "Epoch [12/30], Step [104/139], Loss: 0.0957\n",
            "Epoch [12/30], Step [105/139], Loss: 0.0904\n",
            "Epoch [12/30], Step [106/139], Loss: 0.0942\n",
            "Epoch [12/30], Step [107/139], Loss: 0.0976\n",
            "Epoch [12/30], Step [108/139], Loss: 0.0860\n",
            "Epoch [12/30], Step [109/139], Loss: 0.1005\n",
            "Epoch [12/30], Step [110/139], Loss: 0.0722\n",
            "Epoch [12/30], Step [111/139], Loss: 0.0789\n",
            "Epoch [12/30], Step [112/139], Loss: 0.0920\n",
            "Epoch [12/30], Step [113/139], Loss: 0.0920\n",
            "Epoch [12/30], Step [114/139], Loss: 0.1247\n",
            "Epoch [12/30], Step [115/139], Loss: 0.0948\n",
            "Epoch [12/30], Step [116/139], Loss: 0.1085\n",
            "Epoch [12/30], Step [117/139], Loss: 0.0630\n",
            "Epoch [12/30], Step [118/139], Loss: 0.0770\n",
            "Epoch [12/30], Step [119/139], Loss: 0.0701\n",
            "Epoch [12/30], Step [120/139], Loss: 0.0822\n",
            "Epoch [12/30], Step [121/139], Loss: 0.0891\n",
            "Epoch [12/30], Step [122/139], Loss: 0.0921\n",
            "Epoch [12/30], Step [123/139], Loss: 0.0800\n",
            "Epoch [12/30], Step [124/139], Loss: 0.0974\n",
            "Epoch [12/30], Step [125/139], Loss: 0.1007\n",
            "Epoch [12/30], Step [126/139], Loss: 0.0997\n",
            "Epoch [12/30], Step [127/139], Loss: 0.0889\n",
            "Epoch [12/30], Step [128/139], Loss: 0.0867\n",
            "Epoch [12/30], Step [129/139], Loss: 0.0958\n",
            "Epoch [12/30], Step [130/139], Loss: 0.0873\n",
            "Epoch [12/30], Step [131/139], Loss: 0.0682\n",
            "Epoch [12/30], Step [132/139], Loss: 0.0982\n",
            "Epoch [12/30], Step [133/139], Loss: 0.0754\n",
            "Epoch [12/30], Step [134/139], Loss: 0.0931\n",
            "Epoch [12/30], Step [135/139], Loss: 0.0919\n",
            "Epoch [12/30], Step [136/139], Loss: 0.1016\n",
            "Epoch [12/30], Step [137/139], Loss: 0.0765\n",
            "Epoch [12/30], Step [138/139], Loss: 0.1402\n",
            "Epoch [12/30], Step [139/139], Loss: 0.0720\n",
            "Start validation #12\n",
            "Validation #12  Average Loss: 0.1016\n",
            "Best performance at epoch: 12\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [13/30], Step [1/139], Loss: 0.0740\n",
            "Epoch [13/30], Step [2/139], Loss: 0.0950\n",
            "Epoch [13/30], Step [3/139], Loss: 0.0805\n",
            "Epoch [13/30], Step [4/139], Loss: 0.1047\n",
            "Epoch [13/30], Step [5/139], Loss: 0.0881\n",
            "Epoch [13/30], Step [6/139], Loss: 0.1008\n",
            "Epoch [13/30], Step [7/139], Loss: 0.0820\n",
            "Epoch [13/30], Step [8/139], Loss: 0.0924\n",
            "Epoch [13/30], Step [9/139], Loss: 0.0651\n",
            "Epoch [13/30], Step [10/139], Loss: 0.0684\n",
            "Epoch [13/30], Step [11/139], Loss: 0.1017\n",
            "Epoch [13/30], Step [12/139], Loss: 0.1128\n",
            "Epoch [13/30], Step [13/139], Loss: 0.0962\n",
            "Epoch [13/30], Step [14/139], Loss: 0.0946\n",
            "Epoch [13/30], Step [15/139], Loss: 0.1044\n",
            "Epoch [13/30], Step [16/139], Loss: 0.0767\n",
            "Epoch [13/30], Step [17/139], Loss: 0.0710\n",
            "Epoch [13/30], Step [18/139], Loss: 0.0698\n",
            "Epoch [13/30], Step [19/139], Loss: 0.1014\n",
            "Epoch [13/30], Step [20/139], Loss: 0.1126\n",
            "Epoch [13/30], Step [21/139], Loss: 0.0770\n",
            "Epoch [13/30], Step [22/139], Loss: 0.1085\n",
            "Epoch [13/30], Step [23/139], Loss: 0.0905\n",
            "Epoch [13/30], Step [24/139], Loss: 0.0731\n",
            "Epoch [13/30], Step [25/139], Loss: 0.0870\n",
            "Epoch [13/30], Step [26/139], Loss: 0.0571\n",
            "Epoch [13/30], Step [27/139], Loss: 0.0855\n",
            "Epoch [13/30], Step [28/139], Loss: 0.0818\n",
            "Epoch [13/30], Step [29/139], Loss: 0.0900\n",
            "Epoch [13/30], Step [30/139], Loss: 0.1010\n",
            "Epoch [13/30], Step [31/139], Loss: 0.1036\n",
            "Epoch [13/30], Step [32/139], Loss: 0.0772\n",
            "Epoch [13/30], Step [33/139], Loss: 0.0819\n",
            "Epoch [13/30], Step [34/139], Loss: 0.1057\n",
            "Epoch [13/30], Step [35/139], Loss: 0.1182\n",
            "Epoch [13/30], Step [36/139], Loss: 0.1106\n",
            "Epoch [13/30], Step [37/139], Loss: 0.0880\n",
            "Epoch [13/30], Step [38/139], Loss: 0.1021\n",
            "Epoch [13/30], Step [39/139], Loss: 0.1141\n",
            "Epoch [13/30], Step [40/139], Loss: 0.0804\n",
            "Epoch [13/30], Step [41/139], Loss: 0.0775\n",
            "Epoch [13/30], Step [42/139], Loss: 0.1029\n",
            "Epoch [13/30], Step [43/139], Loss: 0.1187\n",
            "Epoch [13/30], Step [44/139], Loss: 0.0694\n",
            "Epoch [13/30], Step [45/139], Loss: 0.0960\n",
            "Epoch [13/30], Step [46/139], Loss: 0.0763\n",
            "Epoch [13/30], Step [47/139], Loss: 0.0761\n",
            "Epoch [13/30], Step [48/139], Loss: 0.1011\n",
            "Epoch [13/30], Step [49/139], Loss: 0.1180\n",
            "Epoch [13/30], Step [50/139], Loss: 0.1184\n",
            "Epoch [13/30], Step [51/139], Loss: 0.0684\n",
            "Epoch [13/30], Step [52/139], Loss: 0.0806\n",
            "Epoch [13/30], Step [53/139], Loss: 0.0865\n",
            "Epoch [13/30], Step [54/139], Loss: 0.1077\n",
            "Epoch [13/30], Step [55/139], Loss: 0.0927\n",
            "Epoch [13/30], Step [56/139], Loss: 0.0842\n",
            "Epoch [13/30], Step [57/139], Loss: 0.1121\n",
            "Epoch [13/30], Step [58/139], Loss: 0.0668\n",
            "Epoch [13/30], Step [59/139], Loss: 0.0906\n",
            "Epoch [13/30], Step [60/139], Loss: 0.0997\n",
            "Epoch [13/30], Step [61/139], Loss: 0.0869\n",
            "Epoch [13/30], Step [62/139], Loss: 0.0835\n",
            "Epoch [13/30], Step [63/139], Loss: 0.1195\n",
            "Epoch [13/30], Step [64/139], Loss: 0.0805\n",
            "Epoch [13/30], Step [65/139], Loss: 0.0894\n",
            "Epoch [13/30], Step [66/139], Loss: 0.1058\n",
            "Epoch [13/30], Step [67/139], Loss: 0.1033\n",
            "Epoch [13/30], Step [68/139], Loss: 0.0703\n",
            "Epoch [13/30], Step [69/139], Loss: 0.1114\n",
            "Epoch [13/30], Step [70/139], Loss: 0.0713\n",
            "Epoch [13/30], Step [71/139], Loss: 0.0773\n",
            "Epoch [13/30], Step [72/139], Loss: 0.0889\n",
            "Epoch [13/30], Step [73/139], Loss: 0.0903\n",
            "Epoch [13/30], Step [74/139], Loss: 0.0884\n",
            "Epoch [13/30], Step [75/139], Loss: 0.0868\n",
            "Epoch [13/30], Step [76/139], Loss: 0.0864\n",
            "Epoch [13/30], Step [77/139], Loss: 0.0602\n",
            "Epoch [13/30], Step [78/139], Loss: 0.0998\n",
            "Epoch [13/30], Step [79/139], Loss: 0.0801\n",
            "Epoch [13/30], Step [80/139], Loss: 0.0833\n",
            "Epoch [13/30], Step [81/139], Loss: 0.0808\n",
            "Epoch [13/30], Step [82/139], Loss: 0.0950\n",
            "Epoch [13/30], Step [83/139], Loss: 0.0982\n",
            "Epoch [13/30], Step [84/139], Loss: 0.0726\n",
            "Epoch [13/30], Step [85/139], Loss: 0.1005\n",
            "Epoch [13/30], Step [86/139], Loss: 0.1143\n",
            "Epoch [13/30], Step [87/139], Loss: 0.0786\n",
            "Epoch [13/30], Step [88/139], Loss: 0.0923\n",
            "Epoch [13/30], Step [89/139], Loss: 0.0815\n",
            "Epoch [13/30], Step [90/139], Loss: 0.1128\n",
            "Epoch [13/30], Step [91/139], Loss: 0.1050\n",
            "Epoch [13/30], Step [92/139], Loss: 0.0693\n",
            "Epoch [13/30], Step [93/139], Loss: 0.0891\n",
            "Epoch [13/30], Step [94/139], Loss: 0.0966\n",
            "Epoch [13/30], Step [95/139], Loss: 0.0776\n",
            "Epoch [13/30], Step [96/139], Loss: 0.0950\n",
            "Epoch [13/30], Step [97/139], Loss: 0.0471\n",
            "Epoch [13/30], Step [98/139], Loss: 0.0777\n",
            "Epoch [13/30], Step [99/139], Loss: 0.0800\n",
            "Epoch [13/30], Step [100/139], Loss: 0.0742\n",
            "Epoch [13/30], Step [101/139], Loss: 0.0614\n",
            "Epoch [13/30], Step [102/139], Loss: 0.1094\n",
            "Epoch [13/30], Step [103/139], Loss: 0.1019\n",
            "Epoch [13/30], Step [104/139], Loss: 0.0633\n",
            "Epoch [13/30], Step [105/139], Loss: 0.0897\n",
            "Epoch [13/30], Step [106/139], Loss: 0.0830\n",
            "Epoch [13/30], Step [107/139], Loss: 0.0928\n",
            "Epoch [13/30], Step [108/139], Loss: 0.0778\n",
            "Epoch [13/30], Step [109/139], Loss: 0.0690\n",
            "Epoch [13/30], Step [110/139], Loss: 0.0779\n",
            "Epoch [13/30], Step [111/139], Loss: 0.1023\n",
            "Epoch [13/30], Step [112/139], Loss: 0.0947\n",
            "Epoch [13/30], Step [113/139], Loss: 0.0915\n",
            "Epoch [13/30], Step [114/139], Loss: 0.0812\n",
            "Epoch [13/30], Step [115/139], Loss: 0.0898\n",
            "Epoch [13/30], Step [116/139], Loss: 0.0896\n",
            "Epoch [13/30], Step [117/139], Loss: 0.0738\n",
            "Epoch [13/30], Step [118/139], Loss: 0.0749\n",
            "Epoch [13/30], Step [119/139], Loss: 0.0784\n",
            "Epoch [13/30], Step [120/139], Loss: 0.1065\n",
            "Epoch [13/30], Step [121/139], Loss: 0.1106\n",
            "Epoch [13/30], Step [122/139], Loss: 0.1074\n",
            "Epoch [13/30], Step [123/139], Loss: 0.0621\n",
            "Epoch [13/30], Step [124/139], Loss: 0.0887\n",
            "Epoch [13/30], Step [125/139], Loss: 0.0698\n",
            "Epoch [13/30], Step [126/139], Loss: 0.0802\n",
            "Epoch [13/30], Step [127/139], Loss: 0.0780\n",
            "Epoch [13/30], Step [128/139], Loss: 0.0971\n",
            "Epoch [13/30], Step [129/139], Loss: 0.0899\n",
            "Epoch [13/30], Step [130/139], Loss: 0.1220\n",
            "Epoch [13/30], Step [131/139], Loss: 0.1198\n",
            "Epoch [13/30], Step [132/139], Loss: 0.0639\n",
            "Epoch [13/30], Step [133/139], Loss: 0.0731\n",
            "Epoch [13/30], Step [134/139], Loss: 0.1061\n",
            "Epoch [13/30], Step [135/139], Loss: 0.0889\n",
            "Epoch [13/30], Step [136/139], Loss: 0.0742\n",
            "Epoch [13/30], Step [137/139], Loss: 0.1226\n",
            "Epoch [13/30], Step [138/139], Loss: 0.0815\n",
            "Epoch [13/30], Step [139/139], Loss: 0.1161\n",
            "Start validation #13\n",
            "Validation #13  Average Loss: 0.0986\n",
            "Best performance at epoch: 13\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [14/30], Step [1/139], Loss: 0.0910\n",
            "Epoch [14/30], Step [2/139], Loss: 0.0831\n",
            "Epoch [14/30], Step [3/139], Loss: 0.1045\n",
            "Epoch [14/30], Step [4/139], Loss: 0.0735\n",
            "Epoch [14/30], Step [5/139], Loss: 0.0943\n",
            "Epoch [14/30], Step [6/139], Loss: 0.0894\n",
            "Epoch [14/30], Step [7/139], Loss: 0.0733\n",
            "Epoch [14/30], Step [8/139], Loss: 0.1039\n",
            "Epoch [14/30], Step [9/139], Loss: 0.0670\n",
            "Epoch [14/30], Step [10/139], Loss: 0.1067\n",
            "Epoch [14/30], Step [11/139], Loss: 0.1018\n",
            "Epoch [14/30], Step [12/139], Loss: 0.0767\n",
            "Epoch [14/30], Step [13/139], Loss: 0.0690\n",
            "Epoch [14/30], Step [14/139], Loss: 0.1004\n",
            "Epoch [14/30], Step [15/139], Loss: 0.0931\n",
            "Epoch [14/30], Step [16/139], Loss: 0.1181\n",
            "Epoch [14/30], Step [17/139], Loss: 0.0755\n",
            "Epoch [14/30], Step [18/139], Loss: 0.1143\n",
            "Epoch [14/30], Step [19/139], Loss: 0.0863\n",
            "Epoch [14/30], Step [20/139], Loss: 0.0842\n",
            "Epoch [14/30], Step [21/139], Loss: 0.1137\n",
            "Epoch [14/30], Step [22/139], Loss: 0.0956\n",
            "Epoch [14/30], Step [23/139], Loss: 0.1358\n",
            "Epoch [14/30], Step [24/139], Loss: 0.1102\n",
            "Epoch [14/30], Step [25/139], Loss: 0.0759\n",
            "Epoch [14/30], Step [26/139], Loss: 0.0839\n",
            "Epoch [14/30], Step [27/139], Loss: 0.0947\n",
            "Epoch [14/30], Step [28/139], Loss: 0.0638\n",
            "Epoch [14/30], Step [29/139], Loss: 0.0858\n",
            "Epoch [14/30], Step [30/139], Loss: 0.0762\n",
            "Epoch [14/30], Step [31/139], Loss: 0.0828\n",
            "Epoch [14/30], Step [32/139], Loss: 0.0730\n",
            "Epoch [14/30], Step [33/139], Loss: 0.0716\n",
            "Epoch [14/30], Step [34/139], Loss: 0.0838\n",
            "Epoch [14/30], Step [35/139], Loss: 0.0752\n",
            "Epoch [14/30], Step [36/139], Loss: 0.1022\n",
            "Epoch [14/30], Step [37/139], Loss: 0.0910\n",
            "Epoch [14/30], Step [38/139], Loss: 0.0959\n",
            "Epoch [14/30], Step [39/139], Loss: 0.1099\n",
            "Epoch [14/30], Step [40/139], Loss: 0.0979\n",
            "Epoch [14/30], Step [41/139], Loss: 0.0704\n",
            "Epoch [14/30], Step [42/139], Loss: 0.1021\n",
            "Epoch [14/30], Step [43/139], Loss: 0.0857\n",
            "Epoch [14/30], Step [44/139], Loss: 0.1100\n",
            "Epoch [14/30], Step [45/139], Loss: 0.0752\n",
            "Epoch [14/30], Step [46/139], Loss: 0.1040\n",
            "Epoch [14/30], Step [47/139], Loss: 0.0706\n",
            "Epoch [14/30], Step [48/139], Loss: 0.0966\n",
            "Epoch [14/30], Step [49/139], Loss: 0.1105\n",
            "Epoch [14/30], Step [50/139], Loss: 0.0702\n",
            "Epoch [14/30], Step [51/139], Loss: 0.0853\n",
            "Epoch [14/30], Step [52/139], Loss: 0.0727\n",
            "Epoch [14/30], Step [53/139], Loss: 0.0803\n",
            "Epoch [14/30], Step [54/139], Loss: 0.1027\n",
            "Epoch [14/30], Step [55/139], Loss: 0.0573\n",
            "Epoch [14/30], Step [56/139], Loss: 0.0607\n",
            "Epoch [14/30], Step [57/139], Loss: 0.0970\n",
            "Epoch [14/30], Step [58/139], Loss: 0.0732\n",
            "Epoch [14/30], Step [59/139], Loss: 0.1024\n",
            "Epoch [14/30], Step [60/139], Loss: 0.0948\n",
            "Epoch [14/30], Step [61/139], Loss: 0.1021\n",
            "Epoch [14/30], Step [62/139], Loss: 0.0750\n",
            "Epoch [14/30], Step [63/139], Loss: 0.1324\n",
            "Epoch [14/30], Step [64/139], Loss: 0.0882\n",
            "Epoch [14/30], Step [65/139], Loss: 0.0831\n",
            "Epoch [14/30], Step [66/139], Loss: 0.0924\n",
            "Epoch [14/30], Step [67/139], Loss: 0.0951\n",
            "Epoch [14/30], Step [68/139], Loss: 0.0920\n",
            "Epoch [14/30], Step [69/139], Loss: 0.0820\n",
            "Epoch [14/30], Step [70/139], Loss: 0.1125\n",
            "Epoch [14/30], Step [71/139], Loss: 0.1018\n",
            "Epoch [14/30], Step [72/139], Loss: 0.0714\n",
            "Epoch [14/30], Step [73/139], Loss: 0.1171\n",
            "Epoch [14/30], Step [74/139], Loss: 0.0744\n",
            "Epoch [14/30], Step [75/139], Loss: 0.1018\n",
            "Epoch [14/30], Step [76/139], Loss: 0.0933\n",
            "Epoch [14/30], Step [77/139], Loss: 0.0702\n",
            "Epoch [14/30], Step [78/139], Loss: 0.0589\n",
            "Epoch [14/30], Step [79/139], Loss: 0.0834\n",
            "Epoch [14/30], Step [80/139], Loss: 0.0611\n",
            "Epoch [14/30], Step [81/139], Loss: 0.0859\n",
            "Epoch [14/30], Step [82/139], Loss: 0.0750\n",
            "Epoch [14/30], Step [83/139], Loss: 0.0609\n",
            "Epoch [14/30], Step [84/139], Loss: 0.1092\n",
            "Epoch [14/30], Step [85/139], Loss: 0.1042\n",
            "Epoch [14/30], Step [86/139], Loss: 0.0672\n",
            "Epoch [14/30], Step [87/139], Loss: 0.0719\n",
            "Epoch [14/30], Step [88/139], Loss: 0.0857\n",
            "Epoch [14/30], Step [89/139], Loss: 0.0828\n",
            "Epoch [14/30], Step [90/139], Loss: 0.0742\n",
            "Epoch [14/30], Step [91/139], Loss: 0.0843\n",
            "Epoch [14/30], Step [92/139], Loss: 0.0775\n",
            "Epoch [14/30], Step [93/139], Loss: 0.1322\n",
            "Epoch [14/30], Step [94/139], Loss: 0.1006\n",
            "Epoch [14/30], Step [95/139], Loss: 0.1239\n",
            "Epoch [14/30], Step [96/139], Loss: 0.0845\n",
            "Epoch [14/30], Step [97/139], Loss: 0.0960\n",
            "Epoch [14/30], Step [98/139], Loss: 0.0789\n",
            "Epoch [14/30], Step [99/139], Loss: 0.0843\n",
            "Epoch [14/30], Step [100/139], Loss: 0.0955\n",
            "Epoch [14/30], Step [101/139], Loss: 0.0781\n",
            "Epoch [14/30], Step [102/139], Loss: 0.0668\n",
            "Epoch [14/30], Step [103/139], Loss: 0.0863\n",
            "Epoch [14/30], Step [104/139], Loss: 0.0770\n",
            "Epoch [14/30], Step [105/139], Loss: 0.0638\n",
            "Epoch [14/30], Step [106/139], Loss: 0.0603\n",
            "Epoch [14/30], Step [107/139], Loss: 0.0740\n",
            "Epoch [14/30], Step [108/139], Loss: 0.0690\n",
            "Epoch [14/30], Step [109/139], Loss: 0.0878\n",
            "Epoch [14/30], Step [110/139], Loss: 0.0796\n",
            "Epoch [14/30], Step [111/139], Loss: 0.0816\n",
            "Epoch [14/30], Step [112/139], Loss: 0.0962\n",
            "Epoch [14/30], Step [113/139], Loss: 0.0884\n",
            "Epoch [14/30], Step [114/139], Loss: 0.0815\n",
            "Epoch [14/30], Step [115/139], Loss: 0.1056\n",
            "Epoch [14/30], Step [116/139], Loss: 0.0706\n",
            "Epoch [14/30], Step [117/139], Loss: 0.0772\n",
            "Epoch [14/30], Step [118/139], Loss: 0.0742\n",
            "Epoch [14/30], Step [119/139], Loss: 0.0699\n",
            "Epoch [14/30], Step [120/139], Loss: 0.0744\n",
            "Epoch [14/30], Step [121/139], Loss: 0.1100\n",
            "Epoch [14/30], Step [122/139], Loss: 0.0956\n",
            "Epoch [14/30], Step [123/139], Loss: 0.0699\n",
            "Epoch [14/30], Step [124/139], Loss: 0.0885\n",
            "Epoch [14/30], Step [125/139], Loss: 0.0820\n",
            "Epoch [14/30], Step [126/139], Loss: 0.0751\n",
            "Epoch [14/30], Step [127/139], Loss: 0.0807\n",
            "Epoch [14/30], Step [128/139], Loss: 0.0906\n",
            "Epoch [14/30], Step [129/139], Loss: 0.0978\n",
            "Epoch [14/30], Step [130/139], Loss: 0.0666\n",
            "Epoch [14/30], Step [131/139], Loss: 0.0620\n",
            "Epoch [14/30], Step [132/139], Loss: 0.0977\n",
            "Epoch [14/30], Step [133/139], Loss: 0.0923\n",
            "Epoch [14/30], Step [134/139], Loss: 0.0772\n",
            "Epoch [14/30], Step [135/139], Loss: 0.1042\n",
            "Epoch [14/30], Step [136/139], Loss: 0.0986\n",
            "Epoch [14/30], Step [137/139], Loss: 0.0755\n",
            "Epoch [14/30], Step [138/139], Loss: 0.1290\n",
            "Epoch [14/30], Step [139/139], Loss: 0.0793\n",
            "Start validation #14\n",
            "Validation #14  Average Loss: 0.0963\n",
            "Best performance at epoch: 14\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [15/30], Step [1/139], Loss: 0.0570\n",
            "Epoch [15/30], Step [2/139], Loss: 0.0924\n",
            "Epoch [15/30], Step [3/139], Loss: 0.0821\n",
            "Epoch [15/30], Step [4/139], Loss: 0.0827\n",
            "Epoch [15/30], Step [5/139], Loss: 0.0833\n",
            "Epoch [15/30], Step [6/139], Loss: 0.0821\n",
            "Epoch [15/30], Step [7/139], Loss: 0.0646\n",
            "Epoch [15/30], Step [8/139], Loss: 0.0867\n",
            "Epoch [15/30], Step [9/139], Loss: 0.0654\n",
            "Epoch [15/30], Step [10/139], Loss: 0.0749\n",
            "Epoch [15/30], Step [11/139], Loss: 0.0919\n",
            "Epoch [15/30], Step [12/139], Loss: 0.0887\n",
            "Epoch [15/30], Step [13/139], Loss: 0.0761\n",
            "Epoch [15/30], Step [14/139], Loss: 0.0834\n",
            "Epoch [15/30], Step [15/139], Loss: 0.1030\n",
            "Epoch [15/30], Step [16/139], Loss: 0.0927\n",
            "Epoch [15/30], Step [17/139], Loss: 0.0937\n",
            "Epoch [15/30], Step [18/139], Loss: 0.0634\n",
            "Epoch [15/30], Step [19/139], Loss: 0.0713\n",
            "Epoch [15/30], Step [20/139], Loss: 0.1015\n",
            "Epoch [15/30], Step [21/139], Loss: 0.0974\n",
            "Epoch [15/30], Step [22/139], Loss: 0.0802\n",
            "Epoch [15/30], Step [23/139], Loss: 0.0789\n",
            "Epoch [15/30], Step [24/139], Loss: 0.0871\n",
            "Epoch [15/30], Step [25/139], Loss: 0.0845\n",
            "Epoch [15/30], Step [26/139], Loss: 0.0883\n",
            "Epoch [15/30], Step [27/139], Loss: 0.0671\n",
            "Epoch [15/30], Step [28/139], Loss: 0.0832\n",
            "Epoch [15/30], Step [29/139], Loss: 0.0841\n",
            "Epoch [15/30], Step [30/139], Loss: 0.0862\n",
            "Epoch [15/30], Step [31/139], Loss: 0.0939\n",
            "Epoch [15/30], Step [32/139], Loss: 0.0842\n",
            "Epoch [15/30], Step [33/139], Loss: 0.1096\n",
            "Epoch [15/30], Step [34/139], Loss: 0.1399\n",
            "Epoch [15/30], Step [35/139], Loss: 0.0850\n",
            "Epoch [15/30], Step [36/139], Loss: 0.0830\n",
            "Epoch [15/30], Step [37/139], Loss: 0.0969\n",
            "Epoch [15/30], Step [38/139], Loss: 0.0893\n",
            "Epoch [15/30], Step [39/139], Loss: 0.0776\n",
            "Epoch [15/30], Step [40/139], Loss: 0.0603\n",
            "Epoch [15/30], Step [41/139], Loss: 0.0821\n",
            "Epoch [15/30], Step [42/139], Loss: 0.0790\n",
            "Epoch [15/30], Step [43/139], Loss: 0.1153\n",
            "Epoch [15/30], Step [44/139], Loss: 0.0648\n",
            "Epoch [15/30], Step [45/139], Loss: 0.0779\n",
            "Epoch [15/30], Step [46/139], Loss: 0.0727\n",
            "Epoch [15/30], Step [47/139], Loss: 0.0814\n",
            "Epoch [15/30], Step [48/139], Loss: 0.0935\n",
            "Epoch [15/30], Step [49/139], Loss: 0.0788\n",
            "Epoch [15/30], Step [50/139], Loss: 0.0713\n",
            "Epoch [15/30], Step [51/139], Loss: 0.1018\n",
            "Epoch [15/30], Step [52/139], Loss: 0.0875\n",
            "Epoch [15/30], Step [53/139], Loss: 0.0834\n",
            "Epoch [15/30], Step [54/139], Loss: 0.0975\n",
            "Epoch [15/30], Step [55/139], Loss: 0.0602\n",
            "Epoch [15/30], Step [56/139], Loss: 0.0589\n",
            "Epoch [15/30], Step [57/139], Loss: 0.1209\n",
            "Epoch [15/30], Step [58/139], Loss: 0.0721\n",
            "Epoch [15/30], Step [59/139], Loss: 0.0965\n",
            "Epoch [15/30], Step [60/139], Loss: 0.0961\n",
            "Epoch [15/30], Step [61/139], Loss: 0.0587\n",
            "Epoch [15/30], Step [62/139], Loss: 0.0739\n",
            "Epoch [15/30], Step [63/139], Loss: 0.0720\n",
            "Epoch [15/30], Step [64/139], Loss: 0.1070\n",
            "Epoch [15/30], Step [65/139], Loss: 0.0889\n",
            "Epoch [15/30], Step [66/139], Loss: 0.1123\n",
            "Epoch [15/30], Step [67/139], Loss: 0.0816\n",
            "Epoch [15/30], Step [68/139], Loss: 0.0660\n",
            "Epoch [15/30], Step [69/139], Loss: 0.0915\n",
            "Epoch [15/30], Step [70/139], Loss: 0.0905\n",
            "Epoch [15/30], Step [71/139], Loss: 0.1058\n",
            "Epoch [15/30], Step [72/139], Loss: 0.0775\n",
            "Epoch [15/30], Step [73/139], Loss: 0.1057\n",
            "Epoch [15/30], Step [74/139], Loss: 0.0721\n",
            "Epoch [15/30], Step [75/139], Loss: 0.0715\n",
            "Epoch [15/30], Step [76/139], Loss: 0.0846\n",
            "Epoch [15/30], Step [77/139], Loss: 0.1015\n",
            "Epoch [15/30], Step [78/139], Loss: 0.0857\n",
            "Epoch [15/30], Step [79/139], Loss: 0.0647\n",
            "Epoch [15/30], Step [80/139], Loss: 0.0781\n",
            "Epoch [15/30], Step [81/139], Loss: 0.1130\n",
            "Epoch [15/30], Step [82/139], Loss: 0.0769\n",
            "Epoch [15/30], Step [83/139], Loss: 0.1034\n",
            "Epoch [15/30], Step [84/139], Loss: 0.0991\n",
            "Epoch [15/30], Step [85/139], Loss: 0.0905\n",
            "Epoch [15/30], Step [86/139], Loss: 0.0819\n",
            "Epoch [15/30], Step [87/139], Loss: 0.0961\n",
            "Epoch [15/30], Step [88/139], Loss: 0.0928\n",
            "Epoch [15/30], Step [89/139], Loss: 0.0804\n",
            "Epoch [15/30], Step [90/139], Loss: 0.0916\n",
            "Epoch [15/30], Step [91/139], Loss: 0.0772\n",
            "Epoch [15/30], Step [92/139], Loss: 0.0978\n",
            "Epoch [15/30], Step [93/139], Loss: 0.0891\n",
            "Epoch [15/30], Step [94/139], Loss: 0.0768\n",
            "Epoch [15/30], Step [95/139], Loss: 0.0948\n",
            "Epoch [15/30], Step [96/139], Loss: 0.1127\n",
            "Epoch [15/30], Step [97/139], Loss: 0.0845\n",
            "Epoch [15/30], Step [98/139], Loss: 0.0745\n",
            "Epoch [15/30], Step [99/139], Loss: 0.0947\n",
            "Epoch [15/30], Step [100/139], Loss: 0.0910\n",
            "Epoch [15/30], Step [101/139], Loss: 0.1160\n",
            "Epoch [15/30], Step [102/139], Loss: 0.0745\n",
            "Epoch [15/30], Step [103/139], Loss: 0.0673\n",
            "Epoch [15/30], Step [104/139], Loss: 0.0540\n",
            "Epoch [15/30], Step [105/139], Loss: 0.0524\n",
            "Epoch [15/30], Step [106/139], Loss: 0.0831\n",
            "Epoch [15/30], Step [107/139], Loss: 0.0949\n",
            "Epoch [15/30], Step [108/139], Loss: 0.0732\n",
            "Epoch [15/30], Step [109/139], Loss: 0.1050\n",
            "Epoch [15/30], Step [110/139], Loss: 0.0940\n",
            "Epoch [15/30], Step [111/139], Loss: 0.1027\n",
            "Epoch [15/30], Step [112/139], Loss: 0.0711\n",
            "Epoch [15/30], Step [113/139], Loss: 0.0694\n",
            "Epoch [15/30], Step [114/139], Loss: 0.0804\n",
            "Epoch [15/30], Step [115/139], Loss: 0.0993\n",
            "Epoch [15/30], Step [116/139], Loss: 0.1135\n",
            "Epoch [15/30], Step [117/139], Loss: 0.0919\n",
            "Epoch [15/30], Step [118/139], Loss: 0.0760\n",
            "Epoch [15/30], Step [119/139], Loss: 0.0812\n",
            "Epoch [15/30], Step [120/139], Loss: 0.0740\n",
            "Epoch [15/30], Step [121/139], Loss: 0.0622\n",
            "Epoch [15/30], Step [122/139], Loss: 0.0958\n",
            "Epoch [15/30], Step [123/139], Loss: 0.0825\n",
            "Epoch [15/30], Step [124/139], Loss: 0.0887\n",
            "Epoch [15/30], Step [125/139], Loss: 0.0789\n",
            "Epoch [15/30], Step [126/139], Loss: 0.0667\n",
            "Epoch [15/30], Step [127/139], Loss: 0.0962\n",
            "Epoch [15/30], Step [128/139], Loss: 0.0649\n",
            "Epoch [15/30], Step [129/139], Loss: 0.0893\n",
            "Epoch [15/30], Step [130/139], Loss: 0.1065\n",
            "Epoch [15/30], Step [131/139], Loss: 0.1080\n",
            "Epoch [15/30], Step [132/139], Loss: 0.0795\n",
            "Epoch [15/30], Step [133/139], Loss: 0.0818\n",
            "Epoch [15/30], Step [134/139], Loss: 0.0807\n",
            "Epoch [15/30], Step [135/139], Loss: 0.0591\n",
            "Epoch [15/30], Step [136/139], Loss: 0.1251\n",
            "Epoch [15/30], Step [137/139], Loss: 0.0886\n",
            "Epoch [15/30], Step [138/139], Loss: 0.0891\n",
            "Epoch [15/30], Step [139/139], Loss: 0.0775\n",
            "Start validation #15\n",
            "Validation #15  Average Loss: 0.0940\n",
            "Best performance at epoch: 15\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [16/30], Step [1/139], Loss: 0.0673\n",
            "Epoch [16/30], Step [2/139], Loss: 0.0941\n",
            "Epoch [16/30], Step [3/139], Loss: 0.0763\n",
            "Epoch [16/30], Step [4/139], Loss: 0.1058\n",
            "Epoch [16/30], Step [5/139], Loss: 0.0798\n",
            "Epoch [16/30], Step [6/139], Loss: 0.1031\n",
            "Epoch [16/30], Step [7/139], Loss: 0.0740\n",
            "Epoch [16/30], Step [8/139], Loss: 0.1247\n",
            "Epoch [16/30], Step [9/139], Loss: 0.1119\n",
            "Epoch [16/30], Step [10/139], Loss: 0.0957\n",
            "Epoch [16/30], Step [11/139], Loss: 0.0785\n",
            "Epoch [16/30], Step [12/139], Loss: 0.0778\n",
            "Epoch [16/30], Step [13/139], Loss: 0.0852\n",
            "Epoch [16/30], Step [14/139], Loss: 0.0636\n",
            "Epoch [16/30], Step [15/139], Loss: 0.1059\n",
            "Epoch [16/30], Step [16/139], Loss: 0.0958\n",
            "Epoch [16/30], Step [17/139], Loss: 0.0736\n",
            "Epoch [16/30], Step [18/139], Loss: 0.0758\n",
            "Epoch [16/30], Step [19/139], Loss: 0.0608\n",
            "Epoch [16/30], Step [20/139], Loss: 0.1026\n",
            "Epoch [16/30], Step [21/139], Loss: 0.0763\n",
            "Epoch [16/30], Step [22/139], Loss: 0.0840\n",
            "Epoch [16/30], Step [23/139], Loss: 0.0645\n",
            "Epoch [16/30], Step [24/139], Loss: 0.0773\n",
            "Epoch [16/30], Step [25/139], Loss: 0.0694\n",
            "Epoch [16/30], Step [26/139], Loss: 0.0763\n",
            "Epoch [16/30], Step [27/139], Loss: 0.0789\n",
            "Epoch [16/30], Step [28/139], Loss: 0.0666\n",
            "Epoch [16/30], Step [29/139], Loss: 0.0723\n",
            "Epoch [16/30], Step [30/139], Loss: 0.0842\n",
            "Epoch [16/30], Step [31/139], Loss: 0.0906\n",
            "Epoch [16/30], Step [32/139], Loss: 0.0609\n",
            "Epoch [16/30], Step [33/139], Loss: 0.0989\n",
            "Epoch [16/30], Step [34/139], Loss: 0.0999\n",
            "Epoch [16/30], Step [35/139], Loss: 0.0617\n",
            "Epoch [16/30], Step [36/139], Loss: 0.0665\n",
            "Epoch [16/30], Step [37/139], Loss: 0.0731\n",
            "Epoch [16/30], Step [38/139], Loss: 0.0881\n",
            "Epoch [16/30], Step [39/139], Loss: 0.0546\n",
            "Epoch [16/30], Step [40/139], Loss: 0.0905\n",
            "Epoch [16/30], Step [41/139], Loss: 0.1083\n",
            "Epoch [16/30], Step [42/139], Loss: 0.0752\n",
            "Epoch [16/30], Step [43/139], Loss: 0.0827\n",
            "Epoch [16/30], Step [44/139], Loss: 0.0865\n",
            "Epoch [16/30], Step [45/139], Loss: 0.0874\n",
            "Epoch [16/30], Step [46/139], Loss: 0.1027\n",
            "Epoch [16/30], Step [47/139], Loss: 0.1016\n",
            "Epoch [16/30], Step [48/139], Loss: 0.0598\n",
            "Epoch [16/30], Step [49/139], Loss: 0.0815\n",
            "Epoch [16/30], Step [50/139], Loss: 0.0547\n",
            "Epoch [16/30], Step [51/139], Loss: 0.0937\n",
            "Epoch [16/30], Step [52/139], Loss: 0.0738\n",
            "Epoch [16/30], Step [53/139], Loss: 0.0690\n",
            "Epoch [16/30], Step [54/139], Loss: 0.0796\n",
            "Epoch [16/30], Step [55/139], Loss: 0.0904\n",
            "Epoch [16/30], Step [56/139], Loss: 0.0671\n",
            "Epoch [16/30], Step [57/139], Loss: 0.0926\n",
            "Epoch [16/30], Step [58/139], Loss: 0.0818\n",
            "Epoch [16/30], Step [59/139], Loss: 0.0730\n",
            "Epoch [16/30], Step [60/139], Loss: 0.1025\n",
            "Epoch [16/30], Step [61/139], Loss: 0.1030\n",
            "Epoch [16/30], Step [62/139], Loss: 0.0812\n",
            "Epoch [16/30], Step [63/139], Loss: 0.0870\n",
            "Epoch [16/30], Step [64/139], Loss: 0.0822\n",
            "Epoch [16/30], Step [65/139], Loss: 0.1004\n",
            "Epoch [16/30], Step [66/139], Loss: 0.0776\n",
            "Epoch [16/30], Step [67/139], Loss: 0.0714\n",
            "Epoch [16/30], Step [68/139], Loss: 0.0922\n",
            "Epoch [16/30], Step [69/139], Loss: 0.1052\n",
            "Epoch [16/30], Step [70/139], Loss: 0.0693\n",
            "Epoch [16/30], Step [71/139], Loss: 0.0713\n",
            "Epoch [16/30], Step [72/139], Loss: 0.0916\n",
            "Epoch [16/30], Step [73/139], Loss: 0.0872\n",
            "Epoch [16/30], Step [74/139], Loss: 0.1065\n",
            "Epoch [16/30], Step [75/139], Loss: 0.0799\n",
            "Epoch [16/30], Step [76/139], Loss: 0.1007\n",
            "Epoch [16/30], Step [77/139], Loss: 0.1055\n",
            "Epoch [16/30], Step [78/139], Loss: 0.0909\n",
            "Epoch [16/30], Step [79/139], Loss: 0.0761\n",
            "Epoch [16/30], Step [80/139], Loss: 0.0876\n",
            "Epoch [16/30], Step [81/139], Loss: 0.0907\n",
            "Epoch [16/30], Step [82/139], Loss: 0.0660\n",
            "Epoch [16/30], Step [83/139], Loss: 0.0768\n",
            "Epoch [16/30], Step [84/139], Loss: 0.0714\n",
            "Epoch [16/30], Step [85/139], Loss: 0.0832\n",
            "Epoch [16/30], Step [86/139], Loss: 0.0542\n",
            "Epoch [16/30], Step [87/139], Loss: 0.0796\n",
            "Epoch [16/30], Step [88/139], Loss: 0.0775\n",
            "Epoch [16/30], Step [89/139], Loss: 0.0966\n",
            "Epoch [16/30], Step [90/139], Loss: 0.0819\n",
            "Epoch [16/30], Step [91/139], Loss: 0.0937\n",
            "Epoch [16/30], Step [92/139], Loss: 0.0825\n",
            "Epoch [16/30], Step [93/139], Loss: 0.1027\n",
            "Epoch [16/30], Step [94/139], Loss: 0.0702\n",
            "Epoch [16/30], Step [95/139], Loss: 0.0724\n",
            "Epoch [16/30], Step [96/139], Loss: 0.0677\n",
            "Epoch [16/30], Step [97/139], Loss: 0.0978\n",
            "Epoch [16/30], Step [98/139], Loss: 0.1035\n",
            "Epoch [16/30], Step [99/139], Loss: 0.0911\n",
            "Epoch [16/30], Step [100/139], Loss: 0.0798\n",
            "Epoch [16/30], Step [101/139], Loss: 0.0790\n",
            "Epoch [16/30], Step [102/139], Loss: 0.0735\n",
            "Epoch [16/30], Step [103/139], Loss: 0.1091\n",
            "Epoch [16/30], Step [104/139], Loss: 0.1078\n",
            "Epoch [16/30], Step [105/139], Loss: 0.0813\n",
            "Epoch [16/30], Step [106/139], Loss: 0.0766\n",
            "Epoch [16/30], Step [107/139], Loss: 0.0700\n",
            "Epoch [16/30], Step [108/139], Loss: 0.0895\n",
            "Epoch [16/30], Step [109/139], Loss: 0.0869\n",
            "Epoch [16/30], Step [110/139], Loss: 0.0756\n",
            "Epoch [16/30], Step [111/139], Loss: 0.0653\n",
            "Epoch [16/30], Step [112/139], Loss: 0.0606\n",
            "Epoch [16/30], Step [113/139], Loss: 0.0897\n",
            "Epoch [16/30], Step [114/139], Loss: 0.0980\n",
            "Epoch [16/30], Step [115/139], Loss: 0.0856\n",
            "Epoch [16/30], Step [116/139], Loss: 0.1070\n",
            "Epoch [16/30], Step [117/139], Loss: 0.0848\n",
            "Epoch [16/30], Step [118/139], Loss: 0.0935\n",
            "Epoch [16/30], Step [119/139], Loss: 0.0925\n",
            "Epoch [16/30], Step [120/139], Loss: 0.0678\n",
            "Epoch [16/30], Step [121/139], Loss: 0.0782\n",
            "Epoch [16/30], Step [122/139], Loss: 0.0659\n",
            "Epoch [16/30], Step [123/139], Loss: 0.0646\n",
            "Epoch [16/30], Step [124/139], Loss: 0.0844\n",
            "Epoch [16/30], Step [125/139], Loss: 0.1041\n",
            "Epoch [16/30], Step [126/139], Loss: 0.0957\n",
            "Epoch [16/30], Step [127/139], Loss: 0.0754\n",
            "Epoch [16/30], Step [128/139], Loss: 0.0742\n",
            "Epoch [16/30], Step [129/139], Loss: 0.0855\n",
            "Epoch [16/30], Step [130/139], Loss: 0.0601\n",
            "Epoch [16/30], Step [131/139], Loss: 0.1311\n",
            "Epoch [16/30], Step [132/139], Loss: 0.0749\n",
            "Epoch [16/30], Step [133/139], Loss: 0.1006\n",
            "Epoch [16/30], Step [134/139], Loss: 0.0871\n",
            "Epoch [16/30], Step [135/139], Loss: 0.0812\n",
            "Epoch [16/30], Step [136/139], Loss: 0.0924\n",
            "Epoch [16/30], Step [137/139], Loss: 0.0965\n",
            "Epoch [16/30], Step [138/139], Loss: 0.0921\n",
            "Epoch [16/30], Step [139/139], Loss: 0.0726\n",
            "Start validation #16\n",
            "Validation #16  Average Loss: 0.0917\n",
            "Best performance at epoch: 16\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [17/30], Step [1/139], Loss: 0.0740\n",
            "Epoch [17/30], Step [2/139], Loss: 0.0801\n",
            "Epoch [17/30], Step [3/139], Loss: 0.0794\n",
            "Epoch [17/30], Step [4/139], Loss: 0.0779\n",
            "Epoch [17/30], Step [5/139], Loss: 0.0893\n",
            "Epoch [17/30], Step [6/139], Loss: 0.0885\n",
            "Epoch [17/30], Step [7/139], Loss: 0.0767\n",
            "Epoch [17/30], Step [8/139], Loss: 0.0769\n",
            "Epoch [17/30], Step [9/139], Loss: 0.0770\n",
            "Epoch [17/30], Step [10/139], Loss: 0.1032\n",
            "Epoch [17/30], Step [11/139], Loss: 0.0951\n",
            "Epoch [17/30], Step [12/139], Loss: 0.0801\n",
            "Epoch [17/30], Step [13/139], Loss: 0.0902\n",
            "Epoch [17/30], Step [14/139], Loss: 0.0925\n",
            "Epoch [17/30], Step [15/139], Loss: 0.0787\n",
            "Epoch [17/30], Step [16/139], Loss: 0.1262\n",
            "Epoch [17/30], Step [17/139], Loss: 0.0866\n",
            "Epoch [17/30], Step [18/139], Loss: 0.0728\n",
            "Epoch [17/30], Step [19/139], Loss: 0.0738\n",
            "Epoch [17/30], Step [20/139], Loss: 0.0944\n",
            "Epoch [17/30], Step [21/139], Loss: 0.1065\n",
            "Epoch [17/30], Step [22/139], Loss: 0.0977\n",
            "Epoch [17/30], Step [23/139], Loss: 0.1013\n",
            "Epoch [17/30], Step [24/139], Loss: 0.0712\n",
            "Epoch [17/30], Step [25/139], Loss: 0.0845\n",
            "Epoch [17/30], Step [26/139], Loss: 0.1175\n",
            "Epoch [17/30], Step [27/139], Loss: 0.0818\n",
            "Epoch [17/30], Step [28/139], Loss: 0.0723\n",
            "Epoch [17/30], Step [29/139], Loss: 0.0901\n",
            "Epoch [17/30], Step [30/139], Loss: 0.0829\n",
            "Epoch [17/30], Step [31/139], Loss: 0.0687\n",
            "Epoch [17/30], Step [32/139], Loss: 0.0796\n",
            "Epoch [17/30], Step [33/139], Loss: 0.1077\n",
            "Epoch [17/30], Step [34/139], Loss: 0.0824\n",
            "Epoch [17/30], Step [35/139], Loss: 0.0716\n",
            "Epoch [17/30], Step [36/139], Loss: 0.0749\n",
            "Epoch [17/30], Step [37/139], Loss: 0.0734\n",
            "Epoch [17/30], Step [38/139], Loss: 0.0571\n",
            "Epoch [17/30], Step [39/139], Loss: 0.1006\n",
            "Epoch [17/30], Step [40/139], Loss: 0.0797\n",
            "Epoch [17/30], Step [41/139], Loss: 0.0819\n",
            "Epoch [17/30], Step [42/139], Loss: 0.0732\n",
            "Epoch [17/30], Step [43/139], Loss: 0.0821\n",
            "Epoch [17/30], Step [44/139], Loss: 0.0838\n",
            "Epoch [17/30], Step [45/139], Loss: 0.0849\n",
            "Epoch [17/30], Step [46/139], Loss: 0.0916\n",
            "Epoch [17/30], Step [47/139], Loss: 0.0999\n",
            "Epoch [17/30], Step [48/139], Loss: 0.0626\n",
            "Epoch [17/30], Step [49/139], Loss: 0.0853\n",
            "Epoch [17/30], Step [50/139], Loss: 0.0777\n",
            "Epoch [17/30], Step [51/139], Loss: 0.0887\n",
            "Epoch [17/30], Step [52/139], Loss: 0.0813\n",
            "Epoch [17/30], Step [53/139], Loss: 0.1107\n",
            "Epoch [17/30], Step [54/139], Loss: 0.0835\n",
            "Epoch [17/30], Step [55/139], Loss: 0.0703\n",
            "Epoch [17/30], Step [56/139], Loss: 0.0754\n",
            "Epoch [17/30], Step [57/139], Loss: 0.0802\n",
            "Epoch [17/30], Step [58/139], Loss: 0.0720\n",
            "Epoch [17/30], Step [59/139], Loss: 0.0765\n",
            "Epoch [17/30], Step [60/139], Loss: 0.0684\n",
            "Epoch [17/30], Step [61/139], Loss: 0.0883\n",
            "Epoch [17/30], Step [62/139], Loss: 0.0565\n",
            "Epoch [17/30], Step [63/139], Loss: 0.0725\n",
            "Epoch [17/30], Step [64/139], Loss: 0.0713\n",
            "Epoch [17/30], Step [65/139], Loss: 0.0764\n",
            "Epoch [17/30], Step [66/139], Loss: 0.0883\n",
            "Epoch [17/30], Step [67/139], Loss: 0.0658\n",
            "Epoch [17/30], Step [68/139], Loss: 0.0886\n",
            "Epoch [17/30], Step [69/139], Loss: 0.0836\n",
            "Epoch [17/30], Step [70/139], Loss: 0.0645\n",
            "Epoch [17/30], Step [71/139], Loss: 0.0785\n",
            "Epoch [17/30], Step [72/139], Loss: 0.0868\n",
            "Epoch [17/30], Step [73/139], Loss: 0.0744\n",
            "Epoch [17/30], Step [74/139], Loss: 0.0881\n",
            "Epoch [17/30], Step [75/139], Loss: 0.0941\n",
            "Epoch [17/30], Step [76/139], Loss: 0.0879\n",
            "Epoch [17/30], Step [77/139], Loss: 0.0605\n",
            "Epoch [17/30], Step [78/139], Loss: 0.0848\n",
            "Epoch [17/30], Step [79/139], Loss: 0.1025\n",
            "Epoch [17/30], Step [80/139], Loss: 0.1041\n",
            "Epoch [17/30], Step [81/139], Loss: 0.0751\n",
            "Epoch [17/30], Step [82/139], Loss: 0.0954\n",
            "Epoch [17/30], Step [83/139], Loss: 0.0813\n",
            "Epoch [17/30], Step [84/139], Loss: 0.0755\n",
            "Epoch [17/30], Step [85/139], Loss: 0.0784\n",
            "Epoch [17/30], Step [86/139], Loss: 0.0616\n",
            "Epoch [17/30], Step [87/139], Loss: 0.0696\n",
            "Epoch [17/30], Step [88/139], Loss: 0.1112\n",
            "Epoch [17/30], Step [89/139], Loss: 0.1063\n",
            "Epoch [17/30], Step [90/139], Loss: 0.0705\n",
            "Epoch [17/30], Step [91/139], Loss: 0.0863\n",
            "Epoch [17/30], Step [92/139], Loss: 0.0791\n",
            "Epoch [17/30], Step [93/139], Loss: 0.0816\n",
            "Epoch [17/30], Step [94/139], Loss: 0.0708\n",
            "Epoch [17/30], Step [95/139], Loss: 0.0652\n",
            "Epoch [17/30], Step [96/139], Loss: 0.0946\n",
            "Epoch [17/30], Step [97/139], Loss: 0.0772\n",
            "Epoch [17/30], Step [98/139], Loss: 0.1075\n",
            "Epoch [17/30], Step [99/139], Loss: 0.0531\n",
            "Epoch [17/30], Step [100/139], Loss: 0.1011\n",
            "Epoch [17/30], Step [101/139], Loss: 0.0577\n",
            "Epoch [17/30], Step [102/139], Loss: 0.1091\n",
            "Epoch [17/30], Step [103/139], Loss: 0.1068\n",
            "Epoch [17/30], Step [104/139], Loss: 0.0827\n",
            "Epoch [17/30], Step [105/139], Loss: 0.0711\n",
            "Epoch [17/30], Step [106/139], Loss: 0.0738\n",
            "Epoch [17/30], Step [107/139], Loss: 0.0887\n",
            "Epoch [17/30], Step [108/139], Loss: 0.0735\n",
            "Epoch [17/30], Step [109/139], Loss: 0.0875\n",
            "Epoch [17/30], Step [110/139], Loss: 0.0624\n",
            "Epoch [17/30], Step [111/139], Loss: 0.0855\n",
            "Epoch [17/30], Step [112/139], Loss: 0.0952\n",
            "Epoch [17/30], Step [113/139], Loss: 0.0838\n",
            "Epoch [17/30], Step [114/139], Loss: 0.0843\n",
            "Epoch [17/30], Step [115/139], Loss: 0.0793\n",
            "Epoch [17/30], Step [116/139], Loss: 0.0763\n",
            "Epoch [17/30], Step [117/139], Loss: 0.1084\n",
            "Epoch [17/30], Step [118/139], Loss: 0.0784\n",
            "Epoch [17/30], Step [119/139], Loss: 0.0665\n",
            "Epoch [17/30], Step [120/139], Loss: 0.0847\n",
            "Epoch [17/30], Step [121/139], Loss: 0.0679\n",
            "Epoch [17/30], Step [122/139], Loss: 0.0758\n",
            "Epoch [17/30], Step [123/139], Loss: 0.0857\n",
            "Epoch [17/30], Step [124/139], Loss: 0.0834\n",
            "Epoch [17/30], Step [125/139], Loss: 0.0607\n",
            "Epoch [17/30], Step [126/139], Loss: 0.1094\n",
            "Epoch [17/30], Step [127/139], Loss: 0.0883\n",
            "Epoch [17/30], Step [128/139], Loss: 0.0912\n",
            "Epoch [17/30], Step [129/139], Loss: 0.0962\n",
            "Epoch [17/30], Step [130/139], Loss: 0.0931\n",
            "Epoch [17/30], Step [131/139], Loss: 0.0576\n",
            "Epoch [17/30], Step [132/139], Loss: 0.0805\n",
            "Epoch [17/30], Step [133/139], Loss: 0.0730\n",
            "Epoch [17/30], Step [134/139], Loss: 0.0917\n",
            "Epoch [17/30], Step [135/139], Loss: 0.0649\n",
            "Epoch [17/30], Step [136/139], Loss: 0.0678\n",
            "Epoch [17/30], Step [137/139], Loss: 0.0699\n",
            "Epoch [17/30], Step [138/139], Loss: 0.0967\n",
            "Epoch [17/30], Step [139/139], Loss: 0.0678\n",
            "Start validation #17\n",
            "Validation #17  Average Loss: 0.0902\n",
            "Best performance at epoch: 17\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [18/30], Step [1/139], Loss: 0.0588\n",
            "Epoch [18/30], Step [2/139], Loss: 0.0902\n",
            "Epoch [18/30], Step [3/139], Loss: 0.0667\n",
            "Epoch [18/30], Step [4/139], Loss: 0.0779\n",
            "Epoch [18/30], Step [5/139], Loss: 0.0847\n",
            "Epoch [18/30], Step [6/139], Loss: 0.0788\n",
            "Epoch [18/30], Step [7/139], Loss: 0.0862\n",
            "Epoch [18/30], Step [8/139], Loss: 0.0922\n",
            "Epoch [18/30], Step [9/139], Loss: 0.0764\n",
            "Epoch [18/30], Step [10/139], Loss: 0.0822\n",
            "Epoch [18/30], Step [11/139], Loss: 0.0798\n",
            "Epoch [18/30], Step [12/139], Loss: 0.0654\n",
            "Epoch [18/30], Step [13/139], Loss: 0.0664\n",
            "Epoch [18/30], Step [14/139], Loss: 0.0855\n",
            "Epoch [18/30], Step [15/139], Loss: 0.0825\n",
            "Epoch [18/30], Step [16/139], Loss: 0.0785\n",
            "Epoch [18/30], Step [17/139], Loss: 0.0595\n",
            "Epoch [18/30], Step [18/139], Loss: 0.0866\n",
            "Epoch [18/30], Step [19/139], Loss: 0.0948\n",
            "Epoch [18/30], Step [20/139], Loss: 0.0930\n",
            "Epoch [18/30], Step [21/139], Loss: 0.1141\n",
            "Epoch [18/30], Step [22/139], Loss: 0.0761\n",
            "Epoch [18/30], Step [23/139], Loss: 0.0977\n",
            "Epoch [18/30], Step [24/139], Loss: 0.0698\n",
            "Epoch [18/30], Step [25/139], Loss: 0.0611\n",
            "Epoch [18/30], Step [26/139], Loss: 0.0830\n",
            "Epoch [18/30], Step [27/139], Loss: 0.0861\n",
            "Epoch [18/30], Step [28/139], Loss: 0.0699\n",
            "Epoch [18/30], Step [29/139], Loss: 0.0824\n",
            "Epoch [18/30], Step [30/139], Loss: 0.1043\n",
            "Epoch [18/30], Step [31/139], Loss: 0.0653\n",
            "Epoch [18/30], Step [32/139], Loss: 0.1113\n",
            "Epoch [18/30], Step [33/139], Loss: 0.0540\n",
            "Epoch [18/30], Step [34/139], Loss: 0.0744\n",
            "Epoch [18/30], Step [35/139], Loss: 0.0788\n",
            "Epoch [18/30], Step [36/139], Loss: 0.1119\n",
            "Epoch [18/30], Step [37/139], Loss: 0.0601\n",
            "Epoch [18/30], Step [38/139], Loss: 0.0836\n",
            "Epoch [18/30], Step [39/139], Loss: 0.1148\n",
            "Epoch [18/30], Step [40/139], Loss: 0.0680\n",
            "Epoch [18/30], Step [41/139], Loss: 0.0881\n",
            "Epoch [18/30], Step [42/139], Loss: 0.1006\n",
            "Epoch [18/30], Step [43/139], Loss: 0.0738\n",
            "Epoch [18/30], Step [44/139], Loss: 0.0571\n",
            "Epoch [18/30], Step [45/139], Loss: 0.0637\n",
            "Epoch [18/30], Step [46/139], Loss: 0.0965\n",
            "Epoch [18/30], Step [47/139], Loss: 0.0870\n",
            "Epoch [18/30], Step [48/139], Loss: 0.0783\n",
            "Epoch [18/30], Step [49/139], Loss: 0.0888\n",
            "Epoch [18/30], Step [50/139], Loss: 0.0712\n",
            "Epoch [18/30], Step [51/139], Loss: 0.0619\n",
            "Epoch [18/30], Step [52/139], Loss: 0.0615\n",
            "Epoch [18/30], Step [53/139], Loss: 0.1004\n",
            "Epoch [18/30], Step [54/139], Loss: 0.0840\n",
            "Epoch [18/30], Step [55/139], Loss: 0.0754\n",
            "Epoch [18/30], Step [56/139], Loss: 0.1008\n",
            "Epoch [18/30], Step [57/139], Loss: 0.1016\n",
            "Epoch [18/30], Step [58/139], Loss: 0.0872\n",
            "Epoch [18/30], Step [59/139], Loss: 0.0833\n",
            "Epoch [18/30], Step [60/139], Loss: 0.0992\n",
            "Epoch [18/30], Step [61/139], Loss: 0.0894\n",
            "Epoch [18/30], Step [62/139], Loss: 0.1169\n",
            "Epoch [18/30], Step [63/139], Loss: 0.0661\n",
            "Epoch [18/30], Step [64/139], Loss: 0.0742\n",
            "Epoch [18/30], Step [65/139], Loss: 0.0906\n",
            "Epoch [18/30], Step [66/139], Loss: 0.0792\n",
            "Epoch [18/30], Step [67/139], Loss: 0.0837\n",
            "Epoch [18/30], Step [68/139], Loss: 0.0809\n",
            "Epoch [18/30], Step [69/139], Loss: 0.0804\n",
            "Epoch [18/30], Step [70/139], Loss: 0.0634\n",
            "Epoch [18/30], Step [71/139], Loss: 0.0702\n",
            "Epoch [18/30], Step [72/139], Loss: 0.0983\n",
            "Epoch [18/30], Step [73/139], Loss: 0.0954\n",
            "Epoch [18/30], Step [74/139], Loss: 0.0864\n",
            "Epoch [18/30], Step [75/139], Loss: 0.0484\n",
            "Epoch [18/30], Step [76/139], Loss: 0.0615\n",
            "Epoch [18/30], Step [77/139], Loss: 0.0797\n",
            "Epoch [18/30], Step [78/139], Loss: 0.0783\n",
            "Epoch [18/30], Step [79/139], Loss: 0.0651\n",
            "Epoch [18/30], Step [80/139], Loss: 0.1010\n",
            "Epoch [18/30], Step [81/139], Loss: 0.0797\n",
            "Epoch [18/30], Step [82/139], Loss: 0.0991\n",
            "Epoch [18/30], Step [83/139], Loss: 0.1035\n",
            "Epoch [18/30], Step [84/139], Loss: 0.0519\n",
            "Epoch [18/30], Step [85/139], Loss: 0.0793\n",
            "Epoch [18/30], Step [86/139], Loss: 0.0857\n",
            "Epoch [18/30], Step [87/139], Loss: 0.0744\n",
            "Epoch [18/30], Step [88/139], Loss: 0.0737\n",
            "Epoch [18/30], Step [89/139], Loss: 0.0711\n",
            "Epoch [18/30], Step [90/139], Loss: 0.1300\n",
            "Epoch [18/30], Step [91/139], Loss: 0.0942\n",
            "Epoch [18/30], Step [92/139], Loss: 0.0755\n",
            "Epoch [18/30], Step [93/139], Loss: 0.0946\n",
            "Epoch [18/30], Step [94/139], Loss: 0.0975\n",
            "Epoch [18/30], Step [95/139], Loss: 0.0926\n",
            "Epoch [18/30], Step [96/139], Loss: 0.0883\n",
            "Epoch [18/30], Step [97/139], Loss: 0.0682\n",
            "Epoch [18/30], Step [98/139], Loss: 0.0948\n",
            "Epoch [18/30], Step [99/139], Loss: 0.0511\n",
            "Epoch [18/30], Step [100/139], Loss: 0.0828\n",
            "Epoch [18/30], Step [101/139], Loss: 0.0622\n",
            "Epoch [18/30], Step [102/139], Loss: 0.1016\n",
            "Epoch [18/30], Step [103/139], Loss: 0.0682\n",
            "Epoch [18/30], Step [104/139], Loss: 0.0694\n",
            "Epoch [18/30], Step [105/139], Loss: 0.0786\n",
            "Epoch [18/30], Step [106/139], Loss: 0.0724\n",
            "Epoch [18/30], Step [107/139], Loss: 0.0984\n",
            "Epoch [18/30], Step [108/139], Loss: 0.0645\n",
            "Epoch [18/30], Step [109/139], Loss: 0.0849\n",
            "Epoch [18/30], Step [110/139], Loss: 0.0706\n",
            "Epoch [18/30], Step [111/139], Loss: 0.0910\n",
            "Epoch [18/30], Step [112/139], Loss: 0.0600\n",
            "Epoch [18/30], Step [113/139], Loss: 0.0517\n",
            "Epoch [18/30], Step [114/139], Loss: 0.0671\n",
            "Epoch [18/30], Step [115/139], Loss: 0.0611\n",
            "Epoch [18/30], Step [116/139], Loss: 0.0833\n",
            "Epoch [18/30], Step [117/139], Loss: 0.0815\n",
            "Epoch [18/30], Step [118/139], Loss: 0.0830\n",
            "Epoch [18/30], Step [119/139], Loss: 0.1220\n",
            "Epoch [18/30], Step [120/139], Loss: 0.0690\n",
            "Epoch [18/30], Step [121/139], Loss: 0.0781\n",
            "Epoch [18/30], Step [122/139], Loss: 0.0522\n",
            "Epoch [18/30], Step [123/139], Loss: 0.0694\n",
            "Epoch [18/30], Step [124/139], Loss: 0.0728\n",
            "Epoch [18/30], Step [125/139], Loss: 0.0654\n",
            "Epoch [18/30], Step [126/139], Loss: 0.1019\n",
            "Epoch [18/30], Step [127/139], Loss: 0.1077\n",
            "Epoch [18/30], Step [128/139], Loss: 0.1166\n",
            "Epoch [18/30], Step [129/139], Loss: 0.0699\n",
            "Epoch [18/30], Step [130/139], Loss: 0.0838\n",
            "Epoch [18/30], Step [131/139], Loss: 0.0751\n",
            "Epoch [18/30], Step [132/139], Loss: 0.0708\n",
            "Epoch [18/30], Step [133/139], Loss: 0.0915\n",
            "Epoch [18/30], Step [134/139], Loss: 0.0858\n",
            "Epoch [18/30], Step [135/139], Loss: 0.0632\n",
            "Epoch [18/30], Step [136/139], Loss: 0.0849\n",
            "Epoch [18/30], Step [137/139], Loss: 0.0880\n",
            "Epoch [18/30], Step [138/139], Loss: 0.0726\n",
            "Epoch [18/30], Step [139/139], Loss: 0.0868\n",
            "Start validation #18\n",
            "Validation #18  Average Loss: 0.0885\n",
            "Best performance at epoch: 18\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [19/30], Step [1/139], Loss: 0.0733\n",
            "Epoch [19/30], Step [2/139], Loss: 0.0656\n",
            "Epoch [19/30], Step [3/139], Loss: 0.0702\n",
            "Epoch [19/30], Step [4/139], Loss: 0.0753\n",
            "Epoch [19/30], Step [5/139], Loss: 0.0780\n",
            "Epoch [19/30], Step [6/139], Loss: 0.0732\n",
            "Epoch [19/30], Step [7/139], Loss: 0.1088\n",
            "Epoch [19/30], Step [8/139], Loss: 0.1054\n",
            "Epoch [19/30], Step [9/139], Loss: 0.0817\n",
            "Epoch [19/30], Step [10/139], Loss: 0.0852\n",
            "Epoch [19/30], Step [11/139], Loss: 0.0771\n",
            "Epoch [19/30], Step [12/139], Loss: 0.0978\n",
            "Epoch [19/30], Step [13/139], Loss: 0.0627\n",
            "Epoch [19/30], Step [14/139], Loss: 0.0712\n",
            "Epoch [19/30], Step [15/139], Loss: 0.0963\n",
            "Epoch [19/30], Step [16/139], Loss: 0.0975\n",
            "Epoch [19/30], Step [17/139], Loss: 0.0869\n",
            "Epoch [19/30], Step [18/139], Loss: 0.1007\n",
            "Epoch [19/30], Step [19/139], Loss: 0.0730\n",
            "Epoch [19/30], Step [20/139], Loss: 0.0798\n",
            "Epoch [19/30], Step [21/139], Loss: 0.0711\n",
            "Epoch [19/30], Step [22/139], Loss: 0.0981\n",
            "Epoch [19/30], Step [23/139], Loss: 0.0697\n",
            "Epoch [19/30], Step [24/139], Loss: 0.1080\n",
            "Epoch [19/30], Step [25/139], Loss: 0.0530\n",
            "Epoch [19/30], Step [26/139], Loss: 0.0898\n",
            "Epoch [19/30], Step [27/139], Loss: 0.0591\n",
            "Epoch [19/30], Step [28/139], Loss: 0.0749\n",
            "Epoch [19/30], Step [29/139], Loss: 0.0778\n",
            "Epoch [19/30], Step [30/139], Loss: 0.1006\n",
            "Epoch [19/30], Step [31/139], Loss: 0.0655\n",
            "Epoch [19/30], Step [32/139], Loss: 0.0898\n",
            "Epoch [19/30], Step [33/139], Loss: 0.0520\n",
            "Epoch [19/30], Step [34/139], Loss: 0.0689\n",
            "Epoch [19/30], Step [35/139], Loss: 0.0783\n",
            "Epoch [19/30], Step [36/139], Loss: 0.0877\n",
            "Epoch [19/30], Step [37/139], Loss: 0.0942\n",
            "Epoch [19/30], Step [38/139], Loss: 0.0749\n",
            "Epoch [19/30], Step [39/139], Loss: 0.0558\n",
            "Epoch [19/30], Step [40/139], Loss: 0.0805\n",
            "Epoch [19/30], Step [41/139], Loss: 0.1072\n",
            "Epoch [19/30], Step [42/139], Loss: 0.0721\n",
            "Epoch [19/30], Step [43/139], Loss: 0.0769\n",
            "Epoch [19/30], Step [44/139], Loss: 0.0943\n",
            "Epoch [19/30], Step [45/139], Loss: 0.0707\n",
            "Epoch [19/30], Step [46/139], Loss: 0.0836\n",
            "Epoch [19/30], Step [47/139], Loss: 0.0602\n",
            "Epoch [19/30], Step [48/139], Loss: 0.0714\n",
            "Epoch [19/30], Step [49/139], Loss: 0.1086\n",
            "Epoch [19/30], Step [50/139], Loss: 0.0891\n",
            "Epoch [19/30], Step [51/139], Loss: 0.0841\n",
            "Epoch [19/30], Step [52/139], Loss: 0.0957\n",
            "Epoch [19/30], Step [53/139], Loss: 0.0800\n",
            "Epoch [19/30], Step [54/139], Loss: 0.0661\n",
            "Epoch [19/30], Step [55/139], Loss: 0.0824\n",
            "Epoch [19/30], Step [56/139], Loss: 0.0726\n",
            "Epoch [19/30], Step [57/139], Loss: 0.0580\n",
            "Epoch [19/30], Step [58/139], Loss: 0.0734\n",
            "Epoch [19/30], Step [59/139], Loss: 0.0970\n",
            "Epoch [19/30], Step [60/139], Loss: 0.0674\n",
            "Epoch [19/30], Step [61/139], Loss: 0.0704\n",
            "Epoch [19/30], Step [62/139], Loss: 0.0900\n",
            "Epoch [19/30], Step [63/139], Loss: 0.0797\n",
            "Epoch [19/30], Step [64/139], Loss: 0.1072\n",
            "Epoch [19/30], Step [65/139], Loss: 0.0678\n",
            "Epoch [19/30], Step [66/139], Loss: 0.0712\n",
            "Epoch [19/30], Step [67/139], Loss: 0.0709\n",
            "Epoch [19/30], Step [68/139], Loss: 0.0597\n",
            "Epoch [19/30], Step [69/139], Loss: 0.0917\n",
            "Epoch [19/30], Step [70/139], Loss: 0.0779\n",
            "Epoch [19/30], Step [71/139], Loss: 0.0901\n",
            "Epoch [19/30], Step [72/139], Loss: 0.0789\n",
            "Epoch [19/30], Step [73/139], Loss: 0.0704\n",
            "Epoch [19/30], Step [74/139], Loss: 0.0599\n",
            "Epoch [19/30], Step [75/139], Loss: 0.0832\n",
            "Epoch [19/30], Step [76/139], Loss: 0.0602\n",
            "Epoch [19/30], Step [77/139], Loss: 0.0881\n",
            "Epoch [19/30], Step [78/139], Loss: 0.0684\n",
            "Epoch [19/30], Step [79/139], Loss: 0.0808\n",
            "Epoch [19/30], Step [80/139], Loss: 0.0989\n",
            "Epoch [19/30], Step [81/139], Loss: 0.0763\n",
            "Epoch [19/30], Step [82/139], Loss: 0.0808\n",
            "Epoch [19/30], Step [83/139], Loss: 0.1029\n",
            "Epoch [19/30], Step [84/139], Loss: 0.0721\n",
            "Epoch [19/30], Step [85/139], Loss: 0.1072\n",
            "Epoch [19/30], Step [86/139], Loss: 0.0765\n",
            "Epoch [19/30], Step [87/139], Loss: 0.0897\n",
            "Epoch [19/30], Step [88/139], Loss: 0.0661\n",
            "Epoch [19/30], Step [89/139], Loss: 0.0689\n",
            "Epoch [19/30], Step [90/139], Loss: 0.0856\n",
            "Epoch [19/30], Step [91/139], Loss: 0.0728\n",
            "Epoch [19/30], Step [92/139], Loss: 0.0859\n",
            "Epoch [19/30], Step [93/139], Loss: 0.0721\n",
            "Epoch [19/30], Step [94/139], Loss: 0.0965\n",
            "Epoch [19/30], Step [95/139], Loss: 0.0746\n",
            "Epoch [19/30], Step [96/139], Loss: 0.0580\n",
            "Epoch [19/30], Step [97/139], Loss: 0.0849\n",
            "Epoch [19/30], Step [98/139], Loss: 0.0726\n",
            "Epoch [19/30], Step [99/139], Loss: 0.0780\n",
            "Epoch [19/30], Step [100/139], Loss: 0.0599\n",
            "Epoch [19/30], Step [101/139], Loss: 0.0781\n",
            "Epoch [19/30], Step [102/139], Loss: 0.0770\n",
            "Epoch [19/30], Step [103/139], Loss: 0.0621\n",
            "Epoch [19/30], Step [104/139], Loss: 0.1168\n",
            "Epoch [19/30], Step [105/139], Loss: 0.0875\n",
            "Epoch [19/30], Step [106/139], Loss: 0.0677\n",
            "Epoch [19/30], Step [107/139], Loss: 0.0860\n",
            "Epoch [19/30], Step [108/139], Loss: 0.0921\n",
            "Epoch [19/30], Step [109/139], Loss: 0.0767\n",
            "Epoch [19/30], Step [110/139], Loss: 0.0891\n",
            "Epoch [19/30], Step [111/139], Loss: 0.0786\n",
            "Epoch [19/30], Step [112/139], Loss: 0.0666\n",
            "Epoch [19/30], Step [113/139], Loss: 0.0905\n",
            "Epoch [19/30], Step [114/139], Loss: 0.0660\n",
            "Epoch [19/30], Step [115/139], Loss: 0.0837\n",
            "Epoch [19/30], Step [116/139], Loss: 0.0921\n",
            "Epoch [19/30], Step [117/139], Loss: 0.0649\n",
            "Epoch [19/30], Step [118/139], Loss: 0.0770\n",
            "Epoch [19/30], Step [119/139], Loss: 0.0747\n",
            "Epoch [19/30], Step [120/139], Loss: 0.0633\n",
            "Epoch [19/30], Step [121/139], Loss: 0.0735\n",
            "Epoch [19/30], Step [122/139], Loss: 0.0623\n",
            "Epoch [19/30], Step [123/139], Loss: 0.1067\n",
            "Epoch [19/30], Step [124/139], Loss: 0.0682\n",
            "Epoch [19/30], Step [125/139], Loss: 0.0980\n",
            "Epoch [19/30], Step [126/139], Loss: 0.0999\n",
            "Epoch [19/30], Step [127/139], Loss: 0.0668\n",
            "Epoch [19/30], Step [128/139], Loss: 0.0753\n",
            "Epoch [19/30], Step [129/139], Loss: 0.0713\n",
            "Epoch [19/30], Step [130/139], Loss: 0.0929\n",
            "Epoch [19/30], Step [131/139], Loss: 0.0706\n",
            "Epoch [19/30], Step [132/139], Loss: 0.1119\n",
            "Epoch [19/30], Step [133/139], Loss: 0.0697\n",
            "Epoch [19/30], Step [134/139], Loss: 0.0820\n",
            "Epoch [19/30], Step [135/139], Loss: 0.0918\n",
            "Epoch [19/30], Step [136/139], Loss: 0.0784\n",
            "Epoch [19/30], Step [137/139], Loss: 0.0928\n",
            "Epoch [19/30], Step [138/139], Loss: 0.0599\n",
            "Epoch [19/30], Step [139/139], Loss: 0.0914\n",
            "Start validation #19\n",
            "Validation #19  Average Loss: 0.0875\n",
            "Best performance at epoch: 19\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [20/30], Step [1/139], Loss: 0.0622\n",
            "Epoch [20/30], Step [2/139], Loss: 0.0646\n",
            "Epoch [20/30], Step [3/139], Loss: 0.0793\n",
            "Epoch [20/30], Step [4/139], Loss: 0.0741\n",
            "Epoch [20/30], Step [5/139], Loss: 0.0751\n",
            "Epoch [20/30], Step [6/139], Loss: 0.0692\n",
            "Epoch [20/30], Step [7/139], Loss: 0.0832\n",
            "Epoch [20/30], Step [8/139], Loss: 0.1115\n",
            "Epoch [20/30], Step [9/139], Loss: 0.0840\n",
            "Epoch [20/30], Step [10/139], Loss: 0.0770\n",
            "Epoch [20/30], Step [11/139], Loss: 0.0672\n",
            "Epoch [20/30], Step [12/139], Loss: 0.0607\n",
            "Epoch [20/30], Step [13/139], Loss: 0.0804\n",
            "Epoch [20/30], Step [14/139], Loss: 0.0614\n",
            "Epoch [20/30], Step [15/139], Loss: 0.0806\n",
            "Epoch [20/30], Step [16/139], Loss: 0.0811\n",
            "Epoch [20/30], Step [17/139], Loss: 0.0718\n",
            "Epoch [20/30], Step [18/139], Loss: 0.0568\n",
            "Epoch [20/30], Step [19/139], Loss: 0.0672\n",
            "Epoch [20/30], Step [20/139], Loss: 0.0608\n",
            "Epoch [20/30], Step [21/139], Loss: 0.0884\n",
            "Epoch [20/30], Step [22/139], Loss: 0.0730\n",
            "Epoch [20/30], Step [23/139], Loss: 0.0854\n",
            "Epoch [20/30], Step [24/139], Loss: 0.0712\n",
            "Epoch [20/30], Step [25/139], Loss: 0.0730\n",
            "Epoch [20/30], Step [26/139], Loss: 0.1006\n",
            "Epoch [20/30], Step [27/139], Loss: 0.0994\n",
            "Epoch [20/30], Step [28/139], Loss: 0.0783\n",
            "Epoch [20/30], Step [29/139], Loss: 0.1118\n",
            "Epoch [20/30], Step [30/139], Loss: 0.0658\n",
            "Epoch [20/30], Step [31/139], Loss: 0.0981\n",
            "Epoch [20/30], Step [32/139], Loss: 0.0725\n",
            "Epoch [20/30], Step [33/139], Loss: 0.0749\n",
            "Epoch [20/30], Step [34/139], Loss: 0.1195\n",
            "Epoch [20/30], Step [35/139], Loss: 0.0867\n",
            "Epoch [20/30], Step [36/139], Loss: 0.0828\n",
            "Epoch [20/30], Step [37/139], Loss: 0.0883\n",
            "Epoch [20/30], Step [38/139], Loss: 0.0881\n",
            "Epoch [20/30], Step [39/139], Loss: 0.1081\n",
            "Epoch [20/30], Step [40/139], Loss: 0.0616\n",
            "Epoch [20/30], Step [41/139], Loss: 0.0758\n",
            "Epoch [20/30], Step [42/139], Loss: 0.0719\n",
            "Epoch [20/30], Step [43/139], Loss: 0.0821\n",
            "Epoch [20/30], Step [44/139], Loss: 0.0822\n",
            "Epoch [20/30], Step [45/139], Loss: 0.0939\n",
            "Epoch [20/30], Step [46/139], Loss: 0.0591\n",
            "Epoch [20/30], Step [47/139], Loss: 0.1068\n",
            "Epoch [20/30], Step [48/139], Loss: 0.0484\n",
            "Epoch [20/30], Step [49/139], Loss: 0.0661\n",
            "Epoch [20/30], Step [50/139], Loss: 0.0668\n",
            "Epoch [20/30], Step [51/139], Loss: 0.0822\n",
            "Epoch [20/30], Step [52/139], Loss: 0.0669\n",
            "Epoch [20/30], Step [53/139], Loss: 0.0599\n",
            "Epoch [20/30], Step [54/139], Loss: 0.0653\n",
            "Epoch [20/30], Step [55/139], Loss: 0.0516\n",
            "Epoch [20/30], Step [56/139], Loss: 0.0623\n",
            "Epoch [20/30], Step [57/139], Loss: 0.0849\n",
            "Epoch [20/30], Step [58/139], Loss: 0.0687\n",
            "Epoch [20/30], Step [59/139], Loss: 0.0871\n",
            "Epoch [20/30], Step [60/139], Loss: 0.0730\n",
            "Epoch [20/30], Step [61/139], Loss: 0.0646\n",
            "Epoch [20/30], Step [62/139], Loss: 0.0834\n",
            "Epoch [20/30], Step [63/139], Loss: 0.1033\n",
            "Epoch [20/30], Step [64/139], Loss: 0.0688\n",
            "Epoch [20/30], Step [65/139], Loss: 0.0785\n",
            "Epoch [20/30], Step [66/139], Loss: 0.0676\n",
            "Epoch [20/30], Step [67/139], Loss: 0.1113\n",
            "Epoch [20/30], Step [68/139], Loss: 0.0802\n",
            "Epoch [20/30], Step [69/139], Loss: 0.0545\n",
            "Epoch [20/30], Step [70/139], Loss: 0.0926\n",
            "Epoch [20/30], Step [71/139], Loss: 0.0765\n",
            "Epoch [20/30], Step [72/139], Loss: 0.0767\n",
            "Epoch [20/30], Step [73/139], Loss: 0.0942\n",
            "Epoch [20/30], Step [74/139], Loss: 0.0969\n",
            "Epoch [20/30], Step [75/139], Loss: 0.0749\n",
            "Epoch [20/30], Step [76/139], Loss: 0.0842\n",
            "Epoch [20/30], Step [77/139], Loss: 0.0765\n",
            "Epoch [20/30], Step [78/139], Loss: 0.0815\n",
            "Epoch [20/30], Step [79/139], Loss: 0.0676\n",
            "Epoch [20/30], Step [80/139], Loss: 0.0642\n",
            "Epoch [20/30], Step [81/139], Loss: 0.0745\n",
            "Epoch [20/30], Step [82/139], Loss: 0.0769\n",
            "Epoch [20/30], Step [83/139], Loss: 0.0847\n",
            "Epoch [20/30], Step [84/139], Loss: 0.0793\n",
            "Epoch [20/30], Step [85/139], Loss: 0.0789\n",
            "Epoch [20/30], Step [86/139], Loss: 0.0759\n",
            "Epoch [20/30], Step [87/139], Loss: 0.0836\n",
            "Epoch [20/30], Step [88/139], Loss: 0.0627\n",
            "Epoch [20/30], Step [89/139], Loss: 0.0803\n",
            "Epoch [20/30], Step [90/139], Loss: 0.0994\n",
            "Epoch [20/30], Step [91/139], Loss: 0.0853\n",
            "Epoch [20/30], Step [92/139], Loss: 0.0798\n",
            "Epoch [20/30], Step [93/139], Loss: 0.0791\n",
            "Epoch [20/30], Step [94/139], Loss: 0.0850\n",
            "Epoch [20/30], Step [95/139], Loss: 0.0521\n",
            "Epoch [20/30], Step [96/139], Loss: 0.0628\n",
            "Epoch [20/30], Step [97/139], Loss: 0.0930\n",
            "Epoch [20/30], Step [98/139], Loss: 0.0713\n",
            "Epoch [20/30], Step [99/139], Loss: 0.0746\n",
            "Epoch [20/30], Step [100/139], Loss: 0.0956\n",
            "Epoch [20/30], Step [101/139], Loss: 0.0738\n",
            "Epoch [20/30], Step [102/139], Loss: 0.0743\n",
            "Epoch [20/30], Step [103/139], Loss: 0.0795\n",
            "Epoch [20/30], Step [104/139], Loss: 0.0790\n",
            "Epoch [20/30], Step [105/139], Loss: 0.0534\n",
            "Epoch [20/30], Step [106/139], Loss: 0.1011\n",
            "Epoch [20/30], Step [107/139], Loss: 0.0959\n",
            "Epoch [20/30], Step [108/139], Loss: 0.0757\n",
            "Epoch [20/30], Step [109/139], Loss: 0.1029\n",
            "Epoch [20/30], Step [110/139], Loss: 0.0956\n",
            "Epoch [20/30], Step [111/139], Loss: 0.0915\n",
            "Epoch [20/30], Step [112/139], Loss: 0.0500\n",
            "Epoch [20/30], Step [113/139], Loss: 0.0886\n",
            "Epoch [20/30], Step [114/139], Loss: 0.0919\n",
            "Epoch [20/30], Step [115/139], Loss: 0.0838\n",
            "Epoch [20/30], Step [116/139], Loss: 0.0908\n",
            "Epoch [20/30], Step [117/139], Loss: 0.0807\n",
            "Epoch [20/30], Step [118/139], Loss: 0.0686\n",
            "Epoch [20/30], Step [119/139], Loss: 0.0708\n",
            "Epoch [20/30], Step [120/139], Loss: 0.0854\n",
            "Epoch [20/30], Step [121/139], Loss: 0.0659\n",
            "Epoch [20/30], Step [122/139], Loss: 0.0999\n",
            "Epoch [20/30], Step [123/139], Loss: 0.0858\n",
            "Epoch [20/30], Step [124/139], Loss: 0.0951\n",
            "Epoch [20/30], Step [125/139], Loss: 0.0455\n",
            "Epoch [20/30], Step [126/139], Loss: 0.0932\n",
            "Epoch [20/30], Step [127/139], Loss: 0.1000\n",
            "Epoch [20/30], Step [128/139], Loss: 0.0779\n",
            "Epoch [20/30], Step [129/139], Loss: 0.0573\n",
            "Epoch [20/30], Step [130/139], Loss: 0.0675\n",
            "Epoch [20/30], Step [131/139], Loss: 0.0679\n",
            "Epoch [20/30], Step [132/139], Loss: 0.0715\n",
            "Epoch [20/30], Step [133/139], Loss: 0.0851\n",
            "Epoch [20/30], Step [134/139], Loss: 0.0877\n",
            "Epoch [20/30], Step [135/139], Loss: 0.0831\n",
            "Epoch [20/30], Step [136/139], Loss: 0.0810\n",
            "Epoch [20/30], Step [137/139], Loss: 0.0768\n",
            "Epoch [20/30], Step [138/139], Loss: 0.0883\n",
            "Epoch [20/30], Step [139/139], Loss: 0.1001\n",
            "Start validation #20\n",
            "Validation #20  Average Loss: 0.0861\n",
            "Best performance at epoch: 20\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [21/30], Step [1/139], Loss: 0.0799\n",
            "Epoch [21/30], Step [2/139], Loss: 0.0841\n",
            "Epoch [21/30], Step [3/139], Loss: 0.0826\n",
            "Epoch [21/30], Step [4/139], Loss: 0.0740\n",
            "Epoch [21/30], Step [5/139], Loss: 0.0631\n",
            "Epoch [21/30], Step [6/139], Loss: 0.1153\n",
            "Epoch [21/30], Step [7/139], Loss: 0.0620\n",
            "Epoch [21/30], Step [8/139], Loss: 0.0900\n",
            "Epoch [21/30], Step [9/139], Loss: 0.0588\n",
            "Epoch [21/30], Step [10/139], Loss: 0.0704\n",
            "Epoch [21/30], Step [11/139], Loss: 0.0874\n",
            "Epoch [21/30], Step [12/139], Loss: 0.0763\n",
            "Epoch [21/30], Step [13/139], Loss: 0.0801\n",
            "Epoch [21/30], Step [14/139], Loss: 0.0906\n",
            "Epoch [21/30], Step [15/139], Loss: 0.0580\n",
            "Epoch [21/30], Step [16/139], Loss: 0.0758\n",
            "Epoch [21/30], Step [17/139], Loss: 0.1029\n",
            "Epoch [21/30], Step [18/139], Loss: 0.0570\n",
            "Epoch [21/30], Step [19/139], Loss: 0.0622\n",
            "Epoch [21/30], Step [20/139], Loss: 0.0650\n",
            "Epoch [21/30], Step [21/139], Loss: 0.1056\n",
            "Epoch [21/30], Step [22/139], Loss: 0.0755\n",
            "Epoch [21/30], Step [23/139], Loss: 0.0614\n",
            "Epoch [21/30], Step [24/139], Loss: 0.0901\n",
            "Epoch [21/30], Step [25/139], Loss: 0.0698\n",
            "Epoch [21/30], Step [26/139], Loss: 0.0649\n",
            "Epoch [21/30], Step [27/139], Loss: 0.0769\n",
            "Epoch [21/30], Step [28/139], Loss: 0.0914\n",
            "Epoch [21/30], Step [29/139], Loss: 0.0729\n",
            "Epoch [21/30], Step [30/139], Loss: 0.0651\n",
            "Epoch [21/30], Step [31/139], Loss: 0.0817\n",
            "Epoch [21/30], Step [32/139], Loss: 0.0575\n",
            "Epoch [21/30], Step [33/139], Loss: 0.0817\n",
            "Epoch [21/30], Step [34/139], Loss: 0.0650\n",
            "Epoch [21/30], Step [35/139], Loss: 0.0764\n",
            "Epoch [21/30], Step [36/139], Loss: 0.0748\n",
            "Epoch [21/30], Step [37/139], Loss: 0.0580\n",
            "Epoch [21/30], Step [38/139], Loss: 0.0868\n",
            "Epoch [21/30], Step [39/139], Loss: 0.0845\n",
            "Epoch [21/30], Step [40/139], Loss: 0.0694\n",
            "Epoch [21/30], Step [41/139], Loss: 0.0673\n",
            "Epoch [21/30], Step [42/139], Loss: 0.0750\n",
            "Epoch [21/30], Step [43/139], Loss: 0.0755\n",
            "Epoch [21/30], Step [44/139], Loss: 0.0699\n",
            "Epoch [21/30], Step [45/139], Loss: 0.0669\n",
            "Epoch [21/30], Step [46/139], Loss: 0.0589\n",
            "Epoch [21/30], Step [47/139], Loss: 0.0843\n",
            "Epoch [21/30], Step [48/139], Loss: 0.0822\n",
            "Epoch [21/30], Step [49/139], Loss: 0.0877\n",
            "Epoch [21/30], Step [50/139], Loss: 0.0616\n",
            "Epoch [21/30], Step [51/139], Loss: 0.0692\n",
            "Epoch [21/30], Step [52/139], Loss: 0.0704\n",
            "Epoch [21/30], Step [53/139], Loss: 0.0863\n",
            "Epoch [21/30], Step [54/139], Loss: 0.0789\n",
            "Epoch [21/30], Step [55/139], Loss: 0.0861\n",
            "Epoch [21/30], Step [56/139], Loss: 0.0731\n",
            "Epoch [21/30], Step [57/139], Loss: 0.1085\n",
            "Epoch [21/30], Step [58/139], Loss: 0.0874\n",
            "Epoch [21/30], Step [59/139], Loss: 0.0889\n",
            "Epoch [21/30], Step [60/139], Loss: 0.0913\n",
            "Epoch [21/30], Step [61/139], Loss: 0.0805\n",
            "Epoch [21/30], Step [62/139], Loss: 0.0759\n",
            "Epoch [21/30], Step [63/139], Loss: 0.0882\n",
            "Epoch [21/30], Step [64/139], Loss: 0.0547\n",
            "Epoch [21/30], Step [65/139], Loss: 0.0591\n",
            "Epoch [21/30], Step [66/139], Loss: 0.0628\n",
            "Epoch [21/30], Step [67/139], Loss: 0.0834\n",
            "Epoch [21/30], Step [68/139], Loss: 0.0685\n",
            "Epoch [21/30], Step [69/139], Loss: 0.0888\n",
            "Epoch [21/30], Step [70/139], Loss: 0.0724\n",
            "Epoch [21/30], Step [71/139], Loss: 0.0730\n",
            "Epoch [21/30], Step [72/139], Loss: 0.0793\n",
            "Epoch [21/30], Step [73/139], Loss: 0.0799\n",
            "Epoch [21/30], Step [74/139], Loss: 0.0706\n",
            "Epoch [21/30], Step [75/139], Loss: 0.0954\n",
            "Epoch [21/30], Step [76/139], Loss: 0.0829\n",
            "Epoch [21/30], Step [77/139], Loss: 0.1019\n",
            "Epoch [21/30], Step [78/139], Loss: 0.1260\n",
            "Epoch [21/30], Step [79/139], Loss: 0.0665\n",
            "Epoch [21/30], Step [80/139], Loss: 0.0962\n",
            "Epoch [21/30], Step [81/139], Loss: 0.0563\n",
            "Epoch [21/30], Step [82/139], Loss: 0.0508\n",
            "Epoch [21/30], Step [83/139], Loss: 0.0693\n",
            "Epoch [21/30], Step [84/139], Loss: 0.0767\n",
            "Epoch [21/30], Step [85/139], Loss: 0.0569\n",
            "Epoch [21/30], Step [86/139], Loss: 0.0870\n",
            "Epoch [21/30], Step [87/139], Loss: 0.0931\n",
            "Epoch [21/30], Step [88/139], Loss: 0.0666\n",
            "Epoch [21/30], Step [89/139], Loss: 0.0746\n",
            "Epoch [21/30], Step [90/139], Loss: 0.0673\n",
            "Epoch [21/30], Step [91/139], Loss: 0.0540\n",
            "Epoch [21/30], Step [92/139], Loss: 0.0810\n",
            "Epoch [21/30], Step [93/139], Loss: 0.0764\n",
            "Epoch [21/30], Step [94/139], Loss: 0.0653\n",
            "Epoch [21/30], Step [95/139], Loss: 0.0833\n",
            "Epoch [21/30], Step [96/139], Loss: 0.0760\n",
            "Epoch [21/30], Step [97/139], Loss: 0.0925\n",
            "Epoch [21/30], Step [98/139], Loss: 0.0764\n",
            "Epoch [21/30], Step [99/139], Loss: 0.0592\n",
            "Epoch [21/30], Step [100/139], Loss: 0.0911\n",
            "Epoch [21/30], Step [101/139], Loss: 0.0979\n",
            "Epoch [21/30], Step [102/139], Loss: 0.0844\n",
            "Epoch [21/30], Step [103/139], Loss: 0.0716\n",
            "Epoch [21/30], Step [104/139], Loss: 0.1035\n",
            "Epoch [21/30], Step [105/139], Loss: 0.0764\n",
            "Epoch [21/30], Step [106/139], Loss: 0.0961\n",
            "Epoch [21/30], Step [107/139], Loss: 0.0653\n",
            "Epoch [21/30], Step [108/139], Loss: 0.0846\n",
            "Epoch [21/30], Step [109/139], Loss: 0.0798\n",
            "Epoch [21/30], Step [110/139], Loss: 0.0571\n",
            "Epoch [21/30], Step [111/139], Loss: 0.0759\n",
            "Epoch [21/30], Step [112/139], Loss: 0.0839\n",
            "Epoch [21/30], Step [113/139], Loss: 0.1122\n",
            "Epoch [21/30], Step [114/139], Loss: 0.0775\n",
            "Epoch [21/30], Step [115/139], Loss: 0.0619\n",
            "Epoch [21/30], Step [116/139], Loss: 0.0858\n",
            "Epoch [21/30], Step [117/139], Loss: 0.0820\n",
            "Epoch [21/30], Step [118/139], Loss: 0.0842\n",
            "Epoch [21/30], Step [119/139], Loss: 0.0555\n",
            "Epoch [21/30], Step [120/139], Loss: 0.1235\n",
            "Epoch [21/30], Step [121/139], Loss: 0.0758\n",
            "Epoch [21/30], Step [122/139], Loss: 0.0662\n",
            "Epoch [21/30], Step [123/139], Loss: 0.0870\n",
            "Epoch [21/30], Step [124/139], Loss: 0.0839\n",
            "Epoch [21/30], Step [125/139], Loss: 0.0628\n",
            "Epoch [21/30], Step [126/139], Loss: 0.0886\n",
            "Epoch [21/30], Step [127/139], Loss: 0.0596\n",
            "Epoch [21/30], Step [128/139], Loss: 0.0829\n",
            "Epoch [21/30], Step [129/139], Loss: 0.0723\n",
            "Epoch [21/30], Step [130/139], Loss: 0.0859\n",
            "Epoch [21/30], Step [131/139], Loss: 0.0743\n",
            "Epoch [21/30], Step [132/139], Loss: 0.0805\n",
            "Epoch [21/30], Step [133/139], Loss: 0.1009\n",
            "Epoch [21/30], Step [134/139], Loss: 0.0911\n",
            "Epoch [21/30], Step [135/139], Loss: 0.0867\n",
            "Epoch [21/30], Step [136/139], Loss: 0.0760\n",
            "Epoch [21/30], Step [137/139], Loss: 0.0625\n",
            "Epoch [21/30], Step [138/139], Loss: 0.0931\n",
            "Epoch [21/30], Step [139/139], Loss: 0.0813\n",
            "Start validation #21\n",
            "Validation #21  Average Loss: 0.0856\n",
            "Best performance at epoch: 21\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [22/30], Step [1/139], Loss: 0.0727\n",
            "Epoch [22/30], Step [2/139], Loss: 0.0812\n",
            "Epoch [22/30], Step [3/139], Loss: 0.0756\n",
            "Epoch [22/30], Step [4/139], Loss: 0.0703\n",
            "Epoch [22/30], Step [5/139], Loss: 0.0865\n",
            "Epoch [22/30], Step [6/139], Loss: 0.1006\n",
            "Epoch [22/30], Step [7/139], Loss: 0.0633\n",
            "Epoch [22/30], Step [8/139], Loss: 0.0680\n",
            "Epoch [22/30], Step [9/139], Loss: 0.0709\n",
            "Epoch [22/30], Step [10/139], Loss: 0.0532\n",
            "Epoch [22/30], Step [11/139], Loss: 0.0752\n",
            "Epoch [22/30], Step [12/139], Loss: 0.0638\n",
            "Epoch [22/30], Step [13/139], Loss: 0.0805\n",
            "Epoch [22/30], Step [14/139], Loss: 0.0800\n",
            "Epoch [22/30], Step [15/139], Loss: 0.0721\n",
            "Epoch [22/30], Step [16/139], Loss: 0.0573\n",
            "Epoch [22/30], Step [17/139], Loss: 0.0744\n",
            "Epoch [22/30], Step [18/139], Loss: 0.0723\n",
            "Epoch [22/30], Step [19/139], Loss: 0.0734\n",
            "Epoch [22/30], Step [20/139], Loss: 0.0817\n",
            "Epoch [22/30], Step [21/139], Loss: 0.0697\n",
            "Epoch [22/30], Step [22/139], Loss: 0.0962\n",
            "Epoch [22/30], Step [23/139], Loss: 0.0858\n",
            "Epoch [22/30], Step [24/139], Loss: 0.0888\n",
            "Epoch [22/30], Step [25/139], Loss: 0.0841\n",
            "Epoch [22/30], Step [26/139], Loss: 0.0774\n",
            "Epoch [22/30], Step [27/139], Loss: 0.0689\n",
            "Epoch [22/30], Step [28/139], Loss: 0.0804\n",
            "Epoch [22/30], Step [29/139], Loss: 0.0638\n",
            "Epoch [22/30], Step [30/139], Loss: 0.0569\n",
            "Epoch [22/30], Step [31/139], Loss: 0.0749\n",
            "Epoch [22/30], Step [32/139], Loss: 0.0862\n",
            "Epoch [22/30], Step [33/139], Loss: 0.1121\n",
            "Epoch [22/30], Step [34/139], Loss: 0.0715\n",
            "Epoch [22/30], Step [35/139], Loss: 0.0942\n",
            "Epoch [22/30], Step [36/139], Loss: 0.0445\n",
            "Epoch [22/30], Step [37/139], Loss: 0.0670\n",
            "Epoch [22/30], Step [38/139], Loss: 0.1030\n",
            "Epoch [22/30], Step [39/139], Loss: 0.0808\n",
            "Epoch [22/30], Step [40/139], Loss: 0.0839\n",
            "Epoch [22/30], Step [41/139], Loss: 0.0906\n",
            "Epoch [22/30], Step [42/139], Loss: 0.0948\n",
            "Epoch [22/30], Step [43/139], Loss: 0.0709\n",
            "Epoch [22/30], Step [44/139], Loss: 0.0760\n",
            "Epoch [22/30], Step [45/139], Loss: 0.1010\n",
            "Epoch [22/30], Step [46/139], Loss: 0.0754\n",
            "Epoch [22/30], Step [47/139], Loss: 0.0889\n",
            "Epoch [22/30], Step [48/139], Loss: 0.0751\n",
            "Epoch [22/30], Step [49/139], Loss: 0.0976\n",
            "Epoch [22/30], Step [50/139], Loss: 0.0826\n",
            "Epoch [22/30], Step [51/139], Loss: 0.1160\n",
            "Epoch [22/30], Step [52/139], Loss: 0.0799\n",
            "Epoch [22/30], Step [53/139], Loss: 0.0837\n",
            "Epoch [22/30], Step [54/139], Loss: 0.0743\n",
            "Epoch [22/30], Step [55/139], Loss: 0.0667\n",
            "Epoch [22/30], Step [56/139], Loss: 0.0651\n",
            "Epoch [22/30], Step [57/139], Loss: 0.0894\n",
            "Epoch [22/30], Step [58/139], Loss: 0.0769\n",
            "Epoch [22/30], Step [59/139], Loss: 0.0786\n",
            "Epoch [22/30], Step [60/139], Loss: 0.0738\n",
            "Epoch [22/30], Step [61/139], Loss: 0.0797\n",
            "Epoch [22/30], Step [62/139], Loss: 0.0627\n",
            "Epoch [22/30], Step [63/139], Loss: 0.0663\n",
            "Epoch [22/30], Step [64/139], Loss: 0.0848\n",
            "Epoch [22/30], Step [65/139], Loss: 0.0681\n",
            "Epoch [22/30], Step [66/139], Loss: 0.0648\n",
            "Epoch [22/30], Step [67/139], Loss: 0.0975\n",
            "Epoch [22/30], Step [68/139], Loss: 0.0810\n",
            "Epoch [22/30], Step [69/139], Loss: 0.0942\n",
            "Epoch [22/30], Step [70/139], Loss: 0.0926\n",
            "Epoch [22/30], Step [71/139], Loss: 0.0751\n",
            "Epoch [22/30], Step [72/139], Loss: 0.0654\n",
            "Epoch [22/30], Step [73/139], Loss: 0.0930\n",
            "Epoch [22/30], Step [74/139], Loss: 0.0852\n",
            "Epoch [22/30], Step [75/139], Loss: 0.0817\n",
            "Epoch [22/30], Step [76/139], Loss: 0.0906\n",
            "Epoch [22/30], Step [77/139], Loss: 0.0799\n",
            "Epoch [22/30], Step [78/139], Loss: 0.0643\n",
            "Epoch [22/30], Step [79/139], Loss: 0.0646\n",
            "Epoch [22/30], Step [80/139], Loss: 0.0714\n",
            "Epoch [22/30], Step [81/139], Loss: 0.0698\n",
            "Epoch [22/30], Step [82/139], Loss: 0.0865\n",
            "Epoch [22/30], Step [83/139], Loss: 0.0604\n",
            "Epoch [22/30], Step [84/139], Loss: 0.0791\n",
            "Epoch [22/30], Step [85/139], Loss: 0.0737\n",
            "Epoch [22/30], Step [86/139], Loss: 0.0818\n",
            "Epoch [22/30], Step [87/139], Loss: 0.0611\n",
            "Epoch [22/30], Step [88/139], Loss: 0.0978\n",
            "Epoch [22/30], Step [89/139], Loss: 0.0650\n",
            "Epoch [22/30], Step [90/139], Loss: 0.0794\n",
            "Epoch [22/30], Step [91/139], Loss: 0.0650\n",
            "Epoch [22/30], Step [92/139], Loss: 0.0916\n",
            "Epoch [22/30], Step [93/139], Loss: 0.0778\n",
            "Epoch [22/30], Step [94/139], Loss: 0.0889\n",
            "Epoch [22/30], Step [95/139], Loss: 0.0623\n",
            "Epoch [22/30], Step [96/139], Loss: 0.0896\n",
            "Epoch [22/30], Step [97/139], Loss: 0.0898\n",
            "Epoch [22/30], Step [98/139], Loss: 0.0611\n",
            "Epoch [22/30], Step [99/139], Loss: 0.0818\n",
            "Epoch [22/30], Step [100/139], Loss: 0.0744\n",
            "Epoch [22/30], Step [101/139], Loss: 0.0586\n",
            "Epoch [22/30], Step [102/139], Loss: 0.0854\n",
            "Epoch [22/30], Step [103/139], Loss: 0.0603\n",
            "Epoch [22/30], Step [104/139], Loss: 0.0652\n",
            "Epoch [22/30], Step [105/139], Loss: 0.0896\n",
            "Epoch [22/30], Step [106/139], Loss: 0.0705\n",
            "Epoch [22/30], Step [107/139], Loss: 0.0521\n",
            "Epoch [22/30], Step [108/139], Loss: 0.0601\n",
            "Epoch [22/30], Step [109/139], Loss: 0.0495\n",
            "Epoch [22/30], Step [110/139], Loss: 0.1022\n",
            "Epoch [22/30], Step [111/139], Loss: 0.0656\n",
            "Epoch [22/30], Step [112/139], Loss: 0.0749\n",
            "Epoch [22/30], Step [113/139], Loss: 0.0728\n",
            "Epoch [22/30], Step [114/139], Loss: 0.0739\n",
            "Epoch [22/30], Step [115/139], Loss: 0.0838\n",
            "Epoch [22/30], Step [116/139], Loss: 0.0475\n",
            "Epoch [22/30], Step [117/139], Loss: 0.0619\n",
            "Epoch [22/30], Step [118/139], Loss: 0.0616\n",
            "Epoch [22/30], Step [119/139], Loss: 0.0547\n",
            "Epoch [22/30], Step [120/139], Loss: 0.1037\n",
            "Epoch [22/30], Step [121/139], Loss: 0.1021\n",
            "Epoch [22/30], Step [122/139], Loss: 0.0862\n",
            "Epoch [22/30], Step [123/139], Loss: 0.0743\n",
            "Epoch [22/30], Step [124/139], Loss: 0.0810\n",
            "Epoch [22/30], Step [125/139], Loss: 0.0872\n",
            "Epoch [22/30], Step [126/139], Loss: 0.0762\n",
            "Epoch [22/30], Step [127/139], Loss: 0.0932\n",
            "Epoch [22/30], Step [128/139], Loss: 0.0939\n",
            "Epoch [22/30], Step [129/139], Loss: 0.0989\n",
            "Epoch [22/30], Step [130/139], Loss: 0.0749\n",
            "Epoch [22/30], Step [131/139], Loss: 0.0696\n",
            "Epoch [22/30], Step [132/139], Loss: 0.0935\n",
            "Epoch [22/30], Step [133/139], Loss: 0.0649\n",
            "Epoch [22/30], Step [134/139], Loss: 0.0654\n",
            "Epoch [22/30], Step [135/139], Loss: 0.0749\n",
            "Epoch [22/30], Step [136/139], Loss: 0.0932\n",
            "Epoch [22/30], Step [137/139], Loss: 0.0422\n",
            "Epoch [22/30], Step [138/139], Loss: 0.0776\n",
            "Epoch [22/30], Step [139/139], Loss: 0.0660\n",
            "Start validation #22\n",
            "Validation #22  Average Loss: 0.0848\n",
            "Best performance at epoch: 22\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [23/30], Step [1/139], Loss: 0.0818\n",
            "Epoch [23/30], Step [2/139], Loss: 0.0864\n",
            "Epoch [23/30], Step [3/139], Loss: 0.0667\n",
            "Epoch [23/30], Step [4/139], Loss: 0.0854\n",
            "Epoch [23/30], Step [5/139], Loss: 0.0967\n",
            "Epoch [23/30], Step [6/139], Loss: 0.0693\n",
            "Epoch [23/30], Step [7/139], Loss: 0.0729\n",
            "Epoch [23/30], Step [8/139], Loss: 0.0735\n",
            "Epoch [23/30], Step [9/139], Loss: 0.0635\n",
            "Epoch [23/30], Step [10/139], Loss: 0.0667\n",
            "Epoch [23/30], Step [11/139], Loss: 0.0818\n",
            "Epoch [23/30], Step [12/139], Loss: 0.0906\n",
            "Epoch [23/30], Step [13/139], Loss: 0.0663\n",
            "Epoch [23/30], Step [14/139], Loss: 0.0683\n",
            "Epoch [23/30], Step [15/139], Loss: 0.0733\n",
            "Epoch [23/30], Step [16/139], Loss: 0.0747\n",
            "Epoch [23/30], Step [17/139], Loss: 0.0671\n",
            "Epoch [23/30], Step [18/139], Loss: 0.0508\n",
            "Epoch [23/30], Step [19/139], Loss: 0.0660\n",
            "Epoch [23/30], Step [20/139], Loss: 0.0670\n",
            "Epoch [23/30], Step [21/139], Loss: 0.0743\n",
            "Epoch [23/30], Step [22/139], Loss: 0.0716\n",
            "Epoch [23/30], Step [23/139], Loss: 0.0724\n",
            "Epoch [23/30], Step [24/139], Loss: 0.0763\n",
            "Epoch [23/30], Step [25/139], Loss: 0.1025\n",
            "Epoch [23/30], Step [26/139], Loss: 0.0633\n",
            "Epoch [23/30], Step [27/139], Loss: 0.0955\n",
            "Epoch [23/30], Step [28/139], Loss: 0.0867\n",
            "Epoch [23/30], Step [29/139], Loss: 0.0735\n",
            "Epoch [23/30], Step [30/139], Loss: 0.0552\n",
            "Epoch [23/30], Step [31/139], Loss: 0.0813\n",
            "Epoch [23/30], Step [32/139], Loss: 0.0728\n",
            "Epoch [23/30], Step [33/139], Loss: 0.0695\n",
            "Epoch [23/30], Step [34/139], Loss: 0.0731\n",
            "Epoch [23/30], Step [35/139], Loss: 0.0886\n",
            "Epoch [23/30], Step [36/139], Loss: 0.0780\n",
            "Epoch [23/30], Step [37/139], Loss: 0.0762\n",
            "Epoch [23/30], Step [38/139], Loss: 0.0666\n",
            "Epoch [23/30], Step [39/139], Loss: 0.0720\n",
            "Epoch [23/30], Step [40/139], Loss: 0.0788\n",
            "Epoch [23/30], Step [41/139], Loss: 0.0851\n",
            "Epoch [23/30], Step [42/139], Loss: 0.0499\n",
            "Epoch [23/30], Step [43/139], Loss: 0.0695\n",
            "Epoch [23/30], Step [44/139], Loss: 0.1111\n",
            "Epoch [23/30], Step [45/139], Loss: 0.0713\n",
            "Epoch [23/30], Step [46/139], Loss: 0.0936\n",
            "Epoch [23/30], Step [47/139], Loss: 0.0556\n",
            "Epoch [23/30], Step [48/139], Loss: 0.1123\n",
            "Epoch [23/30], Step [49/139], Loss: 0.0708\n",
            "Epoch [23/30], Step [50/139], Loss: 0.0608\n",
            "Epoch [23/30], Step [51/139], Loss: 0.0694\n",
            "Epoch [23/30], Step [52/139], Loss: 0.0634\n",
            "Epoch [23/30], Step [53/139], Loss: 0.0558\n",
            "Epoch [23/30], Step [54/139], Loss: 0.0775\n",
            "Epoch [23/30], Step [55/139], Loss: 0.0896\n",
            "Epoch [23/30], Step [56/139], Loss: 0.0815\n",
            "Epoch [23/30], Step [57/139], Loss: 0.0767\n",
            "Epoch [23/30], Step [58/139], Loss: 0.0577\n",
            "Epoch [23/30], Step [59/139], Loss: 0.1131\n",
            "Epoch [23/30], Step [60/139], Loss: 0.0693\n",
            "Epoch [23/30], Step [61/139], Loss: 0.0741\n",
            "Epoch [23/30], Step [62/139], Loss: 0.0734\n",
            "Epoch [23/30], Step [63/139], Loss: 0.0754\n",
            "Epoch [23/30], Step [64/139], Loss: 0.0915\n",
            "Epoch [23/30], Step [65/139], Loss: 0.0686\n",
            "Epoch [23/30], Step [66/139], Loss: 0.0754\n",
            "Epoch [23/30], Step [67/139], Loss: 0.0750\n",
            "Epoch [23/30], Step [68/139], Loss: 0.1077\n",
            "Epoch [23/30], Step [69/139], Loss: 0.0860\n",
            "Epoch [23/30], Step [70/139], Loss: 0.0504\n",
            "Epoch [23/30], Step [71/139], Loss: 0.0683\n",
            "Epoch [23/30], Step [72/139], Loss: 0.0876\n",
            "Epoch [23/30], Step [73/139], Loss: 0.0612\n",
            "Epoch [23/30], Step [74/139], Loss: 0.0643\n",
            "Epoch [23/30], Step [75/139], Loss: 0.0998\n",
            "Epoch [23/30], Step [76/139], Loss: 0.0705\n",
            "Epoch [23/30], Step [77/139], Loss: 0.0747\n",
            "Epoch [23/30], Step [78/139], Loss: 0.1089\n",
            "Epoch [23/30], Step [79/139], Loss: 0.0766\n",
            "Epoch [23/30], Step [80/139], Loss: 0.0879\n",
            "Epoch [23/30], Step [81/139], Loss: 0.0630\n",
            "Epoch [23/30], Step [82/139], Loss: 0.0772\n",
            "Epoch [23/30], Step [83/139], Loss: 0.0695\n",
            "Epoch [23/30], Step [84/139], Loss: 0.0781\n",
            "Epoch [23/30], Step [85/139], Loss: 0.0574\n",
            "Epoch [23/30], Step [86/139], Loss: 0.1005\n",
            "Epoch [23/30], Step [87/139], Loss: 0.0656\n",
            "Epoch [23/30], Step [88/139], Loss: 0.0812\n",
            "Epoch [23/30], Step [89/139], Loss: 0.0669\n",
            "Epoch [23/30], Step [90/139], Loss: 0.1062\n",
            "Epoch [23/30], Step [91/139], Loss: 0.0663\n",
            "Epoch [23/30], Step [92/139], Loss: 0.0829\n",
            "Epoch [23/30], Step [93/139], Loss: 0.0818\n",
            "Epoch [23/30], Step [94/139], Loss: 0.0710\n",
            "Epoch [23/30], Step [95/139], Loss: 0.0992\n",
            "Epoch [23/30], Step [96/139], Loss: 0.0803\n",
            "Epoch [23/30], Step [97/139], Loss: 0.0641\n",
            "Epoch [23/30], Step [98/139], Loss: 0.0781\n",
            "Epoch [23/30], Step [99/139], Loss: 0.0600\n",
            "Epoch [23/30], Step [100/139], Loss: 0.0945\n",
            "Epoch [23/30], Step [101/139], Loss: 0.0708\n",
            "Epoch [23/30], Step [102/139], Loss: 0.0920\n",
            "Epoch [23/30], Step [103/139], Loss: 0.0886\n",
            "Epoch [23/30], Step [104/139], Loss: 0.0715\n",
            "Epoch [23/30], Step [105/139], Loss: 0.0736\n",
            "Epoch [23/30], Step [106/139], Loss: 0.0821\n",
            "Epoch [23/30], Step [107/139], Loss: 0.0755\n",
            "Epoch [23/30], Step [108/139], Loss: 0.0833\n",
            "Epoch [23/30], Step [109/139], Loss: 0.0586\n",
            "Epoch [23/30], Step [110/139], Loss: 0.0598\n",
            "Epoch [23/30], Step [111/139], Loss: 0.0481\n",
            "Epoch [23/30], Step [112/139], Loss: 0.0912\n",
            "Epoch [23/30], Step [113/139], Loss: 0.0770\n",
            "Epoch [23/30], Step [114/139], Loss: 0.0706\n",
            "Epoch [23/30], Step [115/139], Loss: 0.0615\n",
            "Epoch [23/30], Step [116/139], Loss: 0.0626\n",
            "Epoch [23/30], Step [117/139], Loss: 0.0700\n",
            "Epoch [23/30], Step [118/139], Loss: 0.0687\n",
            "Epoch [23/30], Step [119/139], Loss: 0.0915\n",
            "Epoch [23/30], Step [120/139], Loss: 0.0877\n",
            "Epoch [23/30], Step [121/139], Loss: 0.0487\n",
            "Epoch [23/30], Step [122/139], Loss: 0.1136\n",
            "Epoch [23/30], Step [123/139], Loss: 0.0598\n",
            "Epoch [23/30], Step [124/139], Loss: 0.0717\n",
            "Epoch [23/30], Step [125/139], Loss: 0.1020\n",
            "Epoch [23/30], Step [126/139], Loss: 0.1107\n",
            "Epoch [23/30], Step [127/139], Loss: 0.0681\n",
            "Epoch [23/30], Step [128/139], Loss: 0.0395\n",
            "Epoch [23/30], Step [129/139], Loss: 0.0885\n",
            "Epoch [23/30], Step [130/139], Loss: 0.0824\n",
            "Epoch [23/30], Step [131/139], Loss: 0.0764\n",
            "Epoch [23/30], Step [132/139], Loss: 0.0573\n",
            "Epoch [23/30], Step [133/139], Loss: 0.0733\n",
            "Epoch [23/30], Step [134/139], Loss: 0.0924\n",
            "Epoch [23/30], Step [135/139], Loss: 0.0604\n",
            "Epoch [23/30], Step [136/139], Loss: 0.0995\n",
            "Epoch [23/30], Step [137/139], Loss: 0.0852\n",
            "Epoch [23/30], Step [138/139], Loss: 0.0652\n",
            "Epoch [23/30], Step [139/139], Loss: 0.0810\n",
            "Start validation #23\n",
            "Validation #23  Average Loss: 0.0840\n",
            "Best performance at epoch: 23\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [24/30], Step [1/139], Loss: 0.0951\n",
            "Epoch [24/30], Step [2/139], Loss: 0.0607\n",
            "Epoch [24/30], Step [3/139], Loss: 0.0769\n",
            "Epoch [24/30], Step [4/139], Loss: 0.0543\n",
            "Epoch [24/30], Step [5/139], Loss: 0.0601\n",
            "Epoch [24/30], Step [6/139], Loss: 0.1159\n",
            "Epoch [24/30], Step [7/139], Loss: 0.1088\n",
            "Epoch [24/30], Step [8/139], Loss: 0.0706\n",
            "Epoch [24/30], Step [9/139], Loss: 0.0735\n",
            "Epoch [24/30], Step [10/139], Loss: 0.0606\n",
            "Epoch [24/30], Step [11/139], Loss: 0.0815\n",
            "Epoch [24/30], Step [12/139], Loss: 0.0962\n",
            "Epoch [24/30], Step [13/139], Loss: 0.0825\n",
            "Epoch [24/30], Step [14/139], Loss: 0.0608\n",
            "Epoch [24/30], Step [15/139], Loss: 0.0575\n",
            "Epoch [24/30], Step [16/139], Loss: 0.0486\n",
            "Epoch [24/30], Step [17/139], Loss: 0.0496\n",
            "Epoch [24/30], Step [18/139], Loss: 0.0679\n",
            "Epoch [24/30], Step [19/139], Loss: 0.1038\n",
            "Epoch [24/30], Step [20/139], Loss: 0.0751\n",
            "Epoch [24/30], Step [21/139], Loss: 0.0783\n",
            "Epoch [24/30], Step [22/139], Loss: 0.0723\n",
            "Epoch [24/30], Step [23/139], Loss: 0.0813\n",
            "Epoch [24/30], Step [24/139], Loss: 0.0882\n",
            "Epoch [24/30], Step [25/139], Loss: 0.0637\n",
            "Epoch [24/30], Step [26/139], Loss: 0.0639\n",
            "Epoch [24/30], Step [27/139], Loss: 0.0676\n",
            "Epoch [24/30], Step [28/139], Loss: 0.0627\n",
            "Epoch [24/30], Step [29/139], Loss: 0.0643\n",
            "Epoch [24/30], Step [30/139], Loss: 0.0704\n",
            "Epoch [24/30], Step [31/139], Loss: 0.0668\n",
            "Epoch [24/30], Step [32/139], Loss: 0.0874\n",
            "Epoch [24/30], Step [33/139], Loss: 0.0759\n",
            "Epoch [24/30], Step [34/139], Loss: 0.0565\n",
            "Epoch [24/30], Step [35/139], Loss: 0.0928\n",
            "Epoch [24/30], Step [36/139], Loss: 0.0933\n",
            "Epoch [24/30], Step [37/139], Loss: 0.0910\n",
            "Epoch [24/30], Step [38/139], Loss: 0.1007\n",
            "Epoch [24/30], Step [39/139], Loss: 0.0642\n",
            "Epoch [24/30], Step [40/139], Loss: 0.0746\n",
            "Epoch [24/30], Step [41/139], Loss: 0.0589\n",
            "Epoch [24/30], Step [42/139], Loss: 0.1276\n",
            "Epoch [24/30], Step [43/139], Loss: 0.0745\n",
            "Epoch [24/30], Step [44/139], Loss: 0.0781\n",
            "Epoch [24/30], Step [45/139], Loss: 0.0662\n",
            "Epoch [24/30], Step [46/139], Loss: 0.0888\n",
            "Epoch [24/30], Step [47/139], Loss: 0.0700\n",
            "Epoch [24/30], Step [48/139], Loss: 0.0635\n",
            "Epoch [24/30], Step [49/139], Loss: 0.0597\n",
            "Epoch [24/30], Step [50/139], Loss: 0.0520\n",
            "Epoch [24/30], Step [51/139], Loss: 0.0592\n",
            "Epoch [24/30], Step [52/139], Loss: 0.0910\n",
            "Epoch [24/30], Step [53/139], Loss: 0.0599\n",
            "Epoch [24/30], Step [54/139], Loss: 0.0805\n",
            "Epoch [24/30], Step [55/139], Loss: 0.0718\n",
            "Epoch [24/30], Step [56/139], Loss: 0.0932\n",
            "Epoch [24/30], Step [57/139], Loss: 0.0623\n",
            "Epoch [24/30], Step [58/139], Loss: 0.0513\n",
            "Epoch [24/30], Step [59/139], Loss: 0.0840\n",
            "Epoch [24/30], Step [60/139], Loss: 0.0751\n",
            "Epoch [24/30], Step [61/139], Loss: 0.0720\n",
            "Epoch [24/30], Step [62/139], Loss: 0.0949\n",
            "Epoch [24/30], Step [63/139], Loss: 0.0497\n",
            "Epoch [24/30], Step [64/139], Loss: 0.0767\n",
            "Epoch [24/30], Step [65/139], Loss: 0.0806\n",
            "Epoch [24/30], Step [66/139], Loss: 0.0772\n",
            "Epoch [24/30], Step [67/139], Loss: 0.0678\n",
            "Epoch [24/30], Step [68/139], Loss: 0.0776\n",
            "Epoch [24/30], Step [69/139], Loss: 0.0705\n",
            "Epoch [24/30], Step [70/139], Loss: 0.0816\n",
            "Epoch [24/30], Step [71/139], Loss: 0.0588\n",
            "Epoch [24/30], Step [72/139], Loss: 0.0742\n",
            "Epoch [24/30], Step [73/139], Loss: 0.0898\n",
            "Epoch [24/30], Step [74/139], Loss: 0.0826\n",
            "Epoch [24/30], Step [75/139], Loss: 0.0767\n",
            "Epoch [24/30], Step [76/139], Loss: 0.0793\n",
            "Epoch [24/30], Step [77/139], Loss: 0.0917\n",
            "Epoch [24/30], Step [78/139], Loss: 0.0627\n",
            "Epoch [24/30], Step [79/139], Loss: 0.0765\n",
            "Epoch [24/30], Step [80/139], Loss: 0.0800\n",
            "Epoch [24/30], Step [81/139], Loss: 0.0829\n",
            "Epoch [24/30], Step [82/139], Loss: 0.0895\n",
            "Epoch [24/30], Step [83/139], Loss: 0.1102\n",
            "Epoch [24/30], Step [84/139], Loss: 0.0932\n",
            "Epoch [24/30], Step [85/139], Loss: 0.0946\n",
            "Epoch [24/30], Step [86/139], Loss: 0.0995\n",
            "Epoch [24/30], Step [87/139], Loss: 0.0789\n",
            "Epoch [24/30], Step [88/139], Loss: 0.0649\n",
            "Epoch [24/30], Step [89/139], Loss: 0.1002\n",
            "Epoch [24/30], Step [90/139], Loss: 0.0664\n",
            "Epoch [24/30], Step [91/139], Loss: 0.0578\n",
            "Epoch [24/30], Step [92/139], Loss: 0.0745\n",
            "Epoch [24/30], Step [93/139], Loss: 0.0606\n",
            "Epoch [24/30], Step [94/139], Loss: 0.0767\n",
            "Epoch [24/30], Step [95/139], Loss: 0.0679\n",
            "Epoch [24/30], Step [96/139], Loss: 0.0565\n",
            "Epoch [24/30], Step [97/139], Loss: 0.0908\n",
            "Epoch [24/30], Step [98/139], Loss: 0.0739\n",
            "Epoch [24/30], Step [99/139], Loss: 0.0591\n",
            "Epoch [24/30], Step [100/139], Loss: 0.0517\n",
            "Epoch [24/30], Step [101/139], Loss: 0.0747\n",
            "Epoch [24/30], Step [102/139], Loss: 0.0949\n",
            "Epoch [24/30], Step [103/139], Loss: 0.0758\n",
            "Epoch [24/30], Step [104/139], Loss: 0.0699\n",
            "Epoch [24/30], Step [105/139], Loss: 0.0649\n",
            "Epoch [24/30], Step [106/139], Loss: 0.0640\n",
            "Epoch [24/30], Step [107/139], Loss: 0.0946\n",
            "Epoch [24/30], Step [108/139], Loss: 0.0872\n",
            "Epoch [24/30], Step [109/139], Loss: 0.0630\n",
            "Epoch [24/30], Step [110/139], Loss: 0.0733\n",
            "Epoch [24/30], Step [111/139], Loss: 0.1003\n",
            "Epoch [24/30], Step [112/139], Loss: 0.0825\n",
            "Epoch [24/30], Step [113/139], Loss: 0.0844\n",
            "Epoch [24/30], Step [114/139], Loss: 0.0808\n",
            "Epoch [24/30], Step [115/139], Loss: 0.0721\n",
            "Epoch [24/30], Step [116/139], Loss: 0.0723\n",
            "Epoch [24/30], Step [117/139], Loss: 0.0786\n",
            "Epoch [24/30], Step [118/139], Loss: 0.0802\n",
            "Epoch [24/30], Step [119/139], Loss: 0.0627\n",
            "Epoch [24/30], Step [120/139], Loss: 0.0658\n",
            "Epoch [24/30], Step [121/139], Loss: 0.1066\n",
            "Epoch [24/30], Step [122/139], Loss: 0.0604\n",
            "Epoch [24/30], Step [123/139], Loss: 0.0723\n",
            "Epoch [24/30], Step [124/139], Loss: 0.0663\n",
            "Epoch [24/30], Step [125/139], Loss: 0.0828\n",
            "Epoch [24/30], Step [126/139], Loss: 0.0806\n",
            "Epoch [24/30], Step [127/139], Loss: 0.0610\n",
            "Epoch [24/30], Step [128/139], Loss: 0.0740\n",
            "Epoch [24/30], Step [129/139], Loss: 0.0680\n",
            "Epoch [24/30], Step [130/139], Loss: 0.0973\n",
            "Epoch [24/30], Step [131/139], Loss: 0.0754\n",
            "Epoch [24/30], Step [132/139], Loss: 0.0512\n",
            "Epoch [24/30], Step [133/139], Loss: 0.1003\n",
            "Epoch [24/30], Step [134/139], Loss: 0.0633\n",
            "Epoch [24/30], Step [135/139], Loss: 0.0798\n",
            "Epoch [24/30], Step [136/139], Loss: 0.0660\n",
            "Epoch [24/30], Step [137/139], Loss: 0.0836\n",
            "Epoch [24/30], Step [138/139], Loss: 0.0682\n",
            "Epoch [24/30], Step [139/139], Loss: 0.0723\n",
            "Start validation #24\n",
            "Validation #24  Average Loss: 0.0833\n",
            "Best performance at epoch: 24\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [25/30], Step [1/139], Loss: 0.0806\n",
            "Epoch [25/30], Step [2/139], Loss: 0.0832\n",
            "Epoch [25/30], Step [3/139], Loss: 0.0811\n",
            "Epoch [25/30], Step [4/139], Loss: 0.0960\n",
            "Epoch [25/30], Step [5/139], Loss: 0.0806\n",
            "Epoch [25/30], Step [6/139], Loss: 0.0937\n",
            "Epoch [25/30], Step [7/139], Loss: 0.0760\n",
            "Epoch [25/30], Step [8/139], Loss: 0.0734\n",
            "Epoch [25/30], Step [9/139], Loss: 0.0898\n",
            "Epoch [25/30], Step [10/139], Loss: 0.0976\n",
            "Epoch [25/30], Step [11/139], Loss: 0.0548\n",
            "Epoch [25/30], Step [12/139], Loss: 0.0521\n",
            "Epoch [25/30], Step [13/139], Loss: 0.0624\n",
            "Epoch [25/30], Step [14/139], Loss: 0.0594\n",
            "Epoch [25/30], Step [15/139], Loss: 0.0518\n",
            "Epoch [25/30], Step [16/139], Loss: 0.0674\n",
            "Epoch [25/30], Step [17/139], Loss: 0.0918\n",
            "Epoch [25/30], Step [18/139], Loss: 0.0763\n",
            "Epoch [25/30], Step [19/139], Loss: 0.0770\n",
            "Epoch [25/30], Step [20/139], Loss: 0.0727\n",
            "Epoch [25/30], Step [21/139], Loss: 0.0541\n",
            "Epoch [25/30], Step [22/139], Loss: 0.0685\n",
            "Epoch [25/30], Step [23/139], Loss: 0.0851\n",
            "Epoch [25/30], Step [24/139], Loss: 0.0637\n",
            "Epoch [25/30], Step [25/139], Loss: 0.0676\n",
            "Epoch [25/30], Step [26/139], Loss: 0.0648\n",
            "Epoch [25/30], Step [27/139], Loss: 0.0827\n",
            "Epoch [25/30], Step [28/139], Loss: 0.0797\n",
            "Epoch [25/30], Step [29/139], Loss: 0.0917\n",
            "Epoch [25/30], Step [30/139], Loss: 0.1032\n",
            "Epoch [25/30], Step [31/139], Loss: 0.0709\n",
            "Epoch [25/30], Step [32/139], Loss: 0.0895\n",
            "Epoch [25/30], Step [33/139], Loss: 0.0950\n",
            "Epoch [25/30], Step [34/139], Loss: 0.0863\n",
            "Epoch [25/30], Step [35/139], Loss: 0.0911\n",
            "Epoch [25/30], Step [36/139], Loss: 0.0821\n",
            "Epoch [25/30], Step [37/139], Loss: 0.0944\n",
            "Epoch [25/30], Step [38/139], Loss: 0.0569\n",
            "Epoch [25/30], Step [39/139], Loss: 0.0625\n",
            "Epoch [25/30], Step [40/139], Loss: 0.0816\n",
            "Epoch [25/30], Step [41/139], Loss: 0.0707\n",
            "Epoch [25/30], Step [42/139], Loss: 0.1014\n",
            "Epoch [25/30], Step [43/139], Loss: 0.0724\n",
            "Epoch [25/30], Step [44/139], Loss: 0.0720\n",
            "Epoch [25/30], Step [45/139], Loss: 0.0757\n",
            "Epoch [25/30], Step [46/139], Loss: 0.0732\n",
            "Epoch [25/30], Step [47/139], Loss: 0.0443\n",
            "Epoch [25/30], Step [48/139], Loss: 0.0900\n",
            "Epoch [25/30], Step [49/139], Loss: 0.0623\n",
            "Epoch [25/30], Step [50/139], Loss: 0.0710\n",
            "Epoch [25/30], Step [51/139], Loss: 0.0779\n",
            "Epoch [25/30], Step [52/139], Loss: 0.0929\n",
            "Epoch [25/30], Step [53/139], Loss: 0.0716\n",
            "Epoch [25/30], Step [54/139], Loss: 0.0499\n",
            "Epoch [25/30], Step [55/139], Loss: 0.0758\n",
            "Epoch [25/30], Step [56/139], Loss: 0.0891\n",
            "Epoch [25/30], Step [57/139], Loss: 0.0935\n",
            "Epoch [25/30], Step [58/139], Loss: 0.0579\n",
            "Epoch [25/30], Step [59/139], Loss: 0.0509\n",
            "Epoch [25/30], Step [60/139], Loss: 0.0875\n",
            "Epoch [25/30], Step [61/139], Loss: 0.0747\n",
            "Epoch [25/30], Step [62/139], Loss: 0.0752\n",
            "Epoch [25/30], Step [63/139], Loss: 0.0989\n",
            "Epoch [25/30], Step [64/139], Loss: 0.0710\n",
            "Epoch [25/30], Step [65/139], Loss: 0.0745\n",
            "Epoch [25/30], Step [66/139], Loss: 0.0696\n",
            "Epoch [25/30], Step [67/139], Loss: 0.0685\n",
            "Epoch [25/30], Step [68/139], Loss: 0.0859\n",
            "Epoch [25/30], Step [69/139], Loss: 0.0865\n",
            "Epoch [25/30], Step [70/139], Loss: 0.0617\n",
            "Epoch [25/30], Step [71/139], Loss: 0.0715\n",
            "Epoch [25/30], Step [72/139], Loss: 0.0572\n",
            "Epoch [25/30], Step [73/139], Loss: 0.0815\n",
            "Epoch [25/30], Step [74/139], Loss: 0.0744\n",
            "Epoch [25/30], Step [75/139], Loss: 0.0649\n",
            "Epoch [25/30], Step [76/139], Loss: 0.0782\n",
            "Epoch [25/30], Step [77/139], Loss: 0.0759\n",
            "Epoch [25/30], Step [78/139], Loss: 0.0823\n",
            "Epoch [25/30], Step [79/139], Loss: 0.0544\n",
            "Epoch [25/30], Step [80/139], Loss: 0.0656\n",
            "Epoch [25/30], Step [81/139], Loss: 0.0782\n",
            "Epoch [25/30], Step [82/139], Loss: 0.0546\n",
            "Epoch [25/30], Step [83/139], Loss: 0.0877\n",
            "Epoch [25/30], Step [84/139], Loss: 0.0627\n",
            "Epoch [25/30], Step [85/139], Loss: 0.0843\n",
            "Epoch [25/30], Step [86/139], Loss: 0.0783\n",
            "Epoch [25/30], Step [87/139], Loss: 0.0655\n",
            "Epoch [25/30], Step [88/139], Loss: 0.0692\n",
            "Epoch [25/30], Step [89/139], Loss: 0.0749\n",
            "Epoch [25/30], Step [90/139], Loss: 0.0936\n",
            "Epoch [25/30], Step [91/139], Loss: 0.0744\n",
            "Epoch [25/30], Step [92/139], Loss: 0.0811\n",
            "Epoch [25/30], Step [93/139], Loss: 0.0667\n",
            "Epoch [25/30], Step [94/139], Loss: 0.0790\n",
            "Epoch [25/30], Step [95/139], Loss: 0.0880\n",
            "Epoch [25/30], Step [96/139], Loss: 0.0541\n",
            "Epoch [25/30], Step [97/139], Loss: 0.0673\n",
            "Epoch [25/30], Step [98/139], Loss: 0.0937\n",
            "Epoch [25/30], Step [99/139], Loss: 0.0582\n",
            "Epoch [25/30], Step [100/139], Loss: 0.0762\n",
            "Epoch [25/30], Step [101/139], Loss: 0.0705\n",
            "Epoch [25/30], Step [102/139], Loss: 0.0724\n",
            "Epoch [25/30], Step [103/139], Loss: 0.0832\n",
            "Epoch [25/30], Step [104/139], Loss: 0.0700\n",
            "Epoch [25/30], Step [105/139], Loss: 0.0570\n",
            "Epoch [25/30], Step [106/139], Loss: 0.0590\n",
            "Epoch [25/30], Step [107/139], Loss: 0.0783\n",
            "Epoch [25/30], Step [108/139], Loss: 0.0486\n",
            "Epoch [25/30], Step [109/139], Loss: 0.0779\n",
            "Epoch [25/30], Step [110/139], Loss: 0.0744\n",
            "Epoch [25/30], Step [111/139], Loss: 0.0764\n",
            "Epoch [25/30], Step [112/139], Loss: 0.0727\n",
            "Epoch [25/30], Step [113/139], Loss: 0.0759\n",
            "Epoch [25/30], Step [114/139], Loss: 0.0799\n",
            "Epoch [25/30], Step [115/139], Loss: 0.1007\n",
            "Epoch [25/30], Step [116/139], Loss: 0.0529\n",
            "Epoch [25/30], Step [117/139], Loss: 0.0741\n",
            "Epoch [25/30], Step [118/139], Loss: 0.0749\n",
            "Epoch [25/30], Step [119/139], Loss: 0.0892\n",
            "Epoch [25/30], Step [120/139], Loss: 0.0584\n",
            "Epoch [25/30], Step [121/139], Loss: 0.0646\n",
            "Epoch [25/30], Step [122/139], Loss: 0.0870\n",
            "Epoch [25/30], Step [123/139], Loss: 0.0662\n",
            "Epoch [25/30], Step [124/139], Loss: 0.0956\n",
            "Epoch [25/30], Step [125/139], Loss: 0.0812\n",
            "Epoch [25/30], Step [126/139], Loss: 0.0636\n",
            "Epoch [25/30], Step [127/139], Loss: 0.0774\n",
            "Epoch [25/30], Step [128/139], Loss: 0.1001\n",
            "Epoch [25/30], Step [129/139], Loss: 0.0708\n",
            "Epoch [25/30], Step [130/139], Loss: 0.0809\n",
            "Epoch [25/30], Step [131/139], Loss: 0.0601\n",
            "Epoch [25/30], Step [132/139], Loss: 0.0797\n",
            "Epoch [25/30], Step [133/139], Loss: 0.0864\n",
            "Epoch [25/30], Step [134/139], Loss: 0.0696\n",
            "Epoch [25/30], Step [135/139], Loss: 0.0826\n",
            "Epoch [25/30], Step [136/139], Loss: 0.0812\n",
            "Epoch [25/30], Step [137/139], Loss: 0.0770\n",
            "Epoch [25/30], Step [138/139], Loss: 0.0453\n",
            "Epoch [25/30], Step [139/139], Loss: 0.0573\n",
            "Start validation #25\n",
            "Validation #25  Average Loss: 0.0831\n",
            "Best performance at epoch: 25\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [26/30], Step [1/139], Loss: 0.0730\n",
            "Epoch [26/30], Step [2/139], Loss: 0.0818\n",
            "Epoch [26/30], Step [3/139], Loss: 0.0639\n",
            "Epoch [26/30], Step [4/139], Loss: 0.0841\n",
            "Epoch [26/30], Step [5/139], Loss: 0.0710\n",
            "Epoch [26/30], Step [6/139], Loss: 0.0693\n",
            "Epoch [26/30], Step [7/139], Loss: 0.0750\n",
            "Epoch [26/30], Step [8/139], Loss: 0.0834\n",
            "Epoch [26/30], Step [9/139], Loss: 0.0771\n",
            "Epoch [26/30], Step [10/139], Loss: 0.0820\n",
            "Epoch [26/30], Step [11/139], Loss: 0.0749\n",
            "Epoch [26/30], Step [12/139], Loss: 0.0509\n",
            "Epoch [26/30], Step [13/139], Loss: 0.0601\n",
            "Epoch [26/30], Step [14/139], Loss: 0.0896\n",
            "Epoch [26/30], Step [15/139], Loss: 0.0792\n",
            "Epoch [26/30], Step [16/139], Loss: 0.0797\n",
            "Epoch [26/30], Step [17/139], Loss: 0.0674\n",
            "Epoch [26/30], Step [18/139], Loss: 0.0700\n",
            "Epoch [26/30], Step [19/139], Loss: 0.0746\n",
            "Epoch [26/30], Step [20/139], Loss: 0.0780\n",
            "Epoch [26/30], Step [21/139], Loss: 0.0697\n",
            "Epoch [26/30], Step [22/139], Loss: 0.0686\n",
            "Epoch [26/30], Step [23/139], Loss: 0.0707\n",
            "Epoch [26/30], Step [24/139], Loss: 0.1038\n",
            "Epoch [26/30], Step [25/139], Loss: 0.0719\n",
            "Epoch [26/30], Step [26/139], Loss: 0.0598\n",
            "Epoch [26/30], Step [27/139], Loss: 0.0934\n",
            "Epoch [26/30], Step [28/139], Loss: 0.0828\n",
            "Epoch [26/30], Step [29/139], Loss: 0.0778\n",
            "Epoch [26/30], Step [30/139], Loss: 0.0697\n",
            "Epoch [26/30], Step [31/139], Loss: 0.1084\n",
            "Epoch [26/30], Step [32/139], Loss: 0.0820\n",
            "Epoch [26/30], Step [33/139], Loss: 0.0687\n",
            "Epoch [26/30], Step [34/139], Loss: 0.0676\n",
            "Epoch [26/30], Step [35/139], Loss: 0.0731\n",
            "Epoch [26/30], Step [36/139], Loss: 0.0794\n",
            "Epoch [26/30], Step [37/139], Loss: 0.0681\n",
            "Epoch [26/30], Step [38/139], Loss: 0.0514\n",
            "Epoch [26/30], Step [39/139], Loss: 0.0577\n",
            "Epoch [26/30], Step [40/139], Loss: 0.0587\n",
            "Epoch [26/30], Step [41/139], Loss: 0.0658\n",
            "Epoch [26/30], Step [42/139], Loss: 0.0606\n",
            "Epoch [26/30], Step [43/139], Loss: 0.0964\n",
            "Epoch [26/30], Step [44/139], Loss: 0.0600\n",
            "Epoch [26/30], Step [45/139], Loss: 0.0652\n",
            "Epoch [26/30], Step [46/139], Loss: 0.0732\n",
            "Epoch [26/30], Step [47/139], Loss: 0.1019\n",
            "Epoch [26/30], Step [48/139], Loss: 0.0676\n",
            "Epoch [26/30], Step [49/139], Loss: 0.0944\n",
            "Epoch [26/30], Step [50/139], Loss: 0.0690\n",
            "Epoch [26/30], Step [51/139], Loss: 0.0886\n",
            "Epoch [26/30], Step [52/139], Loss: 0.0698\n",
            "Epoch [26/30], Step [53/139], Loss: 0.0597\n",
            "Epoch [26/30], Step [54/139], Loss: 0.0630\n",
            "Epoch [26/30], Step [55/139], Loss: 0.0789\n",
            "Epoch [26/30], Step [56/139], Loss: 0.0511\n",
            "Epoch [26/30], Step [57/139], Loss: 0.0759\n",
            "Epoch [26/30], Step [58/139], Loss: 0.1125\n",
            "Epoch [26/30], Step [59/139], Loss: 0.0682\n",
            "Epoch [26/30], Step [60/139], Loss: 0.0664\n",
            "Epoch [26/30], Step [61/139], Loss: 0.0751\n",
            "Epoch [26/30], Step [62/139], Loss: 0.0851\n",
            "Epoch [26/30], Step [63/139], Loss: 0.0839\n",
            "Epoch [26/30], Step [64/139], Loss: 0.0747\n",
            "Epoch [26/30], Step [65/139], Loss: 0.1007\n",
            "Epoch [26/30], Step [66/139], Loss: 0.0828\n",
            "Epoch [26/30], Step [67/139], Loss: 0.0722\n",
            "Epoch [26/30], Step [68/139], Loss: 0.0738\n",
            "Epoch [26/30], Step [69/139], Loss: 0.0731\n",
            "Epoch [26/30], Step [70/139], Loss: 0.0652\n",
            "Epoch [26/30], Step [71/139], Loss: 0.0837\n",
            "Epoch [26/30], Step [72/139], Loss: 0.0723\n",
            "Epoch [26/30], Step [73/139], Loss: 0.0795\n",
            "Epoch [26/30], Step [74/139], Loss: 0.0964\n",
            "Epoch [26/30], Step [75/139], Loss: 0.0632\n",
            "Epoch [26/30], Step [76/139], Loss: 0.0571\n",
            "Epoch [26/30], Step [77/139], Loss: 0.0667\n",
            "Epoch [26/30], Step [78/139], Loss: 0.0848\n",
            "Epoch [26/30], Step [79/139], Loss: 0.0909\n",
            "Epoch [26/30], Step [80/139], Loss: 0.0675\n",
            "Epoch [26/30], Step [81/139], Loss: 0.0743\n",
            "Epoch [26/30], Step [82/139], Loss: 0.0650\n",
            "Epoch [26/30], Step [83/139], Loss: 0.0750\n",
            "Epoch [26/30], Step [84/139], Loss: 0.0580\n",
            "Epoch [26/30], Step [85/139], Loss: 0.0587\n",
            "Epoch [26/30], Step [86/139], Loss: 0.0996\n",
            "Epoch [26/30], Step [87/139], Loss: 0.0971\n",
            "Epoch [26/30], Step [88/139], Loss: 0.0854\n",
            "Epoch [26/30], Step [89/139], Loss: 0.0734\n",
            "Epoch [26/30], Step [90/139], Loss: 0.0556\n",
            "Epoch [26/30], Step [91/139], Loss: 0.0683\n",
            "Epoch [26/30], Step [92/139], Loss: 0.0685\n",
            "Epoch [26/30], Step [93/139], Loss: 0.0682\n",
            "Epoch [26/30], Step [94/139], Loss: 0.0940\n",
            "Epoch [26/30], Step [95/139], Loss: 0.0664\n",
            "Epoch [26/30], Step [96/139], Loss: 0.0809\n",
            "Epoch [26/30], Step [97/139], Loss: 0.1027\n",
            "Epoch [26/30], Step [98/139], Loss: 0.0894\n",
            "Epoch [26/30], Step [99/139], Loss: 0.0620\n",
            "Epoch [26/30], Step [100/139], Loss: 0.0689\n",
            "Epoch [26/30], Step [101/139], Loss: 0.0665\n",
            "Epoch [26/30], Step [102/139], Loss: 0.0798\n",
            "Epoch [26/30], Step [103/139], Loss: 0.0602\n",
            "Epoch [26/30], Step [104/139], Loss: 0.0941\n",
            "Epoch [26/30], Step [105/139], Loss: 0.0930\n",
            "Epoch [26/30], Step [106/139], Loss: 0.0570\n",
            "Epoch [26/30], Step [107/139], Loss: 0.0776\n",
            "Epoch [26/30], Step [108/139], Loss: 0.0916\n",
            "Epoch [26/30], Step [109/139], Loss: 0.0498\n",
            "Epoch [26/30], Step [110/139], Loss: 0.0732\n",
            "Epoch [26/30], Step [111/139], Loss: 0.0901\n",
            "Epoch [26/30], Step [112/139], Loss: 0.0632\n",
            "Epoch [26/30], Step [113/139], Loss: 0.0782\n",
            "Epoch [26/30], Step [114/139], Loss: 0.0592\n",
            "Epoch [26/30], Step [115/139], Loss: 0.0937\n",
            "Epoch [26/30], Step [116/139], Loss: 0.0716\n",
            "Epoch [26/30], Step [117/139], Loss: 0.0617\n",
            "Epoch [26/30], Step [118/139], Loss: 0.0620\n",
            "Epoch [26/30], Step [119/139], Loss: 0.0516\n",
            "Epoch [26/30], Step [120/139], Loss: 0.0625\n",
            "Epoch [26/30], Step [121/139], Loss: 0.0616\n",
            "Epoch [26/30], Step [122/139], Loss: 0.0597\n",
            "Epoch [26/30], Step [123/139], Loss: 0.0621\n",
            "Epoch [26/30], Step [124/139], Loss: 0.0583\n",
            "Epoch [26/30], Step [125/139], Loss: 0.0917\n",
            "Epoch [26/30], Step [126/139], Loss: 0.0857\n",
            "Epoch [26/30], Step [127/139], Loss: 0.0755\n",
            "Epoch [26/30], Step [128/139], Loss: 0.0705\n",
            "Epoch [26/30], Step [129/139], Loss: 0.0965\n",
            "Epoch [26/30], Step [130/139], Loss: 0.0734\n",
            "Epoch [26/30], Step [131/139], Loss: 0.0757\n",
            "Epoch [26/30], Step [132/139], Loss: 0.0591\n",
            "Epoch [26/30], Step [133/139], Loss: 0.0609\n",
            "Epoch [26/30], Step [134/139], Loss: 0.0801\n",
            "Epoch [26/30], Step [135/139], Loss: 0.0661\n",
            "Epoch [26/30], Step [136/139], Loss: 0.0702\n",
            "Epoch [26/30], Step [137/139], Loss: 0.0634\n",
            "Epoch [26/30], Step [138/139], Loss: 0.0850\n",
            "Epoch [26/30], Step [139/139], Loss: 0.0745\n",
            "Start validation #26\n",
            "Validation #26  Average Loss: 0.0827\n",
            "Best performance at epoch: 26\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [27/30], Step [1/139], Loss: 0.0840\n",
            "Epoch [27/30], Step [2/139], Loss: 0.0772\n",
            "Epoch [27/30], Step [3/139], Loss: 0.0688\n",
            "Epoch [27/30], Step [4/139], Loss: 0.0922\n",
            "Epoch [27/30], Step [5/139], Loss: 0.0942\n",
            "Epoch [27/30], Step [6/139], Loss: 0.0902\n",
            "Epoch [27/30], Step [7/139], Loss: 0.0733\n",
            "Epoch [27/30], Step [8/139], Loss: 0.0752\n",
            "Epoch [27/30], Step [9/139], Loss: 0.0769\n",
            "Epoch [27/30], Step [10/139], Loss: 0.1066\n",
            "Epoch [27/30], Step [11/139], Loss: 0.0585\n",
            "Epoch [27/30], Step [12/139], Loss: 0.0760\n",
            "Epoch [27/30], Step [13/139], Loss: 0.0755\n",
            "Epoch [27/30], Step [14/139], Loss: 0.0779\n",
            "Epoch [27/30], Step [15/139], Loss: 0.0687\n",
            "Epoch [27/30], Step [16/139], Loss: 0.0931\n",
            "Epoch [27/30], Step [17/139], Loss: 0.0780\n",
            "Epoch [27/30], Step [18/139], Loss: 0.1189\n",
            "Epoch [27/30], Step [19/139], Loss: 0.0634\n",
            "Epoch [27/30], Step [20/139], Loss: 0.0757\n",
            "Epoch [27/30], Step [21/139], Loss: 0.0976\n",
            "Epoch [27/30], Step [22/139], Loss: 0.0555\n",
            "Epoch [27/30], Step [23/139], Loss: 0.0520\n",
            "Epoch [27/30], Step [24/139], Loss: 0.0556\n",
            "Epoch [27/30], Step [25/139], Loss: 0.0647\n",
            "Epoch [27/30], Step [26/139], Loss: 0.0552\n",
            "Epoch [27/30], Step [27/139], Loss: 0.0692\n",
            "Epoch [27/30], Step [28/139], Loss: 0.0904\n",
            "Epoch [27/30], Step [29/139], Loss: 0.0717\n",
            "Epoch [27/30], Step [30/139], Loss: 0.0810\n",
            "Epoch [27/30], Step [31/139], Loss: 0.0537\n",
            "Epoch [27/30], Step [32/139], Loss: 0.0719\n",
            "Epoch [27/30], Step [33/139], Loss: 0.0662\n",
            "Epoch [27/30], Step [34/139], Loss: 0.0692\n",
            "Epoch [27/30], Step [35/139], Loss: 0.0769\n",
            "Epoch [27/30], Step [36/139], Loss: 0.0656\n",
            "Epoch [27/30], Step [37/139], Loss: 0.1123\n",
            "Epoch [27/30], Step [38/139], Loss: 0.0779\n",
            "Epoch [27/30], Step [39/139], Loss: 0.0609\n",
            "Epoch [27/30], Step [40/139], Loss: 0.0789\n",
            "Epoch [27/30], Step [41/139], Loss: 0.0753\n",
            "Epoch [27/30], Step [42/139], Loss: 0.0610\n",
            "Epoch [27/30], Step [43/139], Loss: 0.0546\n",
            "Epoch [27/30], Step [44/139], Loss: 0.0770\n",
            "Epoch [27/30], Step [45/139], Loss: 0.0685\n",
            "Epoch [27/30], Step [46/139], Loss: 0.0712\n",
            "Epoch [27/30], Step [47/139], Loss: 0.0900\n",
            "Epoch [27/30], Step [48/139], Loss: 0.0633\n",
            "Epoch [27/30], Step [49/139], Loss: 0.0585\n",
            "Epoch [27/30], Step [50/139], Loss: 0.0811\n",
            "Epoch [27/30], Step [51/139], Loss: 0.0677\n",
            "Epoch [27/30], Step [52/139], Loss: 0.0686\n",
            "Epoch [27/30], Step [53/139], Loss: 0.0692\n",
            "Epoch [27/30], Step [54/139], Loss: 0.0523\n",
            "Epoch [27/30], Step [55/139], Loss: 0.0794\n",
            "Epoch [27/30], Step [56/139], Loss: 0.0488\n",
            "Epoch [27/30], Step [57/139], Loss: 0.1114\n",
            "Epoch [27/30], Step [58/139], Loss: 0.0629\n",
            "Epoch [27/30], Step [59/139], Loss: 0.0754\n",
            "Epoch [27/30], Step [60/139], Loss: 0.0857\n",
            "Epoch [27/30], Step [61/139], Loss: 0.0619\n",
            "Epoch [27/30], Step [62/139], Loss: 0.0792\n",
            "Epoch [27/30], Step [63/139], Loss: 0.0765\n",
            "Epoch [27/30], Step [64/139], Loss: 0.0808\n",
            "Epoch [27/30], Step [65/139], Loss: 0.0684\n",
            "Epoch [27/30], Step [66/139], Loss: 0.0765\n",
            "Epoch [27/30], Step [67/139], Loss: 0.0604\n",
            "Epoch [27/30], Step [68/139], Loss: 0.0846\n",
            "Epoch [27/30], Step [69/139], Loss: 0.0644\n",
            "Epoch [27/30], Step [70/139], Loss: 0.0909\n",
            "Epoch [27/30], Step [71/139], Loss: 0.0603\n",
            "Epoch [27/30], Step [72/139], Loss: 0.0865\n",
            "Epoch [27/30], Step [73/139], Loss: 0.0789\n",
            "Epoch [27/30], Step [74/139], Loss: 0.0758\n",
            "Epoch [27/30], Step [75/139], Loss: 0.0782\n",
            "Epoch [27/30], Step [76/139], Loss: 0.0660\n",
            "Epoch [27/30], Step [77/139], Loss: 0.0819\n",
            "Epoch [27/30], Step [78/139], Loss: 0.0598\n",
            "Epoch [27/30], Step [79/139], Loss: 0.0477\n",
            "Epoch [27/30], Step [80/139], Loss: 0.0511\n",
            "Epoch [27/30], Step [81/139], Loss: 0.0669\n",
            "Epoch [27/30], Step [82/139], Loss: 0.0681\n",
            "Epoch [27/30], Step [83/139], Loss: 0.0710\n",
            "Epoch [27/30], Step [84/139], Loss: 0.0705\n",
            "Epoch [27/30], Step [85/139], Loss: 0.0857\n",
            "Epoch [27/30], Step [86/139], Loss: 0.0682\n",
            "Epoch [27/30], Step [87/139], Loss: 0.0714\n",
            "Epoch [27/30], Step [88/139], Loss: 0.0658\n",
            "Epoch [27/30], Step [89/139], Loss: 0.0616\n",
            "Epoch [27/30], Step [90/139], Loss: 0.0571\n",
            "Epoch [27/30], Step [91/139], Loss: 0.0756\n",
            "Epoch [27/30], Step [92/139], Loss: 0.0634\n",
            "Epoch [27/30], Step [93/139], Loss: 0.0777\n",
            "Epoch [27/30], Step [94/139], Loss: 0.1061\n",
            "Epoch [27/30], Step [95/139], Loss: 0.0617\n",
            "Epoch [27/30], Step [96/139], Loss: 0.0736\n",
            "Epoch [27/30], Step [97/139], Loss: 0.0979\n",
            "Epoch [27/30], Step [98/139], Loss: 0.0714\n",
            "Epoch [27/30], Step [99/139], Loss: 0.0602\n",
            "Epoch [27/30], Step [100/139], Loss: 0.0773\n",
            "Epoch [27/30], Step [101/139], Loss: 0.0863\n",
            "Epoch [27/30], Step [102/139], Loss: 0.0628\n",
            "Epoch [27/30], Step [103/139], Loss: 0.0697\n",
            "Epoch [27/30], Step [104/139], Loss: 0.0761\n",
            "Epoch [27/30], Step [105/139], Loss: 0.0777\n",
            "Epoch [27/30], Step [106/139], Loss: 0.0987\n",
            "Epoch [27/30], Step [107/139], Loss: 0.0766\n",
            "Epoch [27/30], Step [108/139], Loss: 0.0742\n",
            "Epoch [27/30], Step [109/139], Loss: 0.0918\n",
            "Epoch [27/30], Step [110/139], Loss: 0.0704\n",
            "Epoch [27/30], Step [111/139], Loss: 0.0910\n",
            "Epoch [27/30], Step [112/139], Loss: 0.0566\n",
            "Epoch [27/30], Step [113/139], Loss: 0.0890\n",
            "Epoch [27/30], Step [114/139], Loss: 0.0596\n",
            "Epoch [27/30], Step [115/139], Loss: 0.0682\n",
            "Epoch [27/30], Step [116/139], Loss: 0.0825\n",
            "Epoch [27/30], Step [117/139], Loss: 0.0559\n",
            "Epoch [27/30], Step [118/139], Loss: 0.0894\n",
            "Epoch [27/30], Step [119/139], Loss: 0.0973\n",
            "Epoch [27/30], Step [120/139], Loss: 0.0627\n",
            "Epoch [27/30], Step [121/139], Loss: 0.0540\n",
            "Epoch [27/30], Step [122/139], Loss: 0.1174\n",
            "Epoch [27/30], Step [123/139], Loss: 0.0761\n",
            "Epoch [27/30], Step [124/139], Loss: 0.0553\n",
            "Epoch [27/30], Step [125/139], Loss: 0.0609\n",
            "Epoch [27/30], Step [126/139], Loss: 0.0596\n",
            "Epoch [27/30], Step [127/139], Loss: 0.0662\n",
            "Epoch [27/30], Step [128/139], Loss: 0.0679\n",
            "Epoch [27/30], Step [129/139], Loss: 0.0639\n",
            "Epoch [27/30], Step [130/139], Loss: 0.0642\n",
            "Epoch [27/30], Step [131/139], Loss: 0.0876\n",
            "Epoch [27/30], Step [132/139], Loss: 0.0734\n",
            "Epoch [27/30], Step [133/139], Loss: 0.0814\n",
            "Epoch [27/30], Step [134/139], Loss: 0.0645\n",
            "Epoch [27/30], Step [135/139], Loss: 0.0741\n",
            "Epoch [27/30], Step [136/139], Loss: 0.0659\n",
            "Epoch [27/30], Step [137/139], Loss: 0.0752\n",
            "Epoch [27/30], Step [138/139], Loss: 0.0733\n",
            "Epoch [27/30], Step [139/139], Loss: 0.0669\n",
            "Start validation #27\n",
            "Validation #27  Average Loss: 0.0817\n",
            "Best performance at epoch: 27\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [28/30], Step [1/139], Loss: 0.0745\n",
            "Epoch [28/30], Step [2/139], Loss: 0.0441\n",
            "Epoch [28/30], Step [3/139], Loss: 0.1031\n",
            "Epoch [28/30], Step [4/139], Loss: 0.0552\n",
            "Epoch [28/30], Step [5/139], Loss: 0.0693\n",
            "Epoch [28/30], Step [6/139], Loss: 0.0642\n",
            "Epoch [28/30], Step [7/139], Loss: 0.0767\n",
            "Epoch [28/30], Step [8/139], Loss: 0.1004\n",
            "Epoch [28/30], Step [9/139], Loss: 0.0871\n",
            "Epoch [28/30], Step [10/139], Loss: 0.0761\n",
            "Epoch [28/30], Step [11/139], Loss: 0.0644\n",
            "Epoch [28/30], Step [12/139], Loss: 0.0734\n",
            "Epoch [28/30], Step [13/139], Loss: 0.0862\n",
            "Epoch [28/30], Step [14/139], Loss: 0.0742\n",
            "Epoch [28/30], Step [15/139], Loss: 0.0637\n",
            "Epoch [28/30], Step [16/139], Loss: 0.0668\n",
            "Epoch [28/30], Step [17/139], Loss: 0.0694\n",
            "Epoch [28/30], Step [18/139], Loss: 0.0832\n",
            "Epoch [28/30], Step [19/139], Loss: 0.0956\n",
            "Epoch [28/30], Step [20/139], Loss: 0.0925\n",
            "Epoch [28/30], Step [21/139], Loss: 0.0951\n",
            "Epoch [28/30], Step [22/139], Loss: 0.1102\n",
            "Epoch [28/30], Step [23/139], Loss: 0.0813\n",
            "Epoch [28/30], Step [24/139], Loss: 0.0799\n",
            "Epoch [28/30], Step [25/139], Loss: 0.0712\n",
            "Epoch [28/30], Step [26/139], Loss: 0.0614\n",
            "Epoch [28/30], Step [27/139], Loss: 0.0798\n",
            "Epoch [28/30], Step [28/139], Loss: 0.0731\n",
            "Epoch [28/30], Step [29/139], Loss: 0.0777\n",
            "Epoch [28/30], Step [30/139], Loss: 0.0815\n",
            "Epoch [28/30], Step [31/139], Loss: 0.0901\n",
            "Epoch [28/30], Step [32/139], Loss: 0.0935\n",
            "Epoch [28/30], Step [33/139], Loss: 0.0728\n",
            "Epoch [28/30], Step [34/139], Loss: 0.0766\n",
            "Epoch [28/30], Step [35/139], Loss: 0.0800\n",
            "Epoch [28/30], Step [36/139], Loss: 0.0542\n",
            "Epoch [28/30], Step [37/139], Loss: 0.0684\n",
            "Epoch [28/30], Step [38/139], Loss: 0.0757\n",
            "Epoch [28/30], Step [39/139], Loss: 0.0697\n",
            "Epoch [28/30], Step [40/139], Loss: 0.0606\n",
            "Epoch [28/30], Step [41/139], Loss: 0.0672\n",
            "Epoch [28/30], Step [42/139], Loss: 0.0636\n",
            "Epoch [28/30], Step [43/139], Loss: 0.0578\n",
            "Epoch [28/30], Step [44/139], Loss: 0.0821\n",
            "Epoch [28/30], Step [45/139], Loss: 0.0818\n",
            "Epoch [28/30], Step [46/139], Loss: 0.0806\n",
            "Epoch [28/30], Step [47/139], Loss: 0.0778\n",
            "Epoch [28/30], Step [48/139], Loss: 0.0842\n",
            "Epoch [28/30], Step [49/139], Loss: 0.0750\n",
            "Epoch [28/30], Step [50/139], Loss: 0.0550\n",
            "Epoch [28/30], Step [51/139], Loss: 0.0716\n",
            "Epoch [28/30], Step [52/139], Loss: 0.0505\n",
            "Epoch [28/30], Step [53/139], Loss: 0.0620\n",
            "Epoch [28/30], Step [54/139], Loss: 0.0569\n",
            "Epoch [28/30], Step [55/139], Loss: 0.0698\n",
            "Epoch [28/30], Step [56/139], Loss: 0.0654\n",
            "Epoch [28/30], Step [57/139], Loss: 0.0703\n",
            "Epoch [28/30], Step [58/139], Loss: 0.0585\n",
            "Epoch [28/30], Step [59/139], Loss: 0.0582\n",
            "Epoch [28/30], Step [60/139], Loss: 0.1144\n",
            "Epoch [28/30], Step [61/139], Loss: 0.0695\n",
            "Epoch [28/30], Step [62/139], Loss: 0.0738\n",
            "Epoch [28/30], Step [63/139], Loss: 0.0685\n",
            "Epoch [28/30], Step [64/139], Loss: 0.0693\n",
            "Epoch [28/30], Step [65/139], Loss: 0.0677\n",
            "Epoch [28/30], Step [66/139], Loss: 0.0739\n",
            "Epoch [28/30], Step [67/139], Loss: 0.0574\n",
            "Epoch [28/30], Step [68/139], Loss: 0.0762\n",
            "Epoch [28/30], Step [69/139], Loss: 0.0802\n",
            "Epoch [28/30], Step [70/139], Loss: 0.1038\n",
            "Epoch [28/30], Step [71/139], Loss: 0.0608\n",
            "Epoch [28/30], Step [72/139], Loss: 0.0718\n",
            "Epoch [28/30], Step [73/139], Loss: 0.0518\n",
            "Epoch [28/30], Step [74/139], Loss: 0.0852\n",
            "Epoch [28/30], Step [75/139], Loss: 0.0575\n",
            "Epoch [28/30], Step [76/139], Loss: 0.0626\n",
            "Epoch [28/30], Step [77/139], Loss: 0.0599\n",
            "Epoch [28/30], Step [78/139], Loss: 0.0611\n",
            "Epoch [28/30], Step [79/139], Loss: 0.0797\n",
            "Epoch [28/30], Step [80/139], Loss: 0.0799\n",
            "Epoch [28/30], Step [81/139], Loss: 0.0738\n",
            "Epoch [28/30], Step [82/139], Loss: 0.0701\n",
            "Epoch [28/30], Step [83/139], Loss: 0.0548\n",
            "Epoch [28/30], Step [84/139], Loss: 0.0601\n",
            "Epoch [28/30], Step [85/139], Loss: 0.0698\n",
            "Epoch [28/30], Step [86/139], Loss: 0.0636\n",
            "Epoch [28/30], Step [87/139], Loss: 0.0472\n",
            "Epoch [28/30], Step [88/139], Loss: 0.0749\n",
            "Epoch [28/30], Step [89/139], Loss: 0.0866\n",
            "Epoch [28/30], Step [90/139], Loss: 0.0843\n",
            "Epoch [28/30], Step [91/139], Loss: 0.0729\n",
            "Epoch [28/30], Step [92/139], Loss: 0.0540\n",
            "Epoch [28/30], Step [93/139], Loss: 0.0616\n",
            "Epoch [28/30], Step [94/139], Loss: 0.0672\n",
            "Epoch [28/30], Step [95/139], Loss: 0.0651\n",
            "Epoch [28/30], Step [96/139], Loss: 0.0633\n",
            "Epoch [28/30], Step [97/139], Loss: 0.0757\n",
            "Epoch [28/30], Step [98/139], Loss: 0.0702\n",
            "Epoch [28/30], Step [99/139], Loss: 0.0880\n",
            "Epoch [28/30], Step [100/139], Loss: 0.0557\n",
            "Epoch [28/30], Step [101/139], Loss: 0.0596\n",
            "Epoch [28/30], Step [102/139], Loss: 0.0767\n",
            "Epoch [28/30], Step [103/139], Loss: 0.0753\n",
            "Epoch [28/30], Step [104/139], Loss: 0.0756\n",
            "Epoch [28/30], Step [105/139], Loss: 0.0876\n",
            "Epoch [28/30], Step [106/139], Loss: 0.0549\n",
            "Epoch [28/30], Step [107/139], Loss: 0.0922\n",
            "Epoch [28/30], Step [108/139], Loss: 0.0691\n",
            "Epoch [28/30], Step [109/139], Loss: 0.0688\n",
            "Epoch [28/30], Step [110/139], Loss: 0.1079\n",
            "Epoch [28/30], Step [111/139], Loss: 0.0934\n",
            "Epoch [28/30], Step [112/139], Loss: 0.0865\n",
            "Epoch [28/30], Step [113/139], Loss: 0.0809\n",
            "Epoch [28/30], Step [114/139], Loss: 0.0751\n",
            "Epoch [28/30], Step [115/139], Loss: 0.0541\n",
            "Epoch [28/30], Step [116/139], Loss: 0.0537\n",
            "Epoch [28/30], Step [117/139], Loss: 0.0839\n",
            "Epoch [28/30], Step [118/139], Loss: 0.0736\n",
            "Epoch [28/30], Step [119/139], Loss: 0.0768\n",
            "Epoch [28/30], Step [120/139], Loss: 0.0668\n",
            "Epoch [28/30], Step [121/139], Loss: 0.0854\n",
            "Epoch [28/30], Step [122/139], Loss: 0.0672\n",
            "Epoch [28/30], Step [123/139], Loss: 0.0660\n",
            "Epoch [28/30], Step [124/139], Loss: 0.0803\n",
            "Epoch [28/30], Step [125/139], Loss: 0.0612\n",
            "Epoch [28/30], Step [126/139], Loss: 0.0851\n",
            "Epoch [28/30], Step [127/139], Loss: 0.0771\n",
            "Epoch [28/30], Step [128/139], Loss: 0.0753\n",
            "Epoch [28/30], Step [129/139], Loss: 0.0673\n",
            "Epoch [28/30], Step [130/139], Loss: 0.0618\n",
            "Epoch [28/30], Step [131/139], Loss: 0.0648\n",
            "Epoch [28/30], Step [132/139], Loss: 0.0784\n",
            "Epoch [28/30], Step [133/139], Loss: 0.0771\n",
            "Epoch [28/30], Step [134/139], Loss: 0.0888\n",
            "Epoch [28/30], Step [135/139], Loss: 0.0770\n",
            "Epoch [28/30], Step [136/139], Loss: 0.0682\n",
            "Epoch [28/30], Step [137/139], Loss: 0.0851\n",
            "Epoch [28/30], Step [138/139], Loss: 0.0569\n",
            "Epoch [28/30], Step [139/139], Loss: 0.0843\n",
            "Start validation #28\n",
            "Validation #28  Average Loss: 0.0829\n",
            "Epoch [29/30], Step [1/139], Loss: 0.0613\n",
            "Epoch [29/30], Step [2/139], Loss: 0.0738\n",
            "Epoch [29/30], Step [3/139], Loss: 0.0753\n",
            "Epoch [29/30], Step [4/139], Loss: 0.0828\n",
            "Epoch [29/30], Step [5/139], Loss: 0.0762\n",
            "Epoch [29/30], Step [6/139], Loss: 0.0752\n",
            "Epoch [29/30], Step [7/139], Loss: 0.0838\n",
            "Epoch [29/30], Step [8/139], Loss: 0.0658\n",
            "Epoch [29/30], Step [9/139], Loss: 0.0884\n",
            "Epoch [29/30], Step [10/139], Loss: 0.0786\n",
            "Epoch [29/30], Step [11/139], Loss: 0.0672\n",
            "Epoch [29/30], Step [12/139], Loss: 0.0655\n",
            "Epoch [29/30], Step [13/139], Loss: 0.0738\n",
            "Epoch [29/30], Step [14/139], Loss: 0.0655\n",
            "Epoch [29/30], Step [15/139], Loss: 0.0793\n",
            "Epoch [29/30], Step [16/139], Loss: 0.0775\n",
            "Epoch [29/30], Step [17/139], Loss: 0.0776\n",
            "Epoch [29/30], Step [18/139], Loss: 0.0970\n",
            "Epoch [29/30], Step [19/139], Loss: 0.0578\n",
            "Epoch [29/30], Step [20/139], Loss: 0.0598\n",
            "Epoch [29/30], Step [21/139], Loss: 0.0679\n",
            "Epoch [29/30], Step [22/139], Loss: 0.0766\n",
            "Epoch [29/30], Step [23/139], Loss: 0.0556\n",
            "Epoch [29/30], Step [24/139], Loss: 0.0621\n",
            "Epoch [29/30], Step [25/139], Loss: 0.0956\n",
            "Epoch [29/30], Step [26/139], Loss: 0.0925\n",
            "Epoch [29/30], Step [27/139], Loss: 0.0691\n",
            "Epoch [29/30], Step [28/139], Loss: 0.0525\n",
            "Epoch [29/30], Step [29/139], Loss: 0.1083\n",
            "Epoch [29/30], Step [30/139], Loss: 0.0672\n",
            "Epoch [29/30], Step [31/139], Loss: 0.0741\n",
            "Epoch [29/30], Step [32/139], Loss: 0.0687\n",
            "Epoch [29/30], Step [33/139], Loss: 0.0783\n",
            "Epoch [29/30], Step [34/139], Loss: 0.0734\n",
            "Epoch [29/30], Step [35/139], Loss: 0.0727\n",
            "Epoch [29/30], Step [36/139], Loss: 0.0728\n",
            "Epoch [29/30], Step [37/139], Loss: 0.0738\n",
            "Epoch [29/30], Step [38/139], Loss: 0.0711\n",
            "Epoch [29/30], Step [39/139], Loss: 0.0662\n",
            "Epoch [29/30], Step [40/139], Loss: 0.0914\n",
            "Epoch [29/30], Step [41/139], Loss: 0.0816\n",
            "Epoch [29/30], Step [42/139], Loss: 0.0573\n",
            "Epoch [29/30], Step [43/139], Loss: 0.0880\n",
            "Epoch [29/30], Step [44/139], Loss: 0.0915\n",
            "Epoch [29/30], Step [45/139], Loss: 0.0952\n",
            "Epoch [29/30], Step [46/139], Loss: 0.0699\n",
            "Epoch [29/30], Step [47/139], Loss: 0.0682\n",
            "Epoch [29/30], Step [48/139], Loss: 0.0831\n",
            "Epoch [29/30], Step [49/139], Loss: 0.0701\n",
            "Epoch [29/30], Step [50/139], Loss: 0.0537\n",
            "Epoch [29/30], Step [51/139], Loss: 0.0763\n",
            "Epoch [29/30], Step [52/139], Loss: 0.0570\n",
            "Epoch [29/30], Step [53/139], Loss: 0.0736\n",
            "Epoch [29/30], Step [54/139], Loss: 0.0992\n",
            "Epoch [29/30], Step [55/139], Loss: 0.0537\n",
            "Epoch [29/30], Step [56/139], Loss: 0.0616\n",
            "Epoch [29/30], Step [57/139], Loss: 0.0694\n",
            "Epoch [29/30], Step [58/139], Loss: 0.0550\n",
            "Epoch [29/30], Step [59/139], Loss: 0.0731\n",
            "Epoch [29/30], Step [60/139], Loss: 0.0706\n",
            "Epoch [29/30], Step [61/139], Loss: 0.0735\n",
            "Epoch [29/30], Step [62/139], Loss: 0.0854\n",
            "Epoch [29/30], Step [63/139], Loss: 0.0958\n",
            "Epoch [29/30], Step [64/139], Loss: 0.0810\n",
            "Epoch [29/30], Step [65/139], Loss: 0.0540\n",
            "Epoch [29/30], Step [66/139], Loss: 0.0831\n",
            "Epoch [29/30], Step [67/139], Loss: 0.0495\n",
            "Epoch [29/30], Step [68/139], Loss: 0.0791\n",
            "Epoch [29/30], Step [69/139], Loss: 0.0673\n",
            "Epoch [29/30], Step [70/139], Loss: 0.0828\n",
            "Epoch [29/30], Step [71/139], Loss: 0.0744\n",
            "Epoch [29/30], Step [72/139], Loss: 0.0944\n",
            "Epoch [29/30], Step [73/139], Loss: 0.0755\n",
            "Epoch [29/30], Step [74/139], Loss: 0.0975\n",
            "Epoch [29/30], Step [75/139], Loss: 0.0810\n",
            "Epoch [29/30], Step [76/139], Loss: 0.0707\n",
            "Epoch [29/30], Step [77/139], Loss: 0.0660\n",
            "Epoch [29/30], Step [78/139], Loss: 0.0633\n",
            "Epoch [29/30], Step [79/139], Loss: 0.0650\n",
            "Epoch [29/30], Step [80/139], Loss: 0.0679\n",
            "Epoch [29/30], Step [81/139], Loss: 0.0684\n",
            "Epoch [29/30], Step [82/139], Loss: 0.0578\n",
            "Epoch [29/30], Step [83/139], Loss: 0.0732\n",
            "Epoch [29/30], Step [84/139], Loss: 0.0730\n",
            "Epoch [29/30], Step [85/139], Loss: 0.0728\n",
            "Epoch [29/30], Step [86/139], Loss: 0.0878\n",
            "Epoch [29/30], Step [87/139], Loss: 0.0840\n",
            "Epoch [29/30], Step [88/139], Loss: 0.0531\n",
            "Epoch [29/30], Step [89/139], Loss: 0.0656\n",
            "Epoch [29/30], Step [90/139], Loss: 0.0819\n",
            "Epoch [29/30], Step [91/139], Loss: 0.0405\n",
            "Epoch [29/30], Step [92/139], Loss: 0.0790\n",
            "Epoch [29/30], Step [93/139], Loss: 0.0613\n",
            "Epoch [29/30], Step [94/139], Loss: 0.0696\n",
            "Epoch [29/30], Step [95/139], Loss: 0.0821\n",
            "Epoch [29/30], Step [96/139], Loss: 0.0741\n",
            "Epoch [29/30], Step [97/139], Loss: 0.0808\n",
            "Epoch [29/30], Step [98/139], Loss: 0.0699\n",
            "Epoch [29/30], Step [99/139], Loss: 0.0650\n",
            "Epoch [29/30], Step [100/139], Loss: 0.0959\n",
            "Epoch [29/30], Step [101/139], Loss: 0.0722\n",
            "Epoch [29/30], Step [102/139], Loss: 0.0621\n",
            "Epoch [29/30], Step [103/139], Loss: 0.0601\n",
            "Epoch [29/30], Step [104/139], Loss: 0.0712\n",
            "Epoch [29/30], Step [105/139], Loss: 0.0879\n",
            "Epoch [29/30], Step [106/139], Loss: 0.0696\n",
            "Epoch [29/30], Step [107/139], Loss: 0.0599\n",
            "Epoch [29/30], Step [108/139], Loss: 0.0542\n",
            "Epoch [29/30], Step [109/139], Loss: 0.0697\n",
            "Epoch [29/30], Step [110/139], Loss: 0.0840\n",
            "Epoch [29/30], Step [111/139], Loss: 0.0692\n",
            "Epoch [29/30], Step [112/139], Loss: 0.0901\n",
            "Epoch [29/30], Step [113/139], Loss: 0.0700\n",
            "Epoch [29/30], Step [114/139], Loss: 0.0591\n",
            "Epoch [29/30], Step [115/139], Loss: 0.0701\n",
            "Epoch [29/30], Step [116/139], Loss: 0.0694\n",
            "Epoch [29/30], Step [117/139], Loss: 0.0565\n",
            "Epoch [29/30], Step [118/139], Loss: 0.0652\n",
            "Epoch [29/30], Step [119/139], Loss: 0.0937\n",
            "Epoch [29/30], Step [120/139], Loss: 0.0757\n",
            "Epoch [29/30], Step [121/139], Loss: 0.0689\n",
            "Epoch [29/30], Step [122/139], Loss: 0.0854\n",
            "Epoch [29/30], Step [123/139], Loss: 0.0657\n",
            "Epoch [29/30], Step [124/139], Loss: 0.0648\n",
            "Epoch [29/30], Step [125/139], Loss: 0.0562\n",
            "Epoch [29/30], Step [126/139], Loss: 0.0671\n",
            "Epoch [29/30], Step [127/139], Loss: 0.1058\n",
            "Epoch [29/30], Step [128/139], Loss: 0.0573\n",
            "Epoch [29/30], Step [129/139], Loss: 0.0794\n",
            "Epoch [29/30], Step [130/139], Loss: 0.0617\n",
            "Epoch [29/30], Step [131/139], Loss: 0.0665\n",
            "Epoch [29/30], Step [132/139], Loss: 0.0919\n",
            "Epoch [29/30], Step [133/139], Loss: 0.0496\n",
            "Epoch [29/30], Step [134/139], Loss: 0.0625\n",
            "Epoch [29/30], Step [135/139], Loss: 0.0587\n",
            "Epoch [29/30], Step [136/139], Loss: 0.0666\n",
            "Epoch [29/30], Step [137/139], Loss: 0.0696\n",
            "Epoch [29/30], Step [138/139], Loss: 0.0923\n",
            "Epoch [29/30], Step [139/139], Loss: 0.0725\n",
            "Start validation #29\n",
            "Validation #29  Average Loss: 0.0811\n",
            "Best performance at epoch: 29\n",
            "Save model in ./saved/LSTM\n",
            "Epoch [30/30], Step [1/139], Loss: 0.0616\n",
            "Epoch [30/30], Step [2/139], Loss: 0.0654\n",
            "Epoch [30/30], Step [3/139], Loss: 0.0601\n",
            "Epoch [30/30], Step [4/139], Loss: 0.0879\n",
            "Epoch [30/30], Step [5/139], Loss: 0.0598\n",
            "Epoch [30/30], Step [6/139], Loss: 0.0605\n",
            "Epoch [30/30], Step [7/139], Loss: 0.0714\n",
            "Epoch [30/30], Step [8/139], Loss: 0.0738\n",
            "Epoch [30/30], Step [9/139], Loss: 0.0793\n",
            "Epoch [30/30], Step [10/139], Loss: 0.0763\n",
            "Epoch [30/30], Step [11/139], Loss: 0.0757\n",
            "Epoch [30/30], Step [12/139], Loss: 0.0608\n",
            "Epoch [30/30], Step [13/139], Loss: 0.0983\n",
            "Epoch [30/30], Step [14/139], Loss: 0.0870\n",
            "Epoch [30/30], Step [15/139], Loss: 0.0671\n",
            "Epoch [30/30], Step [16/139], Loss: 0.0727\n",
            "Epoch [30/30], Step [17/139], Loss: 0.0756\n",
            "Epoch [30/30], Step [18/139], Loss: 0.0649\n",
            "Epoch [30/30], Step [19/139], Loss: 0.0575\n",
            "Epoch [30/30], Step [20/139], Loss: 0.0662\n",
            "Epoch [30/30], Step [21/139], Loss: 0.0574\n",
            "Epoch [30/30], Step [22/139], Loss: 0.0933\n",
            "Epoch [30/30], Step [23/139], Loss: 0.0639\n",
            "Epoch [30/30], Step [24/139], Loss: 0.0818\n",
            "Epoch [30/30], Step [25/139], Loss: 0.0729\n",
            "Epoch [30/30], Step [26/139], Loss: 0.0862\n",
            "Epoch [30/30], Step [27/139], Loss: 0.0582\n",
            "Epoch [30/30], Step [28/139], Loss: 0.0604\n",
            "Epoch [30/30], Step [29/139], Loss: 0.0537\n",
            "Epoch [30/30], Step [30/139], Loss: 0.0632\n",
            "Epoch [30/30], Step [31/139], Loss: 0.0644\n",
            "Epoch [30/30], Step [32/139], Loss: 0.0621\n",
            "Epoch [30/30], Step [33/139], Loss: 0.0822\n",
            "Epoch [30/30], Step [34/139], Loss: 0.0601\n",
            "Epoch [30/30], Step [35/139], Loss: 0.0869\n",
            "Epoch [30/30], Step [36/139], Loss: 0.0591\n",
            "Epoch [30/30], Step [37/139], Loss: 0.0900\n",
            "Epoch [30/30], Step [38/139], Loss: 0.0688\n",
            "Epoch [30/30], Step [39/139], Loss: 0.0798\n",
            "Epoch [30/30], Step [40/139], Loss: 0.0810\n",
            "Epoch [30/30], Step [41/139], Loss: 0.0662\n",
            "Epoch [30/30], Step [42/139], Loss: 0.0794\n",
            "Epoch [30/30], Step [43/139], Loss: 0.0707\n",
            "Epoch [30/30], Step [44/139], Loss: 0.0582\n",
            "Epoch [30/30], Step [45/139], Loss: 0.0713\n",
            "Epoch [30/30], Step [46/139], Loss: 0.0575\n",
            "Epoch [30/30], Step [47/139], Loss: 0.0693\n",
            "Epoch [30/30], Step [48/139], Loss: 0.0664\n",
            "Epoch [30/30], Step [49/139], Loss: 0.0572\n",
            "Epoch [30/30], Step [50/139], Loss: 0.0606\n",
            "Epoch [30/30], Step [51/139], Loss: 0.1092\n",
            "Epoch [30/30], Step [52/139], Loss: 0.0682\n",
            "Epoch [30/30], Step [53/139], Loss: 0.0862\n",
            "Epoch [30/30], Step [54/139], Loss: 0.0914\n",
            "Epoch [30/30], Step [55/139], Loss: 0.0574\n",
            "Epoch [30/30], Step [56/139], Loss: 0.0668\n",
            "Epoch [30/30], Step [57/139], Loss: 0.0578\n",
            "Epoch [30/30], Step [58/139], Loss: 0.0447\n",
            "Epoch [30/30], Step [59/139], Loss: 0.0515\n",
            "Epoch [30/30], Step [60/139], Loss: 0.0588\n",
            "Epoch [30/30], Step [61/139], Loss: 0.0594\n",
            "Epoch [30/30], Step [62/139], Loss: 0.0774\n",
            "Epoch [30/30], Step [63/139], Loss: 0.0596\n",
            "Epoch [30/30], Step [64/139], Loss: 0.0753\n",
            "Epoch [30/30], Step [65/139], Loss: 0.0850\n",
            "Epoch [30/30], Step [66/139], Loss: 0.0830\n",
            "Epoch [30/30], Step [67/139], Loss: 0.0597\n",
            "Epoch [30/30], Step [68/139], Loss: 0.1025\n",
            "Epoch [30/30], Step [69/139], Loss: 0.0686\n",
            "Epoch [30/30], Step [70/139], Loss: 0.0672\n",
            "Epoch [30/30], Step [71/139], Loss: 0.0845\n",
            "Epoch [30/30], Step [72/139], Loss: 0.0696\n",
            "Epoch [30/30], Step [73/139], Loss: 0.0533\n",
            "Epoch [30/30], Step [74/139], Loss: 0.0822\n",
            "Epoch [30/30], Step [75/139], Loss: 0.0854\n",
            "Epoch [30/30], Step [76/139], Loss: 0.1014\n",
            "Epoch [30/30], Step [77/139], Loss: 0.0743\n",
            "Epoch [30/30], Step [78/139], Loss: 0.0694\n",
            "Epoch [30/30], Step [79/139], Loss: 0.0803\n",
            "Epoch [30/30], Step [80/139], Loss: 0.0654\n",
            "Epoch [30/30], Step [81/139], Loss: 0.0894\n",
            "Epoch [30/30], Step [82/139], Loss: 0.0599\n",
            "Epoch [30/30], Step [83/139], Loss: 0.0664\n",
            "Epoch [30/30], Step [84/139], Loss: 0.0740\n",
            "Epoch [30/30], Step [85/139], Loss: 0.0905\n",
            "Epoch [30/30], Step [86/139], Loss: 0.0638\n",
            "Epoch [30/30], Step [87/139], Loss: 0.0624\n",
            "Epoch [30/30], Step [88/139], Loss: 0.0781\n",
            "Epoch [30/30], Step [89/139], Loss: 0.0832\n",
            "Epoch [30/30], Step [90/139], Loss: 0.0649\n",
            "Epoch [30/30], Step [91/139], Loss: 0.0972\n",
            "Epoch [30/30], Step [92/139], Loss: 0.0791\n",
            "Epoch [30/30], Step [93/139], Loss: 0.0795\n",
            "Epoch [30/30], Step [94/139], Loss: 0.0653\n",
            "Epoch [30/30], Step [95/139], Loss: 0.0646\n",
            "Epoch [30/30], Step [96/139], Loss: 0.0716\n",
            "Epoch [30/30], Step [97/139], Loss: 0.0926\n",
            "Epoch [30/30], Step [98/139], Loss: 0.0984\n",
            "Epoch [30/30], Step [99/139], Loss: 0.0893\n",
            "Epoch [30/30], Step [100/139], Loss: 0.0722\n",
            "Epoch [30/30], Step [101/139], Loss: 0.0657\n",
            "Epoch [30/30], Step [102/139], Loss: 0.0625\n",
            "Epoch [30/30], Step [103/139], Loss: 0.0560\n",
            "Epoch [30/30], Step [104/139], Loss: 0.0780\n",
            "Epoch [30/30], Step [105/139], Loss: 0.0703\n",
            "Epoch [30/30], Step [106/139], Loss: 0.0823\n",
            "Epoch [30/30], Step [107/139], Loss: 0.0788\n",
            "Epoch [30/30], Step [108/139], Loss: 0.0582\n",
            "Epoch [30/30], Step [109/139], Loss: 0.0642\n",
            "Epoch [30/30], Step [110/139], Loss: 0.0788\n",
            "Epoch [30/30], Step [111/139], Loss: 0.0741\n",
            "Epoch [30/30], Step [112/139], Loss: 0.0822\n",
            "Epoch [30/30], Step [113/139], Loss: 0.0578\n",
            "Epoch [30/30], Step [114/139], Loss: 0.0642\n",
            "Epoch [30/30], Step [115/139], Loss: 0.0588\n",
            "Epoch [30/30], Step [116/139], Loss: 0.0563\n",
            "Epoch [30/30], Step [117/139], Loss: 0.0697\n",
            "Epoch [30/30], Step [118/139], Loss: 0.0546\n",
            "Epoch [30/30], Step [119/139], Loss: 0.0674\n",
            "Epoch [30/30], Step [120/139], Loss: 0.0819\n",
            "Epoch [30/30], Step [121/139], Loss: 0.0874\n",
            "Epoch [30/30], Step [122/139], Loss: 0.0884\n",
            "Epoch [30/30], Step [123/139], Loss: 0.0743\n",
            "Epoch [30/30], Step [124/139], Loss: 0.0653\n",
            "Epoch [30/30], Step [125/139], Loss: 0.0657\n",
            "Epoch [30/30], Step [126/139], Loss: 0.0690\n",
            "Epoch [30/30], Step [127/139], Loss: 0.0628\n",
            "Epoch [30/30], Step [128/139], Loss: 0.0823\n",
            "Epoch [30/30], Step [129/139], Loss: 0.0864\n",
            "Epoch [30/30], Step [130/139], Loss: 0.0737\n",
            "Epoch [30/30], Step [131/139], Loss: 0.0541\n",
            "Epoch [30/30], Step [132/139], Loss: 0.0922\n",
            "Epoch [30/30], Step [133/139], Loss: 0.0793\n",
            "Epoch [30/30], Step [134/139], Loss: 0.0813\n",
            "Epoch [30/30], Step [135/139], Loss: 0.0792\n",
            "Epoch [30/30], Step [136/139], Loss: 0.0773\n",
            "Epoch [30/30], Step [137/139], Loss: 0.0744\n",
            "Epoch [30/30], Step [138/139], Loss: 0.0615\n",
            "Epoch [30/30], Step [139/139], Loss: 0.0900\n",
            "Start validation #30\n",
            "Validation #30  Average Loss: 0.0804\n",
            "Best performance at epoch: 30\n",
            "Save model in ./saved/LSTM\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SEfORToUXxwU",
        "colab_type": "text"
      },
      "source": [
        "## 12. 저장된 모델 불러오기 및 test\n",
        "\n",
        "학습한 모델의 성능을 테스트합니다. 저장한 모델 파일을 [torch.load](https://pytorch.org/docs/stable/torch.html?highlight=load#torch.load)를 통해 불러옵니다. 위에서 학습을 끝까지 진행하지 않았다면, 아래의 주석 처리된 부분을 주석 해제하면, 제공해드린 미리 학습시킨 모델을 불러올 수 있습니다. \n",
        "\n",
        "이렇게 불러오면 우리가 얻게 되는 건 아까 저장한 **check_point** 딕셔너리입니다. 딕셔너리에 저장한 모델의 파라미터는 **'net'** key에 저장해두었습니다. 이를 불러와 **state_dict**에 저장합니다. 이렇게 불러온 모델의 파라미터를 모델에 실제로 로드하기 위해서는 [nn.Module.load_state_dict](https://pytorch.org/docs/stable/torch.html?highlight=load#torch.load)를 사용하면 됩니다. \n",
        "\n",
        "다음을 읽고 코드를 완성해보세요.\n",
        "- **model_path**의 경로에 있는 모델 파일을 로드하여, 이를 **check_point** 변수에 저장합니다. \n",
        "- **check_point** 딕셔너리에 접근하여 모델의 파라미터를 **state_dict** 변수에 저장합니다.\n",
        "- **state_dict**의 파라미터들을 우리 모델에 로드합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dgu3AKRyXxwW",
        "colab_type": "code",
        "outputId": "838a2cb3-ca81-4f81-8e24-030c7084968c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        }
      },
      "source": [
        "model_path = './saved/LSTM/best_model.pt'\n",
        "# model_path = './saved/pretrained/LSTM/best_model.pt' # 모델 학습을 끝까지 진행하지 않은 경우에 사용\n",
        "model = SimpleLSTM() # 아래의 모델 불러오기를 정확히 구현했는지 확인하기 위해 새로 모델을 선언하여 학습 이전 상태로 초기화\n",
        "# 코드 시작\n",
        "checkpoint = torch.load(model_path)\n",
        "state_dict = checkpoint['net']\n",
        "model.load_state_dict(state_dict)\n",
        "# 코드 종료"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "IncompatibleKeys(missing_keys=[], unexpected_keys=[])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rYTV-JX2Xxwg",
        "colab_type": "text"
      },
      "source": [
        "마지막으로 모델의 성능을 테스트합니다. 베이스라인 성능을 뛰어 넘었다는 문구가 나오면 성공적으로 진행한 것입니다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "puBCf_3rXxwj",
        "colab_type": "code",
        "outputId": "8988cd93-7850-44f7-9773-6560ced08681",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        }
      },
      "source": [
        "test(model, test_loader, criterion, baseline_loss)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Start test..\n",
            "Test  Average Loss: 0.0889  Baseline Loss: 0.0945\n",
            "베이스라인 성능을 뛰어 넘었습니다!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WAXAaW8slq8o",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}