Documentation - Unified Processing Engine.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Unified Processing Engine\n",
    "#### Documentation | Version: 0.6.2 | Updated 2018.7.31\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "The processing engine is at the heart of many hardware-accelerator architectures. This PE is unified implementation that can be topologically configured to support various Deep Neural Network (DNN) accelerator architectures. Based on its configurations, it supports inner products with varying amounts of spatial and temporal parallelism, nonlinear activation functions, scalar addition, scalar multiplication, and max functions. Many existing architectures and dataflows, such as Eyeriss, Flexflow, and ShiDianNao are special cases of this PE."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[36mpath\u001b[39m: \u001b[32mString\u001b[39m = \u001b[32m\"\"\"\n",
       "C:\\Users\\RyanL\\OneDrive\\Research\\SEAL\\processing-engine/source/load-ivy.sc\n",
       "\"\"\"\u001b[39m"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val path = System.getProperty(\"user.dir\") + \"/source/load-ivy.sc\"\n",
    "interp.load.module(ammonite.ops.Path(java.nio.file.FileSystems.getDefault().getPath(path)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[32mimport \u001b[39m\u001b[36mchisel3._\r\n",
       "\u001b[39m\r\n",
       "\u001b[32mimport \u001b[39m\u001b[36mchisel3.util._\r\n",
       "\u001b[39m\r\n",
       "\u001b[32mimport \u001b[39m\u001b[36mchisel3.iotesters.{ChiselFlatSpec, Driver, PeekPokeTester}\r\n",
       "\r\n",
       "\u001b[39m\r\n",
       "\u001b[32mimport \u001b[39m\u001b[36mscala.math.pow\u001b[39m"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import chisel3._\n",
    "import chisel3.util._\n",
    "import chisel3.iotesters.{ChiselFlatSpec, Driver, PeekPokeTester}\n",
    "\n",
    "import scala.math.pow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Register File\n",
    "\n",
    "##### Purpose\n",
    "The Register File (RF) is used as a buffer for weights and activations, as well as a scratch pad for the ALU. These are termed the Weight, Activation, and Scratch RFs. The Weight and Activation RFs interface between the External-PE Array and the IPU. The Scratch RF sits between the ALU and the NLU, with an interface to the External-PE Array\n",
    "\n",
    "##### I/O\n",
    "* Each RF has two buses in and two buses out: one for internal-PE usage and the other for inter-PE communication. These are termed *Internal Write*, *External Write*, *Internal Read*, and *External Read*&mdash;where 'Internal' denotes internal-PE and 'External' denotes inter-PE.\n",
    "* There are two separate control buses: one for the Internal Read/Write buses and one for the External Read/Write buses\n",
    "* Each of these control buses consists of a one-hot write enable, a one-hot read enable, a write address for each input, a read address for each output, and a select signal for each output for bypassing (see bypass section).\n",
    "\n",
    "##### Bypassing\n",
    "The bypass behavior is slightly tricky: the bus for inter-PE writing can be bypassed to the bus for internal-PE reading and the bus for internal-PE writing can be bypassed to the bus for inter-PE reading. In other words, bypasses cross between internal and external I/O. Since the input and output buses can be different sizes, *each* output signal has a select signal that must pick an input signal (of the appropriate bus). \n",
    "* Hard Bypassing is disallowed to prevent combinational loops.\n",
    "* For Firm Bypassing, this select signal *must* always be one-hot, else behavior is undefined. The RF's internal memory is limited to one SInt register per input and read/write addressing is unsupported&mdash;thus the RF acts as a pipelining register. Additionally, Firm Bypassing is disallowed when there is an i/o bus does not have its corresponding pair. *E.g.* if there is an Internal Read, there must be an External Write, *etc.*\n",
    "* For Soft Bypassing, this signal is used to indicate both when to bypass vs read, as well as which input to bypass from. When the select signal is all false, the RF will act as specified for a None Bypass. \n",
    "* For None Bypassing, the bypass select signal is deactivated and it each of the read/write signals can operate completely independently. It is up to the user to avoid read/write race conditions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defined \u001b[32mclass\u001b[39m \u001b[36mPartialRFConfig\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mPartialRFControl\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mRFConfig\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mRFControl\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mRF\u001b[39m"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class PartialRFConfig(\n",
    "        val numInputs: Int,\n",
    "        val numOutputs: Int,\n",
    "        val numCrossInputs: Int,\n",
    "        val addrWidth: Int,\n",
    "        val bpSoft: Boolean,\n",
    "        val bpFirm: Boolean)\n",
    "\n",
    "class PartialRFControl(c: PartialRFConfig) extends Bundle {\n",
    "    val wEnable = Vec(c.numInputs, Bool())\n",
    "    val rEnable = Vec(c.numOutputs, Bool())\n",
    "    val wAddr = if (!c.bpFirm) Some(Vec(c.numInputs, UInt(c.addrWidth.W))) else None\n",
    "    val rAddr = if (!c.bpFirm) Some(Vec(c.numOutputs, UInt(c.addrWidth.W))) else None\n",
    "    // Each output can select which input of the opposite bus to bypass from\n",
    "    val bpSel = if (c.bpSoft || c.bpFirm) Some(Vec(c.numOutputs, Vec(c.numCrossInputs, Bool()))) else None\n",
    "}\n",
    "\n",
    "class RFConfig(\n",
    "        val numIntInputs: Int,\n",
    "        val numExtInputs: Int,\n",
    "        val numIntOutputs: Int,\n",
    "        val numExtOutputs: Int,\n",
    "        val addrWidth: Int,\n",
    "        val dataWidth: Int,\n",
    "        val bpType: String) {\n",
    "    \n",
    "    val bpNone = (bpType == \"None\")\n",
    "    val bpSoft = (bpType == \"Soft\")\n",
    "    val bpFirm = (bpType == \"Firm\")\n",
    "    \n",
    "    require(bpNone || bpSoft || bpFirm, \"Invalid Bypass type.\\n\")\n",
    "    require(numIntInputs > 0 || numExtInputs > 0, \"Must have at least one input.\\n\")\n",
    "    require(numIntOutputs > 0 || numExtOutputs > 0, \"Must have at least one output.\\n\")\n",
    "    require(dataWidth > 0, \"Data bitwidth must be at least one.\\n\") \n",
    "    if (bpFirm) { require(addrWidth == 0, \"Address width must be 0 when Firm Bypassing.\\n\") }\n",
    "    \n",
    "    val intConfig = new PartialRFConfig(\n",
    "        numIntInputs, numIntOutputs, numExtOutputs, addrWidth, bpSoft, bpFirm)\n",
    "    \n",
    "    val extConfig = new PartialRFConfig(\n",
    "        numExtInputs, numExtOutputs, numIntOutputs, addrWidth, bpSoft, bpFirm)\n",
    "}\n",
    "\n",
    "class RFControl(c: RFConfig) extends Bundle {\n",
    "    \n",
    "    override def cloneType = (new RFControl(c)).asInstanceOf[this.type]\n",
    "    \n",
    "    val internal = if (c.numIntInputs > 0 || c.numIntOutputs > 0)\n",
    "        Some(new PartialRFControl(c.intConfig)) else None\n",
    "    val external = if (c.numExtInputs > 0 || c.numExtOutputs > 0)\n",
    "        Some(new PartialRFControl(c.extConfig)) else None\n",
    "}\n",
    "\n",
    "class RF(c: RFConfig) extends Module {\n",
    "    \n",
    "    val io = IO(new Bundle {\n",
    "        val control = Input(new RFControl(c))\n",
    "        val wInternal = Input(Vec(c.numIntInputs, SInt(c.dataWidth.W))) \n",
    "        val wExternal = Input(Vec(c.numExtInputs, SInt(c.dataWidth.W)))\n",
    "        val rInternal = Output(Vec(c.numIntOutputs, SInt(c.dataWidth.W)))\n",
    "        val rExternal = Output(Vec(c.numExtOutputs, SInt(c.dataWidth.W)))\n",
    "    })\n",
    "    \n",
    "    val dataRegister = if (!c.bpFirm) \n",
    "        Some(RegInit(Vec.fill(pow(2, c.addrWidth).toInt){0.S(c.dataWidth.W)})) else None\n",
    "    \n",
    "    // Need to bypass through a register to prevent combinational loops\n",
    "    val bpAny = c.bpSoft || c.bpFirm\n",
    "    val bpRegisterInt = if (bpAny && c.numIntInputs > 0)\n",
    "        Some(RegInit(Vec.fill(c.numIntInputs){0.S(c.dataWidth.W)})) else None\n",
    "    val bpRegisterExt = if (bpAny && c.numExtInputs > 0)\n",
    "        Some(RegInit(Vec.fill(c.numExtInputs){0.S(c.dataWidth.W)})) else None\n",
    "    \n",
    "    for (i <- 0 until c.numIntInputs) {\n",
    "        when (io.control.internal.get.wEnable(i)) {\n",
    "            if (!c.bpFirm) { dataRegister.get(io.control.internal.get.wAddr.get(i)) := io.wInternal(i) }\n",
    "            if (bpRegisterInt.isDefined) { bpRegisterInt.get(i) := io.wInternal(i) }\n",
    "        }\n",
    "    }\n",
    "    \n",
    "    for (i <- 0 until c.numExtInputs) {\n",
    "        when (io.control.external.get.wEnable(i)) {\n",
    "            if (!c.bpFirm) { dataRegister.get(io.control.external.get.wAddr.get(i)) := io.wExternal(i) }\n",
    "            if (bpRegisterExt.isDefined) { bpRegisterExt.get(i) := io.wExternal(i) }\n",
    "        }\n",
    "    }\n",
    "    \n",
    "    for (i <- 0 until c.numIntOutputs) {\n",
    "        when (io.control.internal.get.rEnable(i)) {\n",
    "            if (c.bpFirm) {\n",
    "                io.rInternal(i) := PriorityMux(io.control.internal.get.bpSel.get(i), bpRegisterExt.get)\n",
    "            } else if (c.bpSoft) {\n",
    "                when (io.control.internal.get.bpSel.get(i).contains(true.B)) {\n",
    "                    // External write bypasses to Internal read\n",
    "                    io.rInternal(i) := PriorityMux(io.control.internal.get.bpSel.get(i), bpRegisterExt.get)\n",
    "                } .otherwise {\n",
    "                    io.rInternal(i) := dataRegister.get(io.control.internal.get.rAddr.get(i))\n",
    "                }\n",
    "            } else {\n",
    "                io.rInternal(i) := dataRegister.get(io.control.internal.get.rAddr.get(i))\n",
    "            }\n",
    "        } .otherwise {\n",
    "            io.rInternal(i) := 0.S\n",
    "        }\n",
    "    }\n",
    "    \n",
    "    for (i <- 0 until c.numExtOutputs) {\n",
    "        when (io.control.external.get.rEnable(i)) {\n",
    "            if (c.bpFirm) {\n",
    "                io.rExternal(i) := PriorityMux(io.control.external.get.bpSel.get(i), bpRegisterInt.get)\n",
    "            } else if (c.bpSoft) {\n",
    "                when (io.control.external.get.bpSel.get(i).contains(true.B)) {\n",
    "                    // Internal write bypasses to External read\n",
    "                    io.rExternal(i) := PriorityMux(io.control.external.get.bpSel.get(i), bpRegisterInt.get)\n",
    "                } .otherwise {\n",
    "                    io.rExternal(i) := dataRegister.get(io.control.external.get.rAddr.get(i))\n",
    "                }\n",
    "            } else {\n",
    "                io.rExternal(i) := dataRegister.get(io.control.external.get.rAddr.get(i))\n",
    "            }\n",
    "        } .otherwise {\n",
    "            io.rExternal(i) := 0.S\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inner Product Unit\n",
    "\n",
    "##### Purpose\n",
    "The Inner Product Unit (IPU) performs an inner product between two vectors of a configurable length. For SIMD support, this should be configured to be wide. For stationary dataflows, such as Eyeriss, note that setting the width to 1 is equivalent to a scalar multiplication. The IPU sits between the Weight/Activation RFs and the ALU.\n",
    "\n",
    "##### I/O\n",
    "* The IPU takes two equi-dimension vector inputs.\n",
    "* Control is only necessary during a Firm Bypass; it selects which index of both of the inputs to bypass.\n",
    "* The primary output is the inner product of the two vectors, but during a Firm Bypass there are two extra outputs: one weight and one activation.\n",
    "\n",
    "##### Bypassing\n",
    "* None Bypassing simply outputs the inner product of the two vectors.\n",
    "* Firm Bypassing adds two outputs lines: one for a single weight and one for a single activation. Additionally, there is a select signal to select which index of the inputs to bypass.\n",
    "\n",
    "##### Future Plans\n",
    "* Hard Bypassing should remove the inner product line and keep the bypass lines.\n",
    "* The IPU should support FMA.\n",
    "* For much more complex PE designs, multiple outputs could be desirable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Definition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defined \u001b[32mclass\u001b[39m \u001b[36mIPUConfig\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mIPUOutput\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mIPU\u001b[39m"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class IPUConfig(val width: Int, val inBitWidth: Int, val outBitWidth: Int, val bpType: String) {\n",
    "    \n",
    "    require(width >= 1, \"Width must be at least one.\\n\")\n",
    "    require(List(\"None\", \"Firm\").contains(bpType), \"Bypass must be \\\"None\\\" or \\\"Firm\\\".\\n\")\n",
    "    require(inBitWidth > 0 && outBitWidth > 0, \"Data bitwidth must be greater than 0\\n\")\n",
    "    \n",
    "    val bpFirm = (bpType == \"Firm\")\n",
    "}\n",
    "\n",
    "class IPUOutput(outBitWidth: Int, bp: Boolean) extends Bundle {\n",
    "    \n",
    "    override def cloneType = (new IPUOutput(outBitWidth, bp)).asInstanceOf[this.type]\n",
    "    \n",
    "    val innerProd = SInt(outBitWidth.W)\n",
    "    // Extending the bitwidths for consistency\n",
    "    val bpWeight = if (bp) Some(SInt(outBitWidth.W)) else None\n",
    "    val bpActvtn = if (bp) Some(SInt(outBitWidth.W)) else None\n",
    "}\n",
    "\n",
    "\n",
    "class IPU(c: IPUConfig) extends Module {\n",
    "\n",
    "    val io = IO(new Bundle {\n",
    "        val bpSel = if (c.bpFirm) Some(Input(Vec(c.width, Bool()))) else None\n",
    "        val weightIn = Input(Vec(c.width, SInt(c.inBitWidth.W)))\n",
    "        val actvtnIn = Input(Vec(c.width, SInt(c.inBitWidth.W)))\n",
    "        val out = Output(new IPUOutput(c.outBitWidth, c.bpFirm))\n",
    "    })\n",
    "    \n",
    "    private class PMult extends Module {\n",
    "        val io = IO(new Bundle {\n",
    "            val weightVec = Input(Vec(c.width, SInt(c.inBitWidth.W)))\n",
    "            val actvtnVec = Input(Vec(c.width, SInt(c.inBitWidth.W)))\n",
    "            val pairwiseProd = Output(Vec(c.width, SInt(c.outBitWidth.W)))\n",
    "        })\n",
    "        io.pairwiseProd := (io.weightVec zip io.actvtnVec).map { case(a, b) => a * b }\n",
    "    }\n",
    "    \n",
    "    private class SumTree extends Module {\n",
    "        val io = IO(new Bundle {\n",
    "            val inVec = Input(Vec(c.width, SInt(c.outBitWidth.W)))\n",
    "            val sum = Output(SInt(c.outBitWidth.W))\n",
    "        })\n",
    "        \n",
    "        // Recursively creates a balanced syntax tree\n",
    "        private def adjReduce[A](xs: List[A], op: (A, A) => A): A = xs match {\n",
    "            case List(single) => single\n",
    "            case default => {\n",
    "                val grouped = default.grouped(2).toList\n",
    "                val result = for (g <- grouped) yield { g match {\n",
    "                    case List(a, b) => op(a, b)\n",
    "                    case List(x) => x\n",
    "                }}\n",
    "                adjReduce(result, op)\n",
    "            }\n",
    "        }\n",
    "        \n",
    "        io.sum := adjReduce(io.inVec.toList, (x: SInt, y: SInt) => x + y)\n",
    "    }\n",
    "    \n",
    "    private val pMult = Module(new PMult)\n",
    "    pMult.io.weightVec := io.weightIn\n",
    "    pMult.io.actvtnVec := io.actvtnIn\n",
    "    \n",
    "    private val sumTree = Module(new SumTree)\n",
    "    sumTree.io.inVec := pMult.io.pairwiseProd\n",
    "    \n",
    "    io.out.innerProd := sumTree.io.sum\n",
    "    \n",
    "    if (c.bpFirm) {\n",
    "        io.out.bpWeight.get := PriorityMux(io.bpSel.get, io.weightIn)\n",
    "        io.out.bpActvtn.get := PriorityMux(io.bpSel.get, io.actvtnIn)\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ALU\n",
    "\n",
    "##### Purpose\n",
    "The ALU supports four functions: Addition, Maximum, Accumulation, and Identity. Accumulation and Addition enable Convolution layers, while Maximum enables Max-Pooling layers. *It is important to note the operands of each operation*. Accumulation acts on the inner product from the IPU and the feedback from the Scratch RF. Addition and Maximum operate on the bypassed weights and activations. Identity acts on the inner product alone. The ALU sits between the IPU and the Scratch RF.\n",
    "\n",
    "##### I/O\n",
    "* The ALU has two inputs: one bus from the IPU and one feedback line from the Scratch RF.\n",
    "* The bus from the IPU contains the inner product and may or may not contain a weight and activation bypassed from the Weight and Activation RFs. It is enforced that these bypass lines exist if the ALU is set to support addition or maximum.\n",
    "* It outputs a single signal to the Scratch RF.\n",
    "* Control consists of a bundle of enable signals: one for each supported function. The enables are interpreted with priority in the following order: Identity, Addition, Maximum, Accumulation. If all of the enables are low, the output will read zero.\n",
    "\n",
    "##### Bypassing\n",
    "As of right now, bypassing is only supported via the Identity operation. However, there are plans to implement it via a select signal during Soft and Firm Bypassing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Definition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defined \u001b[32mclass\u001b[39m \u001b[36mALUConfig\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mALUFSel\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mALU\u001b[39m"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class ALUConfig(val dataWidth: Int, val funcs: List[String]) {\n",
    "    \n",
    "    require(funcs.length > 0, \"Must support at least one function.\")\n",
    "    for(x <- funcs) { \n",
    "        require(List(\"Identity\", \"Add\", \"Max\", \"Accumulate\").contains(x), \"Unsupported function.\")\n",
    "    }\n",
    "    \n",
    "    val idnSupp = funcs.contains(\"Identity\")\n",
    "    val addSupp = funcs.contains(\"Add\")\n",
    "    val maxSupp = funcs.contains(\"Max\")\n",
    "    val accSupp = funcs.contains(\"Accumulate\")\n",
    "    val addBypassIn = addSupp || maxSupp\n",
    "}\n",
    "\n",
    "class ALUFSel(c: ALUConfig) extends Bundle {\n",
    "    \n",
    "    override def cloneType = (new ALUFSel(c)).asInstanceOf[this.type]\n",
    "    \n",
    "    // Priority is given from top to bottom\n",
    "    val idnEnable = if (c.idnSupp) Some(Bool()) else None\n",
    "    val addEnable = if (c.addSupp) Some(Bool()) else None\n",
    "    val maxEnable = if (c.maxSupp) Some(Bool()) else None\n",
    "    val accEnable = if (c.accSupp) Some(Bool()) else None\n",
    "}\n",
    "\n",
    "class ALU(c: ALUConfig) extends Module {\n",
    " \n",
    "    val io = IO(new Bundle {\n",
    "        val fSel = Input(new ALUFSel(c))\n",
    "        val ipu = Input(new IPUOutput(c.dataWidth, c.addBypassIn))\n",
    "        val rf = if (c.accSupp) Some(Input(SInt(c.dataWidth.W))) else None\n",
    "        val out = Output(SInt(c.dataWidth.W))\n",
    "    })\n",
    "    \n",
    "    // The inner \"OrElse\" clauses are logically unnecessary,\n",
    "    // but Chisel can't infer that.\n",
    "    when (io.fSel.idnEnable.getOrElse(false.B)) {\n",
    "        io.out := io.ipu.innerProd\n",
    "    } .elsewhen (io.fSel.addEnable.getOrElse(false.B)) {\n",
    "        io.out := io.ipu.bpWeight.getOrElse(0.S) + io.ipu.bpActvtn.getOrElse(0.S)\n",
    "    } .elsewhen (io.fSel.maxEnable.getOrElse(false.B)) {\n",
    "        when (io.ipu.bpWeight.getOrElse(0.S) > io.ipu.bpActvtn.getOrElse(0.S)) {\n",
    "            io.out := io.ipu.bpWeight.getOrElse(0.S)\n",
    "        } .otherwise {\n",
    "            io.out := io.ipu.bpActvtn.getOrElse(0.S)\n",
    "        }\n",
    "    } .elsewhen (io.fSel.accEnable.getOrElse(false.B)) {\n",
    "        io.out := io.ipu.innerProd + io.rf.getOrElse(0.S)\n",
    "    } .otherwise {\n",
    "        io.out := 0.S\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Nonlinear Unit\n",
    "\n",
    "##### Purpose\n",
    "The Nonlinear Unit (NLU) performs the nonlinear activations functions present in neural networks. As of right now, it supports the Identity function and ReLu, but there are plans to implement sigmoid and tanh using LUTs and linear interpolation. The NLU sits between the Scratch RF and the inter-PE network.\n",
    "\n",
    "##### I/O\n",
    "* The NLU sports a single input and a single output, each of an independent bitwidth.\n",
    "* Control is similar to the ALU. There is a one-hot control bus that encodes which function to perform. Provided multiple signals are active, the priority is as follows: Identity, ReLu. (Then tanh, sinh once implemented.)\n",
    "\n",
    "##### Bypassing\n",
    "Bypassing is only supported via the Identity function and with direct reads to the Scratch RF. There are no plans to implement other Bypass functions, as this seems sufficient."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Defintion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defined \u001b[32mclass\u001b[39m \u001b[36mNLUConfig\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mNLUFSel\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mNLU\u001b[39m"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class NLUConfig(val inBitWidth: Int, val outBitWidth: Int, val funcs: List[String]) {\n",
    "    \n",
    "    for(x <- funcs) {\n",
    "        require(List(\"Identity\", \"ReLu\").contains(x), \"Unsupported Function\")\n",
    "    }\n",
    "    \n",
    "    val idSupp = funcs.contains(\"Identity\")\n",
    "    val reluSupp = funcs.contains(\"ReLu\")\n",
    "    val tanhSupp = false //funcs.contains(\"tanh\")\n",
    "    val sinhSupp = false //funcs.contains(\"sinh\")\n",
    "}\n",
    "\n",
    "class NLUFSel(c: NLUConfig) extends Bundle {\n",
    "    \n",
    "    override def cloneType = (new NLUFSel(c)).asInstanceOf[this.type]\n",
    "    \n",
    "    val idEnable = if (c.idSupp) Some(Bool()) else None\n",
    "    val reluEnable = if (c.reluSupp) Some(Bool()) else None\n",
    "    val tanhEnable = if (c.tanhSupp) Some(Bool()) else None \n",
    "    val sinhEnable = if (c.sinhSupp) Some(Bool()) else None\n",
    "}\n",
    "\n",
    "class NLU(c: NLUConfig) extends Module {\n",
    "    \n",
    "    val io = IO(new Bundle {\n",
    "        val fSel = Input(new NLUFSel(c))\n",
    "        val in = Input(SInt(c.inBitWidth.W))\n",
    "        val out = Output(SInt(c.outBitWidth.W))\n",
    "    })\n",
    "    \n",
    "    when (io.fSel.idEnable.getOrElse(false.B)) {\n",
    "        io.out := io.in\n",
    "    } .elsewhen (io.fSel.reluEnable.getOrElse(false.B)) {\n",
    "        when (io.in.data > 0.S) {\n",
    "            io.out := io.in.data\n",
    "        } .otherwise {\n",
    "            io.out := 0.S\n",
    "        }\n",
    "    } .elsewhen (io.fSel.tanhEnable.getOrElse(false.B)) {\n",
    "        // TODO\n",
    "        io.out := 0.S\n",
    "    } .elsewhen (io.fSel.sinhEnable.getOrElse(false.B)) {\n",
    "        // TODO\n",
    "        io.out := 0.S\n",
    "    } .otherwise {\n",
    "        io.out := 0.S\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Control\n",
    "\n",
    "For maximum configurability, control is implemented via a State Machine (Moore) and Decoder. The state machine must have its states and state transitions configured. The Decoder must have its input-output map configured."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Decoder\n",
    "\n",
    "##### Purpose\n",
    "The decoder acts as is typical in processors: it converts the state information into all the control signals associated with the appropriate state. It acts as the interface between the State Machine and every other module in the PE. \n",
    "\n",
    "##### I/O\n",
    "* The decoder has a single input from the State Machine.\n",
    "* It also has a control bus output going to all the Memory, *viz.* Weight RF, Activation RF, Scratch RF. This bus controls Read/Write Addressing and Enabling, as well as Bypassing\n",
    "* It has another control bus output to all Processing Units, *viz.* IPU, ALU, NLU. This bus controls Function Selecting and Bypassing.\n",
    "\n",
    "##### Configuring\n",
    "Since the Decoder needs to know almost everything about the PE, it takes the full PE configuration. The part that configures the decoder's logic is the set of functions that start with \"decode\". There is one for each module. To implement each of these functions, they must generate a Wire for the appropriate bus type and connect each of the appropriate signals to the desired literal, then return it. For example, decodeWeightRF must generate a Wire\\[RFControl\\] and set the signals that exist."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[36mrsDecodeWeightRF\u001b[39m: (\u001b[32mUInt\u001b[39m, \u001b[32mRFConfig\u001b[39m) => \u001b[32mRFControl\u001b[39m = <function2>\r\n",
       "\u001b[36mrsDecodeActvtnRF\u001b[39m: (\u001b[32mUInt\u001b[39m, \u001b[32mRFConfig\u001b[39m) => \u001b[32mRFControl\u001b[39m = <function2>\r\n",
       "\u001b[36mrsDecodeScratchRF\u001b[39m: (\u001b[32mUInt\u001b[39m, \u001b[32mRFConfig\u001b[39m) => \u001b[32mRFControl\u001b[39m = <function2>\r\n",
       "\u001b[36mrsDecodeIPU\u001b[39m: (\u001b[32mUInt\u001b[39m, \u001b[32mIPUConfig\u001b[39m) => \u001b[32mVec\u001b[39m[\u001b[32mBool\u001b[39m] = <function2>\r\n",
       "\u001b[36mrsDecodeALU\u001b[39m: (\u001b[32mUInt\u001b[39m, \u001b[32mALUConfig\u001b[39m) => \u001b[32mALUFSel\u001b[39m = <function2>\r\n",
       "\u001b[36mrsDecodeNLU\u001b[39m: (\u001b[32mUInt\u001b[39m, \u001b[32mNLUConfig\u001b[39m) => \u001b[32mNLUFSel\u001b[39m = <function2>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "/*\n",
    " * Row stationary 1D convolution\n",
    " *\n",
    " * (1, 2, 3) * (1, 2, 3, 4, 5) = (14, 20, 26)\n",
    " */\n",
    "\n",
    "val rsDecodeWeightRF = (state: UInt, c: RFConfig) => {\n",
    "    \n",
    "    val ctrl = Wire(new RFControl(c))\n",
    "    \n",
    "    when (state === 0.U || state === 1.U || state === 2.U) {\n",
    "        ctrl.external.get.wEnable(0) := true.B\n",
    "        ctrl.external.get.wAddr.get(0) := state\n",
    "    } .otherwise {\n",
    "        ctrl.external.get.wEnable(0) := false.B\n",
    "        ctrl.external.get.wAddr.get(0) := DontCare\n",
    "    }\n",
    "    \n",
    "    when (state >= 1.U && state <= 9.U) {\n",
    "        ctrl.internal.get.rEnable(0) := true.B\n",
    "        ctrl.internal.get.rAddr.get(0) := (state - 1.U) % 3.U\n",
    "    } .otherwise {\n",
    "        ctrl.internal.get.rEnable(0) := false.B\n",
    "        ctrl.internal.get.rAddr.get(0) := DontCare\n",
    "    }\n",
    "\n",
    "    ctrl\n",
    "}\n",
    "\n",
    "val rsDecodeActvtnRF = (state: UInt, c: RFConfig) => {\n",
    "    \n",
    "    val ctrl = Wire(new RFControl(c))\n",
    "    \n",
    "    when (state <= 8.U) {\n",
    "        ctrl.external.get.wEnable(0) := true.B\n",
    "        ctrl.external.get.wAddr.get(0) := state % 3.U\n",
    "    } .otherwise {\n",
    "        ctrl.external.get.wEnable(0) := false.B\n",
    "        ctrl.external.get.wAddr.get(0) := DontCare\n",
    "    }\n",
    "    \n",
    "    when (state >= 1.U && state <= 9.U) {\n",
    "        ctrl.internal.get.rEnable(0) := true.B\n",
    "        ctrl.internal.get.rAddr.get(0) := (state - 1.U) % 3.U\n",
    "    } .otherwise {\n",
    "        ctrl.internal.get.rEnable(0) := false.B\n",
    "        ctrl.internal.get.rAddr.get(0) := DontCare\n",
    "    }\n",
    "    \n",
    "    ctrl\n",
    "}\n",
    "\n",
    "val rsDecodeScratchRF = (state: UInt, c: RFConfig) => {\n",
    "    \n",
    "    val ctrl = Wire(new RFControl(c))\n",
    "    \n",
    "    when (state >= 1.U && state <= 9.U) {\n",
    "        ctrl.internal.get.wEnable(0) := true.B\n",
    "    } .otherwise {\n",
    "        ctrl.internal.get.wEnable(0) := false.B\n",
    "    }\n",
    "    \n",
    "    when (state >= 1.U && state <= 10.U && state =/= 4.U && state =/= 7.U) {\n",
    "        ctrl.internal.get.rEnable(0) := true.B\n",
    "    } .otherwise {\n",
    "        ctrl.internal.get.rEnable(0) := false.B\n",
    "    }\n",
    "    \n",
    "    when (state === 1.U) {\n",
    "        ctrl.internal.get.rAddr.get(0) := DontCare\n",
    "        ctrl.internal.get.wAddr.get(0) := 0.U\n",
    "    } .elsewhen (state === 2.U) {\n",
    "        ctrl.internal.get.rAddr.get(0) := 0.U\n",
    "        ctrl.internal.get.wAddr.get(0) := 1.U\n",
    "    } .elsewhen (state === 3.U) {\n",
    "        ctrl.internal.get.rAddr.get(0) := 1.U\n",
    "        ctrl.internal.get.wAddr.get(0) := 0.U\n",
    "    } .elsewhen (state === 4.U) {\n",
    "        ctrl.internal.get.rAddr.get(0) := DontCare\n",
    "        ctrl.internal.get.wAddr.get(0) := 1.U\n",
    "    } .elsewhen (state === 5.U) {\n",
    "        ctrl.internal.get.rAddr.get(0) := 1.U\n",
    "        ctrl.internal.get.wAddr.get(0) := 2.U\n",
    "    } .elsewhen (state === 6.U) {\n",
    "        ctrl.internal.get.rAddr.get(0) := 2.U\n",
    "        ctrl.internal.get.wAddr.get(0) := 1.U\n",
    "    } .elsewhen (state === 7.U) {\n",
    "        ctrl.internal.get.rAddr.get(0) := DontCare\n",
    "        ctrl.internal.get.wAddr.get(0) := 2.U\n",
    "    } .elsewhen (state === 8.U) {\n",
    "        ctrl.internal.get.rAddr.get(0) := 2.U\n",
    "        ctrl.internal.get.wAddr.get(0) := 3.U\n",
    "    } .elsewhen (state === 9.U) {\n",
    "        ctrl.internal.get.rAddr.get(0) := 3.U\n",
    "        ctrl.internal.get.wAddr.get(0) := 2.U\n",
    "    } .elsewhen (state === 10.U) {\n",
    "        ctrl.internal.get.rAddr.get(0) := 2.U\n",
    "        ctrl.internal.get.wAddr.get(0) := DontCare\n",
    "    } .otherwise {\n",
    "        ctrl.internal.get.rAddr.get(0) := DontCare\n",
    "        ctrl.internal.get.wAddr.get(0) := DontCare\n",
    "    }\n",
    "    \n",
    "    ctrl\n",
    "}\n",
    "\n",
    "val rsDecodeIPU = (state: UInt, c: IPUConfig) => {\n",
    "    val bpSel = Wire(Vec(c.width, Bool()))\n",
    "    bpSel\n",
    "}\n",
    "\n",
    "val rsDecodeALU = (state: UInt, c: ALUConfig) => {\n",
    "    val fSel = Wire(new ALUFSel(c)) \n",
    "    when (state === 1.U || state === 4.U || state === 7.U) {\n",
    "        fSel.idnEnable.get := true.B\n",
    "        fSel.accEnable.get := false.B\n",
    "    } .elsewhen (state > 1.U && state < 10.U) {\n",
    "        fSel.idnEnable.get := false.B\n",
    "        fSel.accEnable.get := true.B\n",
    "    } .otherwise {\n",
    "        fSel.idnEnable.get := false.B\n",
    "        fSel.accEnable.get := false.B\n",
    "    }\n",
    "    fSel\n",
    "}\n",
    "\n",
    "val rsDecodeNLU = (state: UInt, c: NLUConfig) => {\n",
    "    val fSel = Wire(new NLUFSel(c))\n",
    "    fSel.idEnable.get := true.B\n",
    "    fSel\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defined \u001b[32mclass\u001b[39m \u001b[36mPEConfig\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mMemoryControl\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mProcessControl\u001b[39m\r\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mDecoder\u001b[39m"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class PEConfig(\n",
    "        val weightRFConfig: RFConfig,\n",
    "        val actvtnRFConfig: RFConfig,\n",
    "        val scratchRFConfig: RFConfig,\n",
    "        val ipuConfig: IPUConfig,\n",
    "        val aluConfig: ALUConfig,\n",
    "        val nluConfig: NLUConfig,\n",
    "        val smWidth: Int,\n",
    "        val decodeWeightRF: (UInt, RFConfig) => Data,\n",
    "        val decodeActvtnRF: (UInt, RFConfig) => Data,\n",
    "        val decodeScratchRF: (UInt, RFConfig) => Data,\n",
    "        val decodeIPU: (UInt, IPUConfig) => Data,\n",
    "        val decodeALU: (UInt, ALUConfig) => Data,\n",
    "        val decodeNLU: (UInt, NLUConfig) => Data) {\n",
    "\n",
    "    require(ipuConfig.width == weightRFConfig.numIntOutputs, \n",
    "        \"IPU input width not equal to Weight RF Internal Output width.\\n\")\n",
    "    require(ipuConfig.width == actvtnRFConfig.numIntOutputs,\n",
    "        \"IPU input width not equal to Activation RF Internal Output width.\\n\")\n",
    "    \n",
    "    if(ipuConfig.bpFirm) {\n",
    "        require(aluConfig.addSupp || aluConfig.maxSupp,\n",
    "            \"Incompatible ALU and IPU Configurations\")\n",
    "    }\n",
    "}\n",
    "\n",
    "class MemoryControl(c: PEConfig) extends Bundle {\n",
    "    \n",
    "    override def cloneType = (new MemoryControl(c)).asInstanceOf[this.type]\n",
    "    \n",
    "    val weightRF = new RFControl(c.weightRFConfig)\n",
    "    val actvtnRF = new RFControl(c.actvtnRFConfig)\n",
    "    val scratchRF = new RFControl(c.scratchRFConfig)\n",
    "}\n",
    "\n",
    "class ProcessControl(c: PEConfig) extends Bundle {\n",
    "    \n",
    "    override def cloneType = (new ProcessControl(c)).asInstanceOf[this.type]\n",
    "    \n",
    "    val aluFSel = Output(new ALUFSel(c.aluConfig))\n",
    "    val nluFSel = Output(new NLUFSel(c.nluConfig))\n",
    "    \n",
    "    val ipuBpSel = if (c.ipuConfig.bpFirm) Some(Output(Vec(c.ipuConfig.width, Bool()))) else None\n",
    "}\n",
    "\n",
    "class Decoder(c: PEConfig) extends Module {\n",
    "    \n",
    "    val io = IO(new Bundle {\n",
    "        val state = Input(UInt(c.smWidth.W))\n",
    "        val mem = Output(new MemoryControl(c))\n",
    "        val proc = Output(new ProcessControl(c))\n",
    "    })\n",
    "    \n",
    "    io.mem.weightRF <> c.decodeWeightRF(io.state, c.weightRFConfig)\n",
    "    io.mem.actvtnRF <> c.decodeActvtnRF(io.state, c.actvtnRFConfig)\n",
    "    io.mem.scratchRF <> c.decodeScratchRF(io.state, c.scratchRFConfig)\n",
    "    \n",
    "    if (c.ipuConfig.bpFirm) { \n",
    "        io.proc.ipuBpSel.get := c.decodeIPU(io.state, c.ipuConfig)\n",
    "    }\n",
    "    \n",
    "    io.proc.aluFSel <> c.decodeALU(io.state, c.aluConfig)\n",
    "    io.proc.nluFSel <> c.decodeNLU(io.state, c.nluConfig)\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defined \u001b[32mclass\u001b[39m \u001b[36mPE\u001b[39m"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class PE(c: PEConfig) extends Module {\n",
    "    \n",
    "    val cw = c.weightRFConfig\n",
    "    val ca = c.actvtnRFConfig\n",
    "    val cs = c.scratchRFConfig\n",
    "    \n",
    "    val io = IO(new Bundle {\n",
    "        val ctrl = Input(UInt(c.smWidth.W))\n",
    "        val toWeightRF = Input(Vec(cw.numExtInputs, SInt(cw.dataWidth.W))) \n",
    "        val toActvtnRF = Input(Vec(ca.numExtInputs, SInt(ca.dataWidth.W)))\n",
    "        val toScratchRF = Input(Vec(cs.numExtInputs, SInt(cs.dataWidth.W)))\n",
    "        val fromWeightRF = Output(Vec(cw.numExtOutputs, SInt(cw.dataWidth.W)))\n",
    "        val fromActvtnRF = Output(Vec(ca.numExtOutputs, SInt(ca.dataWidth.W)))\n",
    "        val fromScratchRF = Output(Vec(cs.numExtOutputs, SInt(cs.dataWidth.W)))\n",
    "        val totalOutput = Output(SInt(c.nluConfig.outBitWidth.W))\n",
    "    })\n",
    "    \n",
    "    val decoder = Module(new Decoder(c))\n",
    "    decoder.io.state := io.ctrl\n",
    "    \n",
    "    val weightRF = Module(new RF(cw))\n",
    "    weightRF.io.control <> decoder.io.mem.weightRF\n",
    "    weightRF.io.wExternal := io.toWeightRF\n",
    "    io.fromWeightRF := weightRF.io.rExternal\n",
    "    \n",
    "    val actvtnRF = Module(new RF(ca))\n",
    "    actvtnRF.io.control <> decoder.io.mem.actvtnRF\n",
    "    actvtnRF.io.wExternal := io.toActvtnRF\n",
    "    io.fromActvtnRF := actvtnRF.io.rExternal\n",
    "       \n",
    "    val ipu = Module(new IPU(c.ipuConfig))\n",
    "    if (ipu.io.bpSel.isDefined) { \n",
    "        ipu.io.bpSel.get := decoder.io.proc.ipuBpSel.get \n",
    "    }\n",
    "    ipu.io.weightIn := weightRF.io.rInternal\n",
    "    ipu.io.actvtnIn := actvtnRF.io.rInternal\n",
    "\n",
    "    val alu = Module(new ALU(c.aluConfig))\n",
    "    alu.io.fSel <> decoder.io.proc.aluFSel\n",
    "    alu.io.ipu <> ipu.io.out\n",
    "    \n",
    "    val scratchRF = Module(new RF(cs))\n",
    "    scratchRF.io.control <> decoder.io.mem.scratchRF\n",
    "    scratchRF.io.wExternal := io.toScratchRF\n",
    "    scratchRF.io.wInternal(0) := alu.io.out // TODO: Add Req. for this\n",
    "    io.fromScratchRF := scratchRF.io.rExternal\n",
    "    if(alu.io.rf.isDefined) { \n",
    "        alu.io.rf.get := scratchRF.io.rInternal(0) // TODO: Add Req. for this\n",
    "    }\n",
    "    \n",
    "    val nlu = Module(new NLU(c.nluConfig))\n",
    "    nlu.io.fSel <> decoder.io.proc.nluFSel\n",
    "    nlu.io.in := scratchRF.io.rInternal(0) // TODO: Add Req. for this\n",
    "    \n",
    "    io.totalOutput := nlu.io.out\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Future Plans\n",
    "\n",
    "Basing the nPE off bypasses (i.e. subtractive modifications to a preconfigured topology) is limiting. A constructive approach would be much more general. This should be accomplished through a DSL and much more fully utilizing the functional programming capabilities of Scala."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Scala",
   "language": "scala",
   "name": "scala"
  },
  "language_info": {
   "codemirror_mode": "text/x-scala",
   "file_extension": ".scala",
   "mimetype": "text/x-scala",
   "name": "scala211",
   "nbconvert_exporter": "script",
   "pygments_lexer": "scala",
   "version": "2.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}