Added website docs and mdoc.

.circleci/config.yml

@@ -139,6 +139,16 @@ jobs:
             - .m2
             - .sbt
 
+  test-chisel-docs:
+    executor: chisel-executor
+    steps:
+      - attach_workspace:
+          at: /home/chisel
+
+      - run:
+          command: |
+            sbt $SBT_ARGS "docs/mdoc"
+
   test-chisel-2_11:
     executor: chisel-executor
     steps:

.gitignore

@@ -6,8 +6,12 @@ generated/
 .idea_modules/
 .project
 target/
+docs-target/
 *.iml
 *.swp
+*.fir
+*.v
+*.json
 test_run_dir
 *~
 \#*\#

README.md

@@ -1,4 +1,4 @@
-![Chisel 3](https://raw.githubusercontent.com/freechipsproject/chisel3/master/doc/images/chisel_logo.svg?sanitize=true)
+![Chisel 3](https://raw.githubusercontent.com/freechipsproject/chisel3/master/docs/src/images/chisel_logo.svg?sanitize=true)
 
 ---
 
@@ -26,7 +26,7 @@ Chisel is powered by [FIRRTL (Flexible Intermediate Representation for RTL)](htt
 
 Consider an FIR filter that implements a convolution operation, as depicted in this block diagram:
 
-<img src="https://raw.githubusercontent.com/freechipsproject/chisel3/master/doc/images/fir_filter.svg?sanitize=true" width="512" />
+<img src="https://raw.githubusercontent.com/freechipsproject/chisel3/master/docs/src/images/fir_filter.svg?sanitize=true" width="512" />
 
 While Chisel provides similar base primitives as synthesizable Verilog, and *could* be used as such:
 

build.sbt

@@ -239,6 +239,21 @@ lazy val chisel = (project in file(".")).
     )
   )
 
+lazy val docs = project       // new documentation project
+  .in(file("docs-target")) // important: it must not be docs/
+  .dependsOn(chisel)
+  .enablePlugins(MdocPlugin)
+  .settings(commonSettings)
+  .settings(
+    scalacOptions += "-language:reflectiveCalls",
+    mdocIn := file("docs/src"),
+    mdocOut := file("docs/generated"),
+    mdocExtraArguments := Seq("--cwd", "docs"),
+    mdocVariables := Map(
+      "BUILD_DIR" -> "docs-target" // build dir for mdoc programs to dump temp files
+    )
+  )
+
 addCommandAlias("com", "all compile")
 addCommandAlias("lint", "; compile:scalafix --check ; test:scalafix --check")
 addCommandAlias("fix", "all compile:scalafix test:scalafix")

docs/README.md

@@ -0,0 +1,33 @@
+# chisel3/docs/README.md
+
+This directory contains documentation on the code within this repository.
+Documents can either be written directly in markdown, or
+use embedded [mdoc](https://scalameta.org/mdoc/)
+which compiles against the `chisel3` (and dependencies) codebase
+as part of the PR CI checks,
+forcing the documentation to remain current with the codebase.
+The `src` folder contains the source from which these are generated.
+
+Previous Wiki documentation, now hosted by the website, is contained in the `src/wiki-deprecated` directory.
+We are in the process of converting this documentation into the four categories as described in
+[Divio's documentation system](https://documentation.divio.com/).
+
+The four documentation types are:
+ 1. Reference (source code scaladoc)
+ 1. Explanation (`src/explanations`)
+ 1. How-To Guides (`src/cookbooks`)
+ 1. Tutorials (currently not located here)
+
+Our documentation strategy for this repository is as follows:
+ * Any new public API requires reference documentation.
+ * Any new user-facing feature requires explanation documentation.
+ * Any bugfixes, corner-cases, or answers to commonly asked questions requires a how-to guide.
+ * For now, tutorials are kept in a separate repository. We are working hosting them here.
+ * Old documentation is contained in the `src/wiki-deprecated` directory and is being incrementally converted to these
+ categories.
+
+To build the documentation, run `docs/mdoc` from SBT in the root directory. The generated documents
+will appear in the `docs/generated` folder. To iterate on the documentation, you can run `docs/mdoc --watch`. For
+more `mdoc` instructions you can visit their [website](https://scalameta.org/mdoc/).
+
+This documentation is hosted on the Chisel [website](https://www.chisel-lang.org).

docs/src/wiki-deprecated/annotations.md

@@ -0,0 +1,141 @@
+---
+layout: docs
+title:  "Annotations"
+section: "chisel3"
+---
+
+`Annotation`s are metadata containers associated with zero or more "things" in a FIRRTL circuit.
+Commonly, `Annotation`s are used to communicate information from Chisel to a specific, custom FIRRTL `Transform`.
+In this way `Annotation`s can be viewed as the "arguments" that a specific `Transform` consumes.
+
+This article focuses on the approach to building a basic library that contains `Annotation`s and `Transform`s.
+
+### Imports
+We need a few basic imports to reference the components we need.
+
+```scala mdoc:silent
+import chisel3._
+import chisel3.experimental.{annotate, ChiselAnnotation, RunFirrtlTransform}
+import chisel3.internal.InstanceId
+
+import firrtl._
+import firrtl.annotations.{Annotation, SingleTargetAnnotation}
+import firrtl.annotations.{CircuitName, ComponentName, ModuleName, Named}
+```
+
+### Define an `Annotation` and a `Transform`
+
+First, define an `Annotation` that contains a string associated with a `Named` thing in the Chisel circuit.
+This `InfoAnnotation` extends [`SingleTargetAnnotation`](https://www.chisel-lang.org/api/firrtl/1.2.0/firrtl/annotations/SingleTargetAnnotation.html), an `Annotation` associated with *one* thing in a FIRRTL circuit:
+
+```scala mdoc:silent
+/** An annotation that contains some string information */
+case class InfoAnnotation(target: Named, info: String) extends SingleTargetAnnotation[Named] {
+  def duplicate(newTarget: Named) = this.copy(target = newTarget)
+}
+```
+
+> Note: `Named` is currently deprecated in favor of the more specific `Target`.
+> Currently, `Named` is still the advised approach for writing `Annotation`s.
+
+Second, define a `Transform` that consumes this `InfoAnnotation`.
+This `InfoTransform` simply reads all annotations, prints any `InfoAnnotation`s it finds, and removes them.
+
+```scala mdoc:invisible
+object Issue1228 {
+  /* Workaround for https://github.com/freechipsproject/firrtl/pull/1228 */
+  abstract class Transform extends firrtl.Transform {
+    override def name: String = this.getClass.getName
+  }
+}
+import Issue1228.Transform
+```
+
+```scala mdoc:silent
+/** A transform that reads InfoAnnotations and prints information about them */
+class InfoTransform() extends Transform with DependencyAPIMigration {
+
+  override def prerequisites = firrtl.stage.Forms.HighForm
+
+  override def execute(state: CircuitState): CircuitState = {
+    println("Starting transform 'IdentityTransform'")
+
+    val annotationsx = state.annotations.flatMap{
+      case InfoAnnotation(a: CircuitName, info) =>
+        println(s"  - Circuit '${a.serialize}' annotated with '$info'")
+        None
+      case InfoAnnotation(a: ModuleName, info) =>
+        println(s"  - Module '${a.serialize}' annotated with '$info'")
+        None
+      case InfoAnnotation(a: ComponentName, info) =>
+        println(s"  - Component '${a.serialize} annotated with '$info''")
+        None
+      case a =>
+        Some(a)
+    }
+
+    state.copy(annotations = annotationsx)
+  }
+}
+```
+
+> Note: `inputForm` and `outputForm` will be deprecated in favor of a new dependency API that allows transforms to specify their dependencies more specifically than with circuit forms.
+> Full backwards compatibility for `inputForm` and `outputForm` will be maintained, however.
+
+### Create a Chisel API/Annotator
+
+Now, define a Chisel API to annotate Chisel things with this `InfoAnnotation`.
+This is commonly referred to as an "annotator".
+
+Here, define an object, `InfoAnnotator` with a method `info` that generates `InfoAnnotation`s.
+This uses the `chisel3.experimental.annotate` passed an anonymous `ChiselAnnotation` object.
+The need for this `ChiselAnnotation` (which is different from an actual FIRRTL `Annotation`) is that no FIRRTL circuit exists at the time the `info` method is called.
+This is delaying the generation of the `InfoAnnotation` until the full circuit is available.
+
+This annotator also mixes in the `RunFirrtlTransform` trait (abstract in the `transformClass` method) because this annotator, whenever used, should result in the FIRRTL compiler running the custom `InfoTransform`.
+
+```scala mdoc:silent
+object InfoAnnotator {
+  def info(component: InstanceId, info: String): Unit = {
+    annotate(new ChiselAnnotation with RunFirrtlTransform {
+      def toFirrtl: Annotation = InfoAnnotation(component.toNamed, info)
+      def transformClass = classOf[InfoTransform]
+    })
+  }
+}
+```
+
+> Note: there are a number of different approaches to writing an annotator.
+> You could use a trait that you mix into a `Module`, an object (like is done above), or any other software approach.
+> The specific choice of how you implement this is up to you!
+
+### Using the Chisel API
+
+Now, we can use the method `InfoAnnotation.info` to create annotations that associate strings with specific things in a FIRRTL circuit.
+Below is a Chisel `Module`, `ModC`, where both the actual module is annotated as well as an output.
+
+```scala mdoc:silent
+class ModC(widthC: Int) extends Module {
+  val io = IO(new Bundle {
+    val in = Input(UInt(widthC.W))
+    val out = Output(UInt(widthC.W))
+  })
+  io.out := io.in
+
+  InfoAnnotator.info(this, s"ModC($widthC)")
+
+  InfoAnnotator.info(io.out, s"ModC(ignore param)")
+}
+```
+
+### Running the Compilation
+
+Compiling this circuit to Verilog will then result in the `InfoTransform` running and the added `println`s showing information about the components annotated.
+
+```scala mdoc
+import chisel3.stage.{ChiselStage, ChiselGeneratorAnnotation}
+
+// This currently doesn't work because of mdoc limitations. However, it will work
+// in your normal Scala code.
+//(new ChiselStage).execute(Array.empty, Seq(ChiselGeneratorAnnotation(() => new ModC(4))))
+```

docs/src/wiki-deprecated/blackboxes.md

@@ -0,0 +1,131 @@
+---
+layout: docs
+title:  "Blackboxes"
+section: "chisel3"
+---
+Chisel *BlackBoxes* are used to instantiate externally defined modules. This construct is useful
+for hardware constructs that cannot be described in Chisel and for connecting to FPGA or other IP not defined in Chisel.
+
+Modules defined as a `BlackBox` will be instantiated in the generated Verilog, but no code
+will be generated to define the behavior of module.
+
+Unlike Module, `BlackBox` has no implicit clock and reset. 
+`BlackBox`'s clock and reset ports must be explicitly declared and connected to input signals.
+Ports declared in the IO Bundle will be generated with the requested name (ie. no preceding `io_`).
+
+### Parameterization
+
+**This is an experimental feature and is subject to API change**
+
+Verilog parameters can be passed as an argument to the BlackBox constructor.
+
+For example, consider instantiating a Xilinx differential clock buffer (IBUFDS) in a Chisel design:
+
+```scala mdoc:silent
+import chisel3._
+import chisel3.util._
+import chisel3.experimental._ // To enable experimental features
+
+class IBUFDS extends BlackBox(Map("DIFF_TERM" -> "TRUE",
+                                  "IOSTANDARD" -> "DEFAULT")) {
+  val io = IO(new Bundle {
+    val O = Output(Clock())
+    val I = Input(Clock())
+    val IB = Input(Clock())
+  })
+}
+
+class Top extends Module {
+  val io = IO(new Bundle{})
+  val ibufds = Module(new IBUFDS)
+  // connecting one of IBUFDS's input clock ports to Top's clock signal
+  ibufds.io.I := clock
+}
+```
+
+In the Chisel-generated Verilog code, `IBUFDS` will be instantiated as:
+
+```verilog
+IBUFDS #(.DIFF_TERM("TRUE"), .IOSTANDARD("DEFAULT")) ibufds (
+  .IB(ibufds_IB),
+  .I(ibufds_I),
+  .O(ibufds_O)
+);
+```
+
+### Providing Implementations for Blackboxes
+Chisel provides the following ways of delivering the code underlying the blackbox.  Consider the following blackbox that adds two real numbers together.  The numbers are represented in chisel3 as 64-bit unsigned integers.
+```scala mdoc:silent:reset
+import chisel3._
+class BlackBoxRealAdd extends BlackBox {
+  val io = IO(new Bundle() {
+    val in1 = Input(UInt(64.W))
+    val in2 = Input(UInt(64.W))
+    val out = Output(UInt(64.W))
+  })
+}
+```
+
+The implementation is described by the following verilog
+```verilog
+module BlackBoxRealAdd(
+    input  [63:0] in1,
+    input  [63:0] in2,
+    output reg [63:0] out
+);
+  always @* begin
+  out <= $realtobits($bitstoreal(in1) + $bitstoreal(in2));
+  end
+endmodule
+```
+
+### Blackboxes with Verilog in a Resource File
+In order to deliver the verilog snippet above to the backend simulator, chisel3 provides the following tools based on the chisel/firrtl [annotation system](annotations.md).  Add the trait ```HasBlackBoxResource``` to the declaration, and then call a function in the body to say where the system can find the verilog.  The Module now looks like
+```mdoc scala:silent:reset
+class BlackBoxRealAdd extends BlackBox with HasBlackBoxResource {
+  val io = IO(new Bundle() {
+    val in1 = Input(UInt(64.W))
+    val in2 = Input(UInt(64.W))
+    val out = Output(UInt(64.W))
+  })
+  setResource("/real_math.v")
+}
+```
+The verilog snippet above gets put into a resource file names ```real_math.v```.  What is a resource file? It comes from a java convention of keeping files in a project that are automatically included in library distributions. In a typical chisel3 project, see [chisel-template](https://github.com/ucb-bar/chisel-template), this would be a directory in the source hierarchy
+```
+src/main/resources/real_math.v
+```
+
+### Blackboxes with In-line Verilog
+It is also possible to place this verilog directly in the scala source.  Instead of ```HasBlackBoxResource``` use ```HasBlackBoxInline``` and instead of ```setResource``` use ```setInline```.  The code will look like this.
+```scala mdoc:silent:reset
+import chisel3._
+import chisel3.util.HasBlackBoxInline
+class BlackBoxRealAdd extends BlackBox with HasBlackBoxInline {
+  val io = IO(new Bundle() {
+    val in1 = Input(UInt(64.W))
+    val in2 = Input(UInt(64.W))
+    val out = Output(UInt(64.W))
+  })
+  setInline("BlackBoxRealAdd.v",
+    """module BlackBoxRealAdd(
+      |    input  [15:0] in1,
+      |    input  [15:0] in2,
+      |    output [15:0] out
+      |);
+      |always @* begin
+      |  out <= $realtobits($bitstoreal(in1) + $bitstoreal(in2));
+      |end
+      |endmodule
+    """.stripMargin)
+}
+```
+This technique will copy the inline verilog into the target directory under the name ```BlackBoxRealAdd.v```
+
+### Under the Hood
+This mechanism of delivering verilog content to the testing backends is implemented via chisel/firrtl annotations.  The two methods, inline and resource, are two kinds of annotations that are created via the ```setInline``` and ```setResource``` methods calls.  Those annotations are passed through to the chisel-testers which in turn passes them on to firrtl.  The default firrtl verilog compilers have a pass that detects the annotations and moves the files or inline test into the build directory.  For each unique file added, the transform adds a line to a file black_box_verilog_files.f, this file is added to the command line constructed for verilator or vcs to inform them where to look.
+The [dsptools project](https://github.com/ucb-bar/dsptools) is a good example of using this feature to build a real number simulation tester based on black boxes.
+
+### The interpreter
+The [firrtl interpreter](https://github.com/ucb-bar/firrtl-interpreter) uses a separate system that allows users to construct scala implementations of the black boxes.  The scala implementation code built into a BlackBoxFactory which is passed down to the interpreter by the execution harness.  The interpreter is a scala simulation tester.  Once again the dsptools project uses this mechanism and is a good place to look at it.
+> It is planned that the BlackBoxFactory will be replaced by integration with the annotation based blackbox methods stuff soon.