docs: Update compilation docs (#119)

* Update compilation docs

* Describe preprocessing in the compilation doc

* Remove precompilation docs as it no longer applies

* Update index.md

* Update index.md

* Remove remarks about passthrough in troubleshooting

README.md

@@ -30,18 +30,18 @@ In order to quickly get started with `bartiq`, you can load Alias Sampling as an
 
 ```python
 import json
-from bartiq import Routine, compile_routine, evaluate
-from bartiq.integrations import qref_to_bartiq
+from bartiq import compile_routine, evaluate
+from qref import SchemaV1
 
-with open("alias_sampling_basic.json", "r") as f:
+with open("docs/data/alias_sampling_basic.json", "r") as f:
     routine_dict = json.load(f)
 
-uncompiled_routine = qref_to_bartiq(routine_dict)
-compiled_routine = compile_routine(uncompiled_routine)
+uncompiled_routine = SchemaV1(**routine_dict)
+compiled_routine = compile_routine(uncompiled_routine).routine
 
-assignments = ["L=100", "mu=10"]
+assignments = {"L": 100, "mu": 10}
 
-evaluated_routine = evaluate(compiled_routine, assignments)
+evaluated_routine = evaluate(compiled_routine, assignments).routine
 ```
 
 Now in order to inspect the results you can do:

docs/concepts/compilation.md

@@ -5,57 +5,117 @@ Please keep in mind, that while we try to keep this page up-to-date, in the end
 
 ## Bird's eye perspective on compilation
 
-Compilation consists of the following steps:
+Compilation is a very overloaded word, so let us start by explaining what we mean by *compiling a routine* in Bartiq.
 
-1. Precompilation - ensures that Routine has all the components needed for the rest of the compilation process.
-2. Routine-to-function conversion – maps routine to functions (see glossary), on which compilation is performed.
-3. Function compilation – compiles all the functions to a 
+Quantum programs are often deeply nested structures. A program can call multiple routines, which themselves call other routines, and so on. One typically defines resources of each routine at the local level using available local symbols and variables. However, in the end, we are typically interested in resources used by each routine expressed in terms of input parameters of the whole program.
 
+Here's an example to illustrate what we're talking about. Below we show a routine with
+two children, each of them having some example resource defined in terms of their local parameters.
 
-## Step 1 – precompilation
 
-This stage "precompiles" the routine by making a series of reasonable assumptions about what the user would have
-wanted when they have not been explicit. For example, insertion of "trivial" routines for known subroutines or
-"filling out" routines which have only been partially defined. It is expected that this precompilation stage will
-grow as more sensible defaults are requested by users. For more details please visit [precompilation page](precompilation.md).
+```yaml
+--8<-- "docs/data/compilation_example.yaml"
+```
 
-## Step 2 – routine to function conversion
+![example routine](../images/compilation_example.png)
 
-Before being compiled, routines must first be mapped to compilable function objects (see glossary).
-At this stage, we perform a number of steps to map the routines into functions that are ready for
-compilation:
 
-1. Map routines to "local" functions: first, we simply map the routines to functions in a local manner, whereby they
-   are unaware of their location within the full definition. To do this we convert `Routine` objects to `RoutineWithFunction`, which augments Routine class with a `symbolic_function` field.
-2. Map "local" functions to "global" functions: we tell the functions where they live in the definition, thereby
-   making each parameter and any named functions unique within the definition by prepending it with its full path from the root.
-3. Pull in input register sizes: in order to allow for input register sizes to be compiled properly, low-level input
-   register sizes must be renamed by the size parameters used by high-level registers. This process is referred to as
-   "pulling in" those high-level register sizes.
-4. Push out output register sizes: to ensure correct compilation for output register sizes, those associated with
-   low-level routines must be "push out" to either other low-level routines or eventually high-level routines.
-5. Parameter inheritance: lastly, any high-level routines parameters are passed down to low-level routines by
-   substituting the low-level routine parameters with the high-level ones.
+As we can see, both `a` and `b` have a resource `x` defined relatively to their input port size `L`. However, when
+looked globally, those resources would have a different value. Indeed, let's see how the port sizes propagate.
 
-Once this is complete, each subroutine has an associated function, which is expressed in terms of global variables.
+1. The input port of the top-level `root` routine of size `N` is connected to the input port of routine `a`. Hence,
+   when viewed through global scope, the input port of `a` has size `N`.
+2. The output port of routine `a` has, by definition, size twice as large as its input port. It is connected to input port
+   of routine `b`, and therefore `b.in_0` has size `2N`.
 
-## Step 3 – functions compilation
+Looking at the resources of `a` and `b` we see that `x` is defined as square of their respective input port sizes. Thus,
+`a.x` has value of `N ** 2`, and `b.x` has value `(2 * N) ** 2 = 4 * N ** 2`.
 
-At the start of this stage routine is annotated with functions which contain all the required information in the global namespace. Compilation starts from the leafs, each routine can only be compiled once all of its children have been compiled ("bottom-up"). This stage contains a number of sub-steps:
+This is what the compilation process is all about. Given a routine with all its components defined in terms of local symbols
+and parameters, Bartiq produces a *compiled routine* in which all port sizes and resources are defined in terms
+of the global input symbols of top-level routine.
 
-1. Compile functions of self and children (non-leaves only): for leaves, no function compilation is necessary, but
-   non-leaves need to update information in associated function with the information contained in their children's functions.
-2. Undo register size pulling and pushing: to ensure a correct local estimate, we must "undo" the pulling in and pushing
-   out of register size params done during routine-to-function conversion.
-3. Derefence global namespace: again, to ensure a correct local routine definition, we dereference the global
-   namespace. This means dropping the path prefix from all the variables: e.g.: `root.child.gchild.N` would become `N`.
-4. Map function to routine: lastly, once the correct local function has been compiled, we map the function object
-   back to a routine by updating routine's attribute and removing associated function object.
 
+## Compilation in details
 
-## Glossary
+Compilation can be viewed as recursive process. At every recursive call, several things need to happen in correct order.
+Below we outline how the compilation proceeds.
 
-- **Function**: functions are objects defined with `SymbolicFunction` class. Each function has a defined set of inputs and outputs. Inputs are independent variables, whic are basically bare symbols, without any structure. Outputs are dependent variables, meaning they are variables defined by expressions which are dependent on the input variables.
-- **Global and local**: in this context "global" means "defined at the root-level of the routine we compile". Local, on the contrary, means "defined for the subroutine currently being operated on". Intuitively, global parameters are those that correspond to the domain of the problem that we are trying to solve, e.g.: number of bits of precision of QPE. On the other hand, local parameters correspond to the implementation details of the algorithm we are analyzing, e.g.: size of the QFT routine in QPE. Compilation process allows us to express all the local parameters in the algorithm in terms of the global parameters.
+### Step 1: preprocessing
 
+The Bartiq's compilation engine makes several assumptions about the routine being compiled, which simplify its code
+at the expense of flexibility. For instance, Bartiq assumes all port sizes are single parameters of size `#port_name`.
+Another, very important assumption, is that there are no parameter links reaching deeper than one level of nesting.
 
+Writing a routine conforming to those requirements by hand is possible, but tedious. Instead, Bartiq allows for
+violation of some of its assumptions, and preprocesses the routine so that all the requirements are met.
+
+As an example, suppose some port `in_0` has size `1`. Bartiq replaces this size with `#in_0`, and then adds a
+constraint saying that `#in_0 = 1`.
+
+
+Currently, there are following preprocessing stages:
+
+1. Introduction of default additive resources. This stage allows users to define the additive resources for leafs,
+   and then adds the same resource to each of the higher-level routines, by defining it as a sum of the resource over all children that define it.
+   In the example we discussed previously, this preprocessing step would allow uas to skip the definition of
+   `x` resoruce in `root` and instead have it automatically defined by Bartiq.
+2. Propagation of linked params. In this stage all linked parameters reaching further than to the direct
+   descendant are converted into a series of direct parameter links. This is useful, because you can e.g.
+   link parameter from the top-level routine to a parameter arbitrarily deep in the program structure,
+   and it will still work, despite Bartiq's compilation engine requirement on having only direct links.
+3. Promotion of ulinked inputs. Compilation cannot handle parameters that are not linked or pased through
+   connections. To avoid unnecessary compilation errors, Bartiq will promote such parameter to a parameter
+   linked to newly introduced parent's input.
+4. Introduction of port variables. As discussed above, this step converts all ports so that they have sizes
+   equal to a single-parameter expression of known name, while also introducing constraints to make sure
+   that no information is lost in the process.
+
+It is hopefully clear by now that preprocessing allows user for writing more concise and readable routines, while at the same time
+ensuring that strict requirements of compilation engine are met.
+
+### Step 2: Recursive compilation of routines
+
+As already mentioned, the compilation process is recursive. While Bartiq processes any given routine, it
+maintains a parameter map for all the routine's children. This map gets populated whenever new piece of
+information is obtained, and then passed to the recursive call when each child is being compiled.
+
+#### Step 2.1: Compilation of constraints
+
+At the very beginning of the compilation, Bartiq evaluates the constraints introduced in preprocessing. If any
+of the constraints is violated, a `BartiqCompiilationError` is raised. Constraints that are trivially satisfied
+are dropped from the system, and other constraints are retained.
+
+#### Step 2.2: Compilation of local variables
+
+As the next step, local variables of the routine are expressed in terms of variables passed from its parent (this
+step does nothing for the root routine). As the result, all local variables are expressed in terms of global
+parameters.
+
+#### Step 2.3: Compilation of non-output ports
+
+Input and through port sizes of any routine can only depend on its inputs and local variables. Thus, those objects
+are perfect to be compiled next.
+
+After each port is compiled, Bartiq checks if this port is connected to other port in the routine. If so, a
+corresponding entry in parameter tree is added. For instance, suppose that port `in_0` got compiled and has now
+size `N`. If this port is connected to port `in_1` of child `a`, a parameter map for `a` will contain entry
+mapping `#in_1` to `N`.
+
+
+#### Step 2.4: Recursive compilation of children
+
+The children are traversed in topological order, which ensures all required entries in the parameter maps are
+populated before other children are compiled.
+
+Once this step is completed, we can be sure that all resources and ports of each child are expressed in terms
+of global variables, which is a requirement for the next step.
+
+#### Step 2.5: Resource compilation
+
+Resources of the routine are compiled, which is possible only at this stage, as they can be expressed in terms
+of resources of its children.
+
+#### Step 2.6: Output port compilation
+
+Finally, the output ports are compiled, and the new object representing compiled routine is created.

docs/data/compilation_example.yaml

@@ -0,0 +1,27 @@
+version: v1
+program:
+  name: root
+  input_params:
+  - N
+  ports:
+  - {"name": "in_0", "size": "N", "direction": "input"}
+  - {"name": "out_0", "size": null, "direction": "output"}
+  resources:
+  - {"name": "x", "value": "a.x + b.x", "type": "additive"}
+  children:
+    - name: "a"
+      ports:
+      - {"name": "in_0", "size": "L", "direction": "input"}
+      - {"name": "out_0", "size": "2 * L", "direction": "output"}
+      resources:
+      - {"name": "x", "value": "L**2", "type": "additive"}
+    - name: "b"
+      ports:
+      - {"name": "in_0", "size": "L", "direction": "input"}
+      - {"name": "out_0", "size": "2 * L", "direction": "output"}
+      resources:
+      - {"name": "x", "value": "L ** 2", "type": "additive"}
+  connections:
+    - in_0 -> a.in_0
+    - a.out_0 -> b.in_0
+    - b.out_0 -> out_0  

docs/index.md

@@ -21,18 +21,18 @@ In order to quickly get started with `bartiq`, you can load Alias Sampling as an
 
 ```python
 import json
-from bartiq import Routine, compile_routine, evaluate
-from bartiq.integrations import qref_to_bartiq
+from bartiq import compile_routine, evaluate
+from qref import SchemaV1
 
 with open("alias_sampling_basic.json", "r") as f:
     routine_dict = json.load(f)
 
-uncompiled_routine = qref_to_bartiq(routine_dict)
-compiled_routine = compile_routine(uncompiled_routine)
+uncompiled_routine = SchemaV1(**routine_dict)
+compiled_routine = compile_routine(uncompiled_routine).routine
 
-assignments = ["L=100", "mu=10"]
+assignments = {"L": 100, "mu": 10}
 
-evaluated_routine = evaluate(compiled_routine, assignments)
+evaluated_routine = evaluate(compiled_routine, assignments).routine gs
 ```
 
 Now in order to inspect the results you can do:

docs/troubleshooting.md

@@ -10,5 +10,3 @@ Debugging `bartiq` is not always straightforward, so please see below for a numb
 - Take a look at [the list of issues on GitHub](https://github.com/PsiQ/bartiq/issues) and see if other didn't have a similar problem!
 	- If not, consider creating one!
 	- Submitting issue is the most transparent way to give us feedback – even if something works, but is extremely unintuitive, we want to make it easier to use. The goal of this tool is to save you time, not waste it on unhelpful error messages.
-
-- If you see a `passthrough` in your error message, but you don't know where it came from, passthroughs are added automatically during one of the precompilation stages (see [precompilation page](concepts/precompilation.md) for more details).