Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job Spec #8

Draft
wants to merge 42 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
e965fed
Let's get this party started
expede Nov 8, 2022
2ea97a6
Typo
expede Nov 8, 2022
a2b337f
WIP -- mocking out the format
expede Nov 8, 2022
0eed3f8
Iterating on lots of small details, trying to cover Bacalhau
expede Nov 8, 2022
238711a
Trying another approach for pipelining
expede Nov 8, 2022
1eb8ab6
Flattening the structure, did some more prototyping, started on text
expede Nov 15, 2022
ec8b020
Slowly discovering the shape of the spec
expede Nov 15, 2022
4c70aab
Simplifying!
expede Nov 15, 2022
2dbdb28
Formalizing
expede Nov 19, 2022
bd5bde2
Start skwtching out WarpForge
expede Nov 19, 2022
35d1bb2
Fleshing out effects and intro sections
expede Nov 20, 2022
933dfc4
Saving WIP
expede Nov 21, 2022
024e952
Flesh out pipeling, rename several fields (e.g. run)
expede Nov 22, 2022
ab7961a
Start to IPLDify
expede Nov 22, 2022
ab95752
job -> workflow, break out task spec
expede Nov 24, 2022
27e75c1
Task spec
expede Nov 24, 2022
ba54b40
Spec collectio table
expede Nov 24, 2022
7d7073f
Add subspecs / roadmap
expede Nov 24, 2022
84c71a4
Mocking up a (hopefully) better Docker
expede Nov 25, 2022
31b1142
CHa!
expede Nov 25, 2022
5acd816
Start on invocations (stage after job request)
expede Nov 26, 2022
be735ba
Forgot to push last week
expede Nov 27, 2022
089ef46
Explicitely link out to UCAN invcoations/actions
expede Nov 28, 2022
2436eee
Envelope type
expede Nov 28, 2022
5e67ddd
Start tightening up and splitting out stuff that's not diretly about …
expede Nov 28, 2022
1cc8b18
Simplify down Task, break out effect, rely more heavily on UCAN Invoc…
expede Nov 28, 2022
c85da36
Laying out new spec
expede Nov 28, 2022
14e23e2
Remove the "provisionally"
expede Nov 28, 2022
2922260
Units, early cleanup
expede Nov 30, 2022
b0aa0d3
Tightening up the types; nesting instead of subtyppin the config
expede Nov 30, 2022
9123050
Tightening up the wrapper format
expede Nov 30, 2022
cc4d1df
More tighteninhg up of just the workflwos
expede Nov 30, 2022
0bfafca
Change based on clarifications about IPLD Schema from rvagg
expede Nov 30, 2022
e1578b4
Add links
expede Nov 30, 2022
4b93f49
Really really getting quite close! Getting late... must sleep
expede Nov 30, 2022
ef904a1
Catch!
expede Nov 30, 2022
a3de19d
Many thank-yous
expede Dec 1, 2022
167250d
Fix many but not all spelling issues
expede Dec 1, 2022
e6ec07a
Rewire on catch
expede Dec 1, 2022
b40ef5d
Eliminate infinite loop
expede Dec 1, 2022
22a827f
Wrapping up cleanup
expede Dec 1, 2022
9572701
Start cutting cruft
expede Dec 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Wrapping up cleanup
  • Loading branch information
expede committed Dec 1, 2022
commit 22a827f710fd33565dab7e57e440980dfcdda52a
63 changes: 29 additions & 34 deletions workflow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "S

# 0 Abstract

An IPVM Workflow is a declarative cofiguration. A Workflow provides everything required to execute one or more tasks: defaults, tasks and their dependencies, authorization, metadata, signatures, and so on.
An IPVM Workflow is a declarative cofiguration that extends a [UCAN Invocation](https://github.com/ucan-wg/invocation). A Workflow provides everything required to execute one or more tasks: defaults, tasks and their dependencies, authorization, metadata, signatures, and so on.

# 1 Introduction

Expand All @@ -48,15 +48,15 @@ The outer wrapper of a workflow MUST contain the following fields:
| `ipvm/workflow` | `Workflow` | IPVM Workflow | Yes |
| `signature` | `VarSig` | [VarSig](https://github.com/ChainAgnostic/varsig/) of serialized fields | Yes |

| Field | Type | Description | Required | Default |
|------------|---------------------|-------------------------------------------------------------------------------------|----------|---------|
| `v` | `"0.1.0"` | IPVM workflow version | Yes | |
| `meta` | `{String : Any}` | User-defined object (tags, comments, etc) | No | `{}` |
| `parent` | `&Workflow or Null` | The CID of the initiating workflow (if any) FIXME probably want the task & workflow | No | `Null` |
| `config` | `Config` | | No | `{}` |
| `defaults` | `Config` | | No | `{}` |
| `tasks` | `UCAN.Invocation` | UCAN Invocation | Yes | |
| `on` | `Listeners` | IPVM event listeners | No | `{}` |
| Field | Type | Description | Required | Default |
|------------|------------------------------|------------------------------------------------------------------------|----------|---------|
| `v` | `"0.1.0"` | IPVM workflow version | Yes | |
| `meta` | `{String : Any}` | User-defined object (tags, comments, etc) | No | `{}` |
| `parent` | `[&Workflow, Label] or Null` | The workflow & task label that initiated the current workflow (if any) | No | `Null` |
| `config` | `Config` | Global configuration (e.g. timeout for the entire workflow) | No | `{}` |
| `defaults` | `Config` | Individual task config defaults | No | `{}` |
| `tasks` | `UCAN.Invocation` | UCAN Invocation | Yes | |
| `catch` | `&WasmTask` | Deterministic Wasm that fires on exceptions | No | `{}` |

## 2.1 Fields

Expand All @@ -74,7 +74,7 @@ The OPTIONAL `parent` field contains the CID of the IPVM Task that initiated it

## 2.1.4 Config

The OPTIONAL global `config` object (FIXME section X.Y) sets the configuration for the workflow itself, and defaults for tasks.
The OPTIONAL global [`config` object](#3-configuration) sets the configuration for the workflow itself, and defaults for tasks.

## 2.1.5 Defaults

Expand All @@ -98,13 +98,18 @@ type SignedWorkflow struct {

type Workflow struct {
v SemVer
meta {String : Any} (implicit {})
parent nullable &Task (implicit Null)
global Config (implicit {})
defauts Config (implicit {})
meta {String : Any} (implicit {})
parent nullable TaskRef (implicit Null)
global Config (implicit {})
defauts Config (implicit {})
tasks UCAN.Invocation
catch nullable &Wasm (implicit Null)
catch nullable &Wasm (implicit Null)
}

type TaskRef struct {
inv &Invocation
task String -- Label for the task
} representation tuple
```

## 2.3 JSON Exmaples
Expand All @@ -127,7 +132,7 @@ type Workflow struct {
"tasks": "ucan/invoke": {
"v": "0.1.0",
"nnc": "02468",
"prf": [ // FIXME having to resend this is a pain!
"prf": [
{"/": "bafkreie2cyfsaqv5jjy2gadr7mmupmearkvcg7llybfdd7b6fvzzmhazuy"},
{"/": "bafkreibbz5pksvfjyima4x4mduqpmvql2l4gh5afaj4ktmw6rwompxynx4"}
],
Expand Down Expand Up @@ -247,8 +252,6 @@ Tasks are the smallest level of work granularity a workflow. Tasks describe ever

Tasks MAY be configured in aggragate in the [global defaults](#215-defaults). Individual Task configuration MUST be embedded inside of a [UCAN Action](https://github.com/ucan-wg/invocation)'s `meta['ipvm/confg']` field.

Note that while all Tasks have a resource (URI) and action, the details MAY be quite different. Each Task is restricted to a specific [safety level](FIXME) based on its resource/action pair (such as a deterministic Wasm module or [effects](FIXME) like an HTTP `GET` request). Tasks MUST be scheduled according to its safety properties, which MAY have a performance impact.

## 4.1 Fields

Recall UCAN Invocation Tasks:
Expand Down Expand Up @@ -285,14 +288,6 @@ An OPTIONAL IPVM `Config` MAY be included at the `meta['ipvm/config']` path. The

``` js
{
"supply-gas": {
"with": "gas:reserve://mine", // Or something... needs work at least FIXME
"do": "gas/supply",
"inputs": {
"on": ["/", "some-wasm"],
"max": 1000
}
},
"some-wasm": {
"with": "wasm:1:Qm12345", // Or something... wasm:Qm12345?
"do": "ipvm/run",
Expand Down Expand Up @@ -410,14 +405,14 @@ type SubPrefix enum {
}

type SuperPrefix enum {
| Deca "da"
| Deca "da"
| Hecto "ha"
| Kilo "k"
| Mega "M"
| Giga "G"
| Tera "T"
| Peta "P"
| Exa "E"
| Kilo "k"
| Mega "M"
| Giga "G"
| Tera "T"
| Peta "P"
| Exa "E"
}

type SIPrefix union {
Expand Down