Merge remote-tracking branch 'origin' into ipni

storacha · Jan 18, 2024 · 3bc0977 · 3bc0977
2 parents 2c212e6 + abf0280
commit 3bc0977
Show file tree

Hide file tree

Showing 5 changed files with 310 additions and 28 deletions.
diff --git a/w3-admin.md b/w3-admin.md
@@ -1,6 +1,6 @@
 # Admin Protocol
 
-![status:wip](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square)
+![status:reliable](https://img.shields.io/badge/status-reliable-green.svg?style=flat-square)
 
 ## Editors
 
@@ -10,7 +10,7 @@
 
 - [Travis Vachon](https://github.com/travis), [Protocol Labs](https://protocol.ai/)
 
-# Abstract
+## Abstract
 
 Storage providers in the w3 family of protocols need to be able to get information about the customers, subscriptions and "consumers" (i.e., spaces)
 they work with. The capabilities described in this document all act on the "service" resource (i.e., `did:web:web3.storage`) and can be delegated
@@ -71,6 +71,11 @@ export const get = capability({
 })
 ```
 
+#### Implementations
+
+- @web3-storage/capabilities: [capability in consumer.js](https://github.com/web3-storage/w3up/blob/231cf1f863f4e9a96c92d9ef5001617ba928028d/packages/capabilities/src/consumer.js#L29C1-L29C1)
+- @web3-storage/upload-api: [invocation handler in consumer/get.js](https://github.com/web3-storage/w3up/blob/231cf1f863f4e9a96c92d9ef5001617ba928028d/packages/upload-api/src/consumer/get.js)
+
 ### `customer/get`
 
 Get information about a customer.
@@ -111,6 +116,11 @@ export const get = capability({
 })
 ```
 
+#### Implementations
+
+- @web3-storage/capabilities: [capability in customer.js](https://github.com/web3-storage/w3up/blob/231cf1f863f4e9a96c92d9ef5001617ba928028d/packages/capabilities/src/customer.js#L12)
+- @web3-storage/upload-api: [invocation handler in customer/get.js](https://github.com/web3-storage/w3up/blob/231cf1f863f4e9a96c92d9ef5001617ba928028d/packages/upload-api/src/customer/get.js)
+
 ### `subscription/get`
 
 Get information about a subscription.
@@ -151,6 +161,11 @@ export const get = capability({
 })
 ```
 
+#### Implementations
+
+- @web3-storage/capabilities: [capability in subscription.js](https://github.com/web3-storage/w3up/blob/main/packages/capabilities/src/subscription.js)
+- @web3-storage/upload-api: [invocation handler in subscription/get.js](https://github.com/web3-storage/w3up/blob/231cf1f863f4e9a96c92d9ef5001617ba928028d/packages/upload-api/src/subscription/get.js)
+
 ### `admin/upload/inspect`
 
 Get information about a content CID. This does not return the actual data identified by the CID, just metadata our
@@ -171,6 +186,11 @@ with the date it was uploaded.
 }
 ```
 
+#### Implementations
+
+- @web3-storage/capabilities: [capability in admin.js](https://github.com/web3-storage/w3up/blob/main/packages/capabilities/src/admin.js)
+- @web3-storage/upload-api: [invocation handler in admin/upload/inspect.js](https://github.com/web3-storage/w3up/blob/231cf1f863f4e9a96c92d9ef5001617ba928028d/packages/upload-api/src/admin/upload/inspect.js)
+
 ### `admin/store/inspect`
 
 Get information about a shard (i.e., a CAR that contains part of an upload) CID. This
@@ -191,3 +211,8 @@ which it was stored.
   stores: Array<{space: SpaceDID, insertedAt: Date}>
 }
 ```
+
+#### Implementations
+
+- @web3-storage/capabilities: [capability in admin.js](https://github.com/web3-storage/w3up/blob/main/packages/capabilities/src/admin.js)
+- @web3-storage/upload-api: [invocation handler in admin/store/inspect.js](https://github.com/web3-storage/w3up/blob/231cf1f863f4e9a96c92d9ef5001617ba928028d/packages/upload-api/src/admin/store/inspect.js)
diff --git a/w3-filecoin.md b/w3-filecoin.md
@@ -1,6 +1,6 @@
 # W3 Filecoin Protocol
 
-![status:wip](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square)
+![status:reliable](https://img.shields.io/badge/status-reliable-green.svg?style=flat-square)
 
 ## Editors
 
@@ -36,6 +36,7 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "S
       - [`filecoin/offer`]
       - [`filecoin/submit`]
       - [`filecoin/accept`]
+      - [`filecoin/info`]
     - [Aggregator Capabilities](#aggregator-capabilities)
       - [`piece/offer`]
       - [`piece/accept`]
@@ -49,6 +50,7 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "S
     - [`filecoin/offer` schema](#filecoinoffer-schema)
     - [`filecoin/submit` schema](#filecoinsubmit-schema)
     - [`filecoin/accept` schema](#filecoinaccept-schema)
+    - [`filecoin/info` schema](#filecoininfo-schema)
     - [`piece/offer` schema](#pieceoffer-schema)
     - [`piece/accept` schema](#pieceaccept-schema)
     - [`aggregate/offer` schema](#aggregateoffer-schema)
@@ -363,6 +365,96 @@ If the added piece is invalid, the reason for the failure is also reported:
 }
 ```
 
+#### `filecoin/info`
+
+An agent MAY invoke the `filecoin/info` capability to request known information about content piece in Filecoin. See [schema](#filecoininfo-schema).
+
+> `did:key:zAliceAgent` invokes `filecoin/info` capability provided by `did:web:web3.storage`
+
+```json
+{
+  "iss": "did:key:zAliceAgent",
+  "aud": "did:web:web3.storage",
+  "att": [
+    {
+      "with": "did:key:zAlice",
+      "can": "filecoin/info",
+      "nb": {
+        /* Commitment proof for piece */
+        "piece": { "/": "bafk...commp" }
+      }
+    }
+  ],
+  "prf": [],
+  "sig": "..."
+}
+```
+
+##### Filecoin Info Failure
+
+```json
+{
+  "ran": "bafy...filInfo",
+  "out": {
+    "error": {
+      "name": "InvalidContentPiece",
+      "content": { "/": "bafk...commp" }
+    }
+  }
+}
+```
+
+##### Filecoin Info Success
+
+```json
+{
+  "ran": "bafy...filInfo",
+  "out": {
+    "ok": {
+      /* commitment proof for piece */
+      "piece": { "/": "commitment...car" },
+      /* aggregates where this piece was included */
+      "aggregates": [
+        {
+          /* commitment proof for aggregate */
+          "aggregate": { "/": "commitment...aggregate" },
+          /** inclusion proof for piece */
+          "inclusion": {
+            "tree": {
+              "path": [
+                "bafk...root",
+                "bafk...parent",
+                "bafk...child",
+                "bag...car"
+              ],
+              "at": 1
+            },
+            "index": {
+              "path": [/** ... */],
+              "at": 7
+            }
+          }
+        }
+      ],
+      /* filecoin deals where this piece was included */
+      "deals": [
+        {
+          /* commitment proof for aggregate */
+          "aggregate": { "/": "commitment...aggregate" },
+          /** deal type */
+          "aux": {
+            "dataType": 0,
+            "dataSource": {
+              "dealID": 1245
+            }
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
 ### _Aggregator_ Capabilities
 
 #### `piece/offer`
@@ -748,6 +840,20 @@ type FilecoinAccept struct {
 type FilecoinAcceptDetail = FilecoinOfferDetail
 ```
 
+### `filecoin/info` schema
+
+```ipldsch
+type FilecoinInfo struct {
+  with AgentDID
+  nb FilecoinInfoDetail
+}
+
+type FilecoinInfoDetail struct {
+  # Piece as Filecoin Piece with padding
+  piece PieceLink
+}
+```
+
 ### `piece/offer` schema
 
 ```ipldsch
@@ -835,6 +941,7 @@ type DealInfoDetail struct {
 [`filecoin/offer`]:#filecoinoffer
 [`filecoin/submit`]:#filecoinsubmit
 [`filecoin/accept`]:#filecoinaccept
+[`filecoin/info`]:#filecoininfo
 [`piece/offer`]:#pieceoffer
 [`piece/accept`]:#pieceaccept
 [`aggregate/offer`]:#aggregateoffer

diff --git a/w3-rate-limit.md b/w3-rate-limit.md
@@ -1,7 +1,5 @@
 # Rate Limit Protocol
 
-![status:wip](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square)
-
 ## Editors
 
 - [Travis Vachon](https://github.com/travis), [Protocol Labs](https://protocol.ai/)
@@ -10,7 +8,14 @@
 
 - [Travis Vachon](https://github.com/travis), [Protocol Labs](https://protocol.ai/)
 
-# Abstract
+## Status
+
+![reliable badge](https://img.shields.io/badge/status-reliable-green.svg?style=flat-square):
+
+Reliable:
+a spec that has been implemented ([in @web3-storage/upload-api](https://github.com/web3-storage/w3up/blob/main/packages/upload-api/src/rate-limit.js)). It will change as we learn how it works in practice.
+
+## Abstract
 
 Storage providers in the w3 family of protocols need to be able to rate limit (and in many cases, fully block) abusive users
 from using their service. We describe a set of capabilities for tracking and administering such rate limits.
@@ -85,6 +90,11 @@ export const add = capability({
 })
 ```
 
+##### Implementations
+
+- @web3-storage/capabilities [defines `rate-limit/add` capability](https://github.com/web3-storage/w3up/blob/3244a26ac10fb76858903f5271111d350cca05e8/packages/capabilities/src/rate-limit.js#L20)
+- @web3-storage/upload-api [handles `rate-limit/add` invocations](https://github.com/web3-storage/w3up/blob/3244a26ac10fb76858903f5271111d350cca05e8/packages/upload-api/src/rate-limit.js#L10)
+
 #### rate-limit/list
 
 Given a subject ID (e.g., a`did:mailto`, a URL, a domain name, etc), list all rate limits that apply to the given subject.
@@ -131,6 +141,11 @@ export const remove = capability({
 })
 ```
 
+##### Implementations
+
+- @web3-storage/capabilities [defines `rate-limit/list` capability](https://github.com/web3-storage/w3up/blob/3244a26ac10fb76858903f5271111d350cca05e8/packages/capabilities/src/rate-limit.js#L58)
+- @web3-storage/upload-api [handles `rate-limit/list` invocations](https://github.com/web3-storage/w3up/blob/3244a26ac10fb76858903f5271111d350cca05e8/packages/upload-api/src/rate-limit.js#L12)
+
 #### rate-limit/remove
 
 Given a rate limit ID (returned from `rate-limit/add` or `rate-limit/list`), remove the identified rate limit.
@@ -167,3 +182,8 @@ export const remove = capability({
   },
 })
 ```
+
+##### Implementations
+
+- @web3-storage/capabilities [defines `rate-limit/remove` capability](https://github.com/web3-storage/w3up/blob/3244a26ac10fb76858903f5271111d350cca05e8/packages/capabilities/src/rate-limit.js#L40)
+- @web3-storage/upload-api [handles `rate-limit/remove` invocations](https://github.com/web3-storage/w3up/blob/3244a26ac10fb76858903f5271111d350cca05e8/packages/upload-api/src/rate-limit.js#L11)
diff --git a/Pinning Service.md → w3-store-ipfs-pinning.md b/Pinning Service.md → w3-store-ipfs-pinning.md
@@ -1,39 +1,43 @@
-# Web3 Pinning Service
+# w3-store Authorization + IPFS Pinning Service API
 
 ![wip](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square)
 
-We reimagine IPFS [pinning service][] as web3 service, where:
+## Implementations
 
-- Users delegate necessary capabilities to the pinning service as opposed to getting [access token][]s from the service which must be kept secret.
-- As an API layer on top of core upload v2 protocol.
+There are no known implementations of w3-store Authorization + IPFS Pinning Service API.
 
-Below table maps [IPFS pinning service API][pinning service] operations to capabilities necessary to execute them
+See [w3-store](./w3-store.md) for implementations of the storage protocol that this refers to.
+
+## Description
+
+We reimagine IPFS [pinning service][] as a web3 service, where:
+
+- Users delegate necessary capabilities to the pinning service as [UCAN][]s opposed to sharing powerful bearer [access token][]s that must be kept secret by both client and pinning service, and client must trust pinning service to only use as intended
+- As a layer on top of [w3-store](./w3-store.md) protocol
+
+The following table maps [IPFS pinning service API][pinning service] operations to capabilities necessary to execute them
 
 | Operation                                                                                             | Capabilities                       |
 | ----------------------------------------------------------------------------------------------------- | ---------------------------------- |
 | [add](https://ipfs.github.io/pinning-services-api-spec/#tag/pins/paths/~1pins/post)                   | [`store/add`][]                    |
 | [remove](https://ipfs.github.io/pinning-services-api-spec/#tag/pins/paths/~1pins~1{requestid}/delete) | [`store/remove`][]                 |
 | [list](https://ipfs.github.io/pinning-services-api-spec/#tag/pins/paths/~1pins/get)                   | [`store/list`][]                   |
 | [replace](https://ipfs.github.io/pinning-services-api-spec/#tag/pins/paths/~1pins~1{requestid}/post)  | [`store/add`][] [`store/remove`][] |
-| [get](https://ipfs.github.io/pinning-services-api-spec/#tag/pins/paths/~1pins~1{requestid}/get)       | [`store/list`][]                   |
+| [get](https://ipfs.github.io/pinning-services-api-spec/#tag/pins/paths/~1pins~1{requestid}/get)       | [`store/get`][]                   |
 
-Our pinning service implementation will accept [access token] that are valid JWT formatted [UCAN][]s where:
+a w3 pinning service accepts requests whose access token is a [UCAN JWT][] where:
 
 1. Root issuer is the same DID as one in `with` field of the delegated capabilities.
-2. DID in `with` field is associated with some account.
-3. Where all the provided [proofs are embedded](https://github.com/ipld/js-dag-ucan#embedding-proofs) inline.
-
-> ⚠️ It is worth calling out that just like typical [access token][]s these do not need to be kept secret as well, since if compromised they could be used to pin arbitrary data.
+2. DID in `with` field is associated with some account on the Pinning Service.
+3. All the linked [proofs are embedded](https://github.com/ipld/js-dag-ucan#embedding-proofs) inline.
 
-This offers following advantages to typical bearer tokens:
+Using UCANs for authorization offers following advantages to typical [bearer tokens](https://oauth.net/2/bearer-tokens/):
 
-1. Users are able to delegate access to [pinning service] to others.
-2. Users could tokens with subset of capabilities and consequently restricting access to desired operations.
-3. Users could revoke and rotate tokens as they wish.
+1. Users are able to delegate access to [pinning service] to other DIDs.
+2. Users can delegate access to only a subset of their authorizations, restricting access to desired operations.
+3. Users can revoke and rotate tokens as they wish.
 
-> This also creates an opportunity for implementing better [pinning service][] clients issue short lived tokens per operation and remove the need for keeping tokens secret.
->
-> [@ipld/dag-ucan](https://www.npmjs.com/package/@ipld/dag-ucan) library could be used to issue such tokens
+This also creates an opportunity for implementing better [pinning service][] clients that issue short lived tokens for each operation and remove the need for keeping tokens secret. [@ipld/dag-ucan](https://www.npmjs.com/package/@ipld/dag-ucan) library could be used to issue such tokens
 
 ## Request ID
 
@@ -77,7 +81,9 @@ While UCAN validation is not strictly necessary (as they get verified downstream
 [pinning service]: https://ipfs.github.io/pinning-services-api-spec/
 [link-type]: https://github.com/ipld/js-dag-ucan/blob/364379b54cae383198fcf6a9c0016b497e62d422/src/ucan.ts#L227-L242
 [access token]: https://ipfs.github.io/pinning-services-api-spec/#section/Authentication/accessToken
-[`store/add`]: https://github.com/web3-storage/ucanto/blob/w3/w3/store/src/type/store.ts#L76-L78
-[`store/remove`]: https://github.com/web3-storage/ucanto/blob/w3/w3/store/src/type/store.ts#L80-L82
-[`store/list`]: https://github.com/web3-storage/ucanto/blob/w3/w3/store/src/type/store.ts#L84
-[ucan]: https://github.com/ucan-wg/spec/
+[`store/add`]: https://github.com/web3-storage/specs/blob/main/w3-store.md#storeadd
+[`store/remove`]: https://github.com/web3-storage/specs/blob/main/w3-store.md#storeremove
+[`store/list`]: https://github.com/web3-storage/specs/blob/main/w3-store.md#storelist
+[`store/get`]: https://github.com/web3-storage/specs/blob/main/w3-store.md#storeget
+[UCAN]: https://github.com/ucan-wg/spec/
+[UCAN JWT]: https://github.com/ucan-wg/spec/tree/692e8aab59b763a783fe1484131c3f40d997b69a#3-jwt-structure