Merge pull request #2007 from owncloud/spaces-api

initial /me/drives implementation

.bingo/oapi-codegen.mod

@@ -0,0 +1,5 @@
+module _ // Auto generated by https://github.com/bwplotka/bingo. DO NOT EDIT
+
+go 1.15
+
+require github.com/deepmap/oapi-codegen v1.6.1 // cmd/oapi-codegen

accounts/go.mod

@@ -9,7 +9,7 @@ require (
 	github.com/asim/go-micro/plugins/client/grpc/v3 v3.0.0-20210408173139-0d57213d3f5c
 	github.com/asim/go-micro/v3 v3.5.1-0.20210217182006-0f0ace1a44a9
 	github.com/cs3org/go-cs3apis v0.0.0-20210614143420-5ee2eb1e7887
-	github.com/cs3org/reva v1.9.1-0.20210628143859-9d29c36c0c3f
+	github.com/cs3org/reva v1.9.1-0.20210708074943-d2f72e0e3b4b
 	github.com/go-chi/chi v4.1.2+incompatible
 	github.com/go-chi/render v1.0.1
 	github.com/gofrs/uuid v3.3.0+incompatible
@@ -31,8 +31,8 @@ require (
 	golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad
 	golang.org/x/net v0.0.0-20210428140749-89ef3d95e781
 	google.golang.org/genproto v0.0.0-20210207032614-bba0dbe2a9ea
-	google.golang.org/grpc v1.38.0
-	google.golang.org/protobuf v1.27.0
+	google.golang.org/grpc v1.39.0
+	google.golang.org/protobuf v1.27.1
 )
 
 replace (

docs/extensions/storage/namespaces.md

@@ -38,7 +38,7 @@ With the spaces concept we are planning to introduce a global namespace to the o
 
 ## CS3 global namespaces
 
-The *CS3 global namespace* in oCIS is configured in the [*storage registry*]({{< ref "./terminology.md#storage-registries" >}}). oCIS uses these defaults:
+The *CS3 global namespace* in oCIS is configured in the [*storage space registry*]({{< ref "./terminology.md#storage-space-registries" >}}). oCIS uses these defaults:
 
 | global namespace | description |
 |-|-|
@@ -48,7 +48,7 @@ The *CS3 global namespace* in oCIS is configured in the [*storage registry*]({{<
 | `/public/<token>` | a virtual folder listing public shares |
 | `/spaces/<spacename>` | *TODO: project or group spaces* |
 
-Technically, the `/home` namespace is not necessary: the [*storage registry*]({{< ref "./terminology.md#storage-registries" >}}) knows the path to a users private space in the `/users` namespace and the gateway can forward the requests to the responsible storage provider.
+Technically, the `/home` namespace is not necessary: the [*storage space registry*]({{< ref "./terminology.md#storage-space-registries" >}}) knows the path to a users private space in the `/users` namespace and the gateway can forward the requests to the responsible storage provider.
 
 {{< hint warning >}}
 *@jfd: Why don't we use `/home/<userlayout>` instead of `/users/<userlayout>`. Then the paths would be consistent with most unix systems.

docs/extensions/storage/terminology.md

@@ -23,7 +23,7 @@ A *reference* is a logical concept that identifies a [*resource*]({{< ref "#reso
 - a [CS3 *id* based reference](https://cs3org.github.io/cs3apis/#cs3.storage.provider.v1beta1.ResourceId), uniquely identifying a [*resource*]({{< ref "#resources" >}}) in the [*namespace*]({{< ref "./namespaces.md" >}}) of a [*storage provider*]({{< ref "#storage-providers" >}}). It consists of a `storage provider id` and an `opaque id`. The `storage provider id` must NOT start with a `/`.
 
 {{< hint info >}}
-The `/` is important because currently the static [*storage registry*]({{< ref "#storage-registries" >}}) uses a map to look up which [*storage provider*]({{< ref "#storage-providers" >}}) is responsible for the resource. Paths must be prefixed with `/` so there can be no collisions between paths and storage provider ids in the same map.
+The `/` is important because currently the static [*storage registry*]({{< ref "#storage-space-registries" >}}) uses a map to look up which [*storage provider*]({{< ref "#storage-providers" >}}) is responsible for the resource. Paths must be prefixed with `/` so there can be no collisions between paths and storage provider ids in the same map.
 {{< /hint >}}
 
 
@@ -108,7 +108,7 @@ by accessing a [*storage system*]({{< ref "#storage-systems" >}}) with a [*stora
 By making [*storage providers*]({{< ref "#storage-providers" >}}) aware of [*storage spaces*]({{< ref "#storage-spaces" >}}) we can get rid of the current `enablehome` flag / hack in reva, which lead to the [spawn of `*home` drivers](https://github.com/cs3org/reva/tree/master/pkg/storage/fs). Furthermore, provisioning a new [*storage space*]({{< ref "#storage-space" >}}) becomes a generic operation, regardless of the need of provisioning a new user home or a new project space.
 {{< /hint >}}
 
-## Storage Registries
+## Storage Space Registries
 
 A *storage registry* manages the [*CS3 global namespace*]({{< ref "./namespaces.md#cs3-global-namespaces" >}}):
 It is used by the *gateway*
@@ -119,7 +119,7 @@ that should handle a [*reference*]({{< ref "#references" >}}).
 
 {{< hint warning >}}
 **Proposed Change**
-A *storage registry* manages the [*namespace*]({{< ref "./namespaces.md" >}}) for a *user*:
+A *storage space registry* manages the [*namespace*]({{< ref "./namespaces.md" >}}) for a *user*:
 It is used by the *gateway*
 to look up `address` and `port` of the [*storage provider*]({{< ref "#storage-providers" >}})
 that is currently serving a [*storage space*]({{< ref "#storage-space" >}}).
@@ -137,7 +137,7 @@ a *quota* and *permissions*, identified by a `storage space id`.
 
 {{< svg src="extensions/storage/static/storagespace.drawio.svg" >}}
 
-Examples would be every user's home storage space, project storage spaces or group storage spaces. While they all serve different purposes and may or may not have workflows like anti virus scanning enabled, we need a way to identify and manage these subtrees in a generic way. By creating a dedicated concept for them this becomes easier and literally makes the codebase cleaner. A [*storage registry*]({{< ref "#storage-registries" >}}) then allows listing the capabilities of [*storage spaces*]({{< ref "#storage-spaces" >}}), e.g. free space, quota, owner, syncable, root etag, upload workflow steps, ...
+Examples would be every user's home storage space, project storage spaces or group storage spaces. While they all serve different purposes and may or may not have workflows like anti virus scanning enabled, we need a way to identify and manage these subtrees in a generic way. By creating a dedicated concept for them this becomes easier and literally makes the codebase cleaner. A [*storage space registry*]({{< ref "#storage-space-registries" >}}) then allows listing the capabilities of [*storage spaces*]({{< ref "#storage-spaces" >}}), e.g. free space, quota, owner, syncable, root etag, upload workflow steps, ...
 
 Finally, a logical `storage space id` is not tied to a specific [*storage provider*]({{< ref "#storage-providers" >}}). If the [*storage driver*]({{< ref "#storage-drivers" >}}) supports it, we can import existing files including their `file id`, which makes it possible to move [*storage spaces*]({{< ref "#storage-spaces" >}}) between [*storage providers*]({{< ref "#storage-providers" >}}) to implement storage classes, e.g. with or without archival, workflows, on SSDs or HDDs.
 

docs/ocis/_index.md

@@ -13,6 +13,28 @@ geekdocFilePath: _index.md
 
 Welcome to oCIS, the modern file-sync and share platform, which is based on our knowledge and experience with the PHP based [ownCloud server](https://owncloud.com/#server).
 
+### The idea of federated storage
+
+To creata a truly federated storage architecture oCIS breaks down the old ownCloud 10 user specific namespace, which is assembled on the server side, and makes the individual parts accessible to clients as storage spaces and storage space registries.
+
+The below diagram shows the core conceps that are the foundation for the new architecture:
+- End user devices can fetch the list of *storage spaces* a user has access to, by querying one or multiple *storage space registries*. The list contains a unique endpoint for every *storage space*.
+- [*Storage space registries*]({{< ref "../extensions/storage/terminology#storage-space-registries" >}}) manage the list of storage spaces a user has access to. They may subscrible to *storage spaces* in order to receive notifications about changes on behalf of an end users mobile or desktop client.
+- [*Storage spaces*]({{< ref "../extensions/storage/terminology#storage-spaces" >}}) represent a collection of files and folders. A users personal files are a *storage space*, a group or project drive is a *storage space*, and even incoming shares are treated and implemented as *storage spaces*. Each with properties like owners, permissions, quota and type.
+- [*Storage providers*]({{< ref "../extensions/storage/terminology#storage-providers" >}}) can hold multiple *storage spaces*. At an oCIS instance, there might be a dedicated *storage provider* responsible for users personal storage spaces. There might be multiple, sharing the load or there might be just one, hosting all types of *storage spaces*.
+
+{{< svg src="ocis/static/idea.drawio.svg" >}}
+
+As an example, Einstein might want to share something with Marie, who has an account at a different identity provider and uses a different storage space registry. The process makes use of [OpenID Connect (OIDC)](https://openid.net/specs/openid-connect-core-1_0.html) for authentication and would look something like this:
+
+To share something with Marie, Einstein would open `https://cloud.zurich.test`. His browser loads oCIS web and presents a login form that uses the [OpenID Connect Discovery](https://openid.net/specs/openid-connect-discovery-1_0.html#EmailSyntax) to look up the OIDC issuer. For `[email protected]` he will end up at `https://idp.zurich.test`, authenticate and get redirected back to `https://cloud.zurich.test`. Now, oCIS web will use a similar discovery to look up the *storage space registry* for the account, based on the email (or username). He will discover that `https://cloud.zurich.test` is also his *storage registry* that the web UI will use to load the list of *storage spaces* that are available to him.
+
+After locating a folder that he wants to share with Marie he enters her email `[email protected]` in the sharing dialog to grant her the editor role. This, in effect, creates a new *storage space* that is registered with the *storage space registry* at `https://cloud.zurich.test`.
+
+Einstein copies the URL in the browser (or an email with the same URL is sent automatically, or the storage registries use a backchannel mechanism). It contains the most specific `storage space id` and a path relative to it: `https://cloud.zurich.test/#/spaces/716199a6-00c0-4fec-93d2-7e00150b1c84/a/rel/path`.
+
+When Marie enters that URL she will be presented with a login form on the `https://cloud.zurich.test` instance, because the share was created on that domain. If `https://cloud.zurich.test` trusts her OpenID Connect identity provider `https://idp.paris.test` she can log in. This time, the *storage space registry* discovery will come up with `https://cloud.paris.test` though. Since that registry is different than the registry tied to `https://cloud.zurich.test` oCIS web can look up the *storage space* `716199a6-00c0-4fec-93d2-7e00150b1c84` and register the WebDAV URL `https://cloud.zurich.test/dav/spaces/716199a6-00c0-4fec-93d2-7e00150b1c84/a/rel/path` in Maries *storage space registry* at `https://cloud.paris.test`. When she accepts that share her clients will be able to sync the new *storage space* at `https://cloud.zurich.test`.
+
 ### oCIS microservice runtime
 
 The oCIS runtime allows us to dynamically manage services running in a single process. We use [suture](https://github.com/thejerf/suture) to create a supervisor tree that starts each service in a dedicated goroutine. By default oCIS will start all built-in oCIS extensions in a single process. Individual services can be moved to other nodes to scale-out and meet specific performance requirements. A [go-micro](https://github.com/asim/go-micro/blob/master/registry/registry.go) based registry allows services in multiple nodes to form a distributed microservice architecture.