From b82b917637714a641b18d589be0fe9efc670c357 Mon Sep 17 00:00:00 2001 From: JinnyYi Date: Thu, 17 Jun 2021 20:02:45 +0800 Subject: [PATCH 1/5] Proposal: Add Service Metadata In Storage Metadata --- ...dd-service-metadata-in-storage-metadata.md | 86 +++++++++++++++++++ 1 file changed, 86 insertions(+) create mode 100644 rfcs/add-service-metadata-in-storage-metadata.md diff --git a/rfcs/add-service-metadata-in-storage-metadata.md b/rfcs/add-service-metadata-in-storage-metadata.md new file mode 100644 index 0000000..9c1b719 --- /dev/null +++ b/rfcs/add-service-metadata-in-storage-metadata.md @@ -0,0 +1,86 @@ +--- +author: JinnyYi +status: draft +updated_at: 2021-06-17 +--- + +# GSP: Add Service Metadata In Storage Metadata + +## Background + +In [GSP-6], we split storage meta and object meta. In [GSP-40], we split all object metadata into four groups: `global` metadata, `standard` metadata, `service` metadata and `user` metadata. For storage, there's no `standard` metadata and user defined metadata is not allowed. + +For now, storage related information carried in `StorageMeta`: + +```go +type StorageMeta struct { + location string + Name string + WorkDir string + + // bit used as a bitmap for object value, 0 means not set, 1 means set + bit uint64 + m map[string]interface{} +} +``` + +It only defines the `global` metadata for storage which could be used in all services, but no field to carry the metadata used in special services. + +## Proposal + +So I propose to add service metadata in storage metadata. + +`service` metadata is provided by service, could only be used in current service. + +The `global` metadata has been included in object type. `service` metadata will be stored in a struct that defined by service: + +- service metadata will be `serviceMetadata interface{}` in `StorageMeta`. + +For services side: + +Example of adding service pair for storage metadata in `service.toml`: + +```go +[infos.storage.meta.] +type = "" +description = "" +``` + +For service metadata, we will introduce `Strong Typed Service Metadata` like `ObjectMetadata`. We will generate following struct for specific service according to service pairs: + +```go +type ServiceMetadata struct { + + ... +} +``` + +And add following generated functions in service packages: + +```go +// Only be used in service to set ServiceMetadata into StorageMeta. +func setServiceMetadata(s *StorageMeta, sm ServiceMetadata) {} +// GetServiceMetadata will get ServiceMetadata from StorageMeta. +func GetServiceMetadata(s *StorageMeta) ServiceMetadata {} +``` + +## Rationale + +This design is highly influenced by [GSP-40]. + +## Compatibility + +No break changes. + +## Implementation + +- `specs` + - Add `service-metadata` with type `any` in [info_storage_meta.toml]. +- `go-storage` + - Generate `serviceMetadata` and corresponding `get/set` functions into `StorageMeta`. + - Support generate `ServiceMetadata` distinguish with `ObjectMetadata` by `Infos.Scope`. + + +[GSP-6]: ./6-normalize-metadata.md +[GSP-40]: ./40-unify-object-metadata.md +[info_storage_meta.toml]: ../definitions/info_storage_meta.toml \ No newline at end of file From 42c7308c180ce2a5989155d6f2e466bdc943064a Mon Sep 17 00:00:00 2001 From: JinnyYi Date: Thu, 17 Jun 2021 20:10:19 +0800 Subject: [PATCH 2/5] Assign number --- ...ata.md => 111-add-service-metadata-in-storage-metadata.md} | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename rfcs/{add-service-metadata-in-storage-metadata.md => 111-add-service-metadata-in-storage-metadata.md} (92%) diff --git a/rfcs/add-service-metadata-in-storage-metadata.md b/rfcs/111-add-service-metadata-in-storage-metadata.md similarity index 92% rename from rfcs/add-service-metadata-in-storage-metadata.md rename to rfcs/111-add-service-metadata-in-storage-metadata.md index 9c1b719..7ca6bfe 100644 --- a/rfcs/add-service-metadata-in-storage-metadata.md +++ b/rfcs/111-add-service-metadata-in-storage-metadata.md @@ -4,7 +4,7 @@ status: draft updated_at: 2021-06-17 --- -# GSP: Add Service Metadata In Storage Metadata +# GSP-111: Add Service Metadata In Storage Metadata ## Background @@ -83,4 +83,4 @@ No break changes. [GSP-6]: ./6-normalize-metadata.md [GSP-40]: ./40-unify-object-metadata.md -[info_storage_meta.toml]: ../definitions/info_storage_meta.toml \ No newline at end of file +[info_storage_meta.toml]: ../definitions/info_storage_meta.toml From e5bd8d924e1d7987b96d853e199b7881be7a767a Mon Sep 17 00:00:00 2001 From: JinnyYi Date: Fri, 18 Jun 2021 17:43:52 +0800 Subject: [PATCH 3/5] Rename service metadata for storage and object --- ...dd-service-metadata-in-storage-metadata.md | 39 ++++++++++++------- 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/rfcs/111-add-service-metadata-in-storage-metadata.md b/rfcs/111-add-service-metadata-in-storage-metadata.md index 7ca6bfe..4ac8f79 100644 --- a/rfcs/111-add-service-metadata-in-storage-metadata.md +++ b/rfcs/111-add-service-metadata-in-storage-metadata.md @@ -8,7 +8,7 @@ updated_at: 2021-06-17 ## Background -In [GSP-6], we split storage meta and object meta. In [GSP-40], we split all object metadata into four groups: `global` metadata, `standard` metadata, `service` metadata and `user` metadata. For storage, there's no `standard` metadata and user defined metadata is not allowed. +In [GSP-6], we split storage meta and object meta. In [GSP-40], we split all metadata into four groups: `global metadata`, `standard metadata`, `service metadata` and `user metadata`. For storage, there's no `standard metadata` and user defined metadata is not allowed. For now, storage related information carried in `StorageMeta`: @@ -24,21 +24,21 @@ type StorageMeta struct { } ``` -It only defines the `global` metadata for storage which could be used in all services, but no field to carry the metadata used in special services. +It only defines the `global metadata` for storage which could be used in all services, but no field to carry the metadata used in special services. ## Proposal -So I propose to add service metadata in storage metadata. +So I propose to add `service metadata` in storage metadata. -`service` metadata is provided by service, could only be used in current service. +### Add service metadata in storage metadata -The `global` metadata has been included in object type. `service` metadata will be stored in a struct that defined by service: +The `global metadata` has been included in object type. `service metadata` for storage will be stored in a struct that defined by service: -- service metadata will be `serviceMetadata interface{}` in `StorageMeta`. +- `service metadata` will be `serviceMetadata interface{}` in `StorageMeta`. -For services side: +For services: -Example of adding service pair for storage metadata in `service.toml`: +Example of adding service's pair for storage metadata in `service.toml`: ```go [infos.storage.meta.] @@ -46,10 +46,10 @@ type = "" description = "" ``` -For service metadata, we will introduce `Strong Typed Service Metadata` like `ObjectMetadata`. We will generate following struct for specific service according to service pairs: +For `service metadata`, we will introduce `Strong Typed Service Metadata`. We will generate following struct for specific service according to service's pairs: ```go -type ServiceMetadata struct { +type StorageServiceMetadata struct { ... } @@ -59,9 +59,20 @@ And add following generated functions in service packages: ```go // Only be used in service to set ServiceMetadata into StorageMeta. -func setServiceMetadata(s *StorageMeta, sm ServiceMetadata) {} +func setStorageServiceMetadata(s *StorageMeta, sm StorageServiceMetadata) {} // GetServiceMetadata will get ServiceMetadata from StorageMeta. -func GetServiceMetadata(s *StorageMeta) ServiceMetadata {} +func GetStorageServiceMetadata(s *StorageMeta) StorageServiceMetadata {} +``` + +### Rename `ObjectMetadata` to `ObjectServiceMetadata` + +For now, `ObjectMetadata` is used to set `service metadata` for `Object`. To avoid confusion, we will rename `ObjectMetadata` to `ObjectServiceMetadata`. Correspondingly, the getter and setter will be updated as follows: + +```go +// Only be used in service to set object service metadata. +func setObjectServiceMetadata(o *Object, om *ObjectServiceMetadata) {} +// Only be used outside service package to get object service metadata. +func GetObjectServiceMetadata(o *Object) *ObjectServiceMetadata {} ``` ## Rationale @@ -70,7 +81,7 @@ This design is highly influenced by [GSP-40]. ## Compatibility -No break changes. +All API call that used object service metadata could be affected. ## Implementation @@ -78,7 +89,7 @@ No break changes. - Add `service-metadata` with type `any` in [info_storage_meta.toml]. - `go-storage` - Generate `serviceMetadata` and corresponding `get/set` functions into `StorageMeta`. - - Support generate `ServiceMetadata` distinguish with `ObjectMetadata` by `Infos.Scope`. + - Support generate `StorageServiceMetadata` distinguish with `ObjectServiceMetadata` by `Infos.Scope`. [GSP-6]: ./6-normalize-metadata.md From cea54e49e2087f6d7e59709365bdcf1faec882e5 Mon Sep 17 00:00:00 2001 From: JinnyYi Date: Fri, 18 Jun 2021 17:52:18 +0800 Subject: [PATCH 4/5] Update comment --- rfcs/111-add-service-metadata-in-storage-metadata.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/111-add-service-metadata-in-storage-metadata.md b/rfcs/111-add-service-metadata-in-storage-metadata.md index 4ac8f79..c92b37d 100644 --- a/rfcs/111-add-service-metadata-in-storage-metadata.md +++ b/rfcs/111-add-service-metadata-in-storage-metadata.md @@ -60,7 +60,7 @@ And add following generated functions in service packages: ```go // Only be used in service to set ServiceMetadata into StorageMeta. func setStorageServiceMetadata(s *StorageMeta, sm StorageServiceMetadata) {} -// GetServiceMetadata will get ServiceMetadata from StorageMeta. +// GetStorageServiceMetadata will get ServiceMetadata from StorageMeta. func GetStorageServiceMetadata(s *StorageMeta) StorageServiceMetadata {} ``` From 4bfcdacdc8393e34ba0364da820bb11178e266dd Mon Sep 17 00:00:00 2001 From: JinnyYi Date: Mon, 21 Jun 2021 15:24:41 +0800 Subject: [PATCH 5/5] Rename service metadata to system metadata --- ...dd-service-metadata-in-storage-metadata.md | 97 ------------------- ...add-system-metadata-in-storage-metadata.md | 85 ++++++++++++++++ 2 files changed, 85 insertions(+), 97 deletions(-) delete mode 100644 rfcs/111-add-service-metadata-in-storage-metadata.md create mode 100644 rfcs/111-add-system-metadata-in-storage-metadata.md diff --git a/rfcs/111-add-service-metadata-in-storage-metadata.md b/rfcs/111-add-service-metadata-in-storage-metadata.md deleted file mode 100644 index c92b37d..0000000 --- a/rfcs/111-add-service-metadata-in-storage-metadata.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -author: JinnyYi -status: draft -updated_at: 2021-06-17 ---- - -# GSP-111: Add Service Metadata In Storage Metadata - -## Background - -In [GSP-6], we split storage meta and object meta. In [GSP-40], we split all metadata into four groups: `global metadata`, `standard metadata`, `service metadata` and `user metadata`. For storage, there's no `standard metadata` and user defined metadata is not allowed. - -For now, storage related information carried in `StorageMeta`: - -```go -type StorageMeta struct { - location string - Name string - WorkDir string - - // bit used as a bitmap for object value, 0 means not set, 1 means set - bit uint64 - m map[string]interface{} -} -``` - -It only defines the `global metadata` for storage which could be used in all services, but no field to carry the metadata used in special services. - -## Proposal - -So I propose to add `service metadata` in storage metadata. - -### Add service metadata in storage metadata - -The `global metadata` has been included in object type. `service metadata` for storage will be stored in a struct that defined by service: - -- `service metadata` will be `serviceMetadata interface{}` in `StorageMeta`. - -For services: - -Example of adding service's pair for storage metadata in `service.toml`: - -```go -[infos.storage.meta.] -type = "" -description = "" -``` - -For `service metadata`, we will introduce `Strong Typed Service Metadata`. We will generate following struct for specific service according to service's pairs: - -```go -type StorageServiceMetadata struct { - - ... -} -``` - -And add following generated functions in service packages: - -```go -// Only be used in service to set ServiceMetadata into StorageMeta. -func setStorageServiceMetadata(s *StorageMeta, sm StorageServiceMetadata) {} -// GetStorageServiceMetadata will get ServiceMetadata from StorageMeta. -func GetStorageServiceMetadata(s *StorageMeta) StorageServiceMetadata {} -``` - -### Rename `ObjectMetadata` to `ObjectServiceMetadata` - -For now, `ObjectMetadata` is used to set `service metadata` for `Object`. To avoid confusion, we will rename `ObjectMetadata` to `ObjectServiceMetadata`. Correspondingly, the getter and setter will be updated as follows: - -```go -// Only be used in service to set object service metadata. -func setObjectServiceMetadata(o *Object, om *ObjectServiceMetadata) {} -// Only be used outside service package to get object service metadata. -func GetObjectServiceMetadata(o *Object) *ObjectServiceMetadata {} -``` - -## Rationale - -This design is highly influenced by [GSP-40]. - -## Compatibility - -All API call that used object service metadata could be affected. - -## Implementation - -- `specs` - - Add `service-metadata` with type `any` in [info_storage_meta.toml]. -- `go-storage` - - Generate `serviceMetadata` and corresponding `get/set` functions into `StorageMeta`. - - Support generate `StorageServiceMetadata` distinguish with `ObjectServiceMetadata` by `Infos.Scope`. - - -[GSP-6]: ./6-normalize-metadata.md -[GSP-40]: ./40-unify-object-metadata.md -[info_storage_meta.toml]: ../definitions/info_storage_meta.toml diff --git a/rfcs/111-add-system-metadata-in-storage-metadata.md b/rfcs/111-add-system-metadata-in-storage-metadata.md new file mode 100644 index 0000000..93cfbd1 --- /dev/null +++ b/rfcs/111-add-system-metadata-in-storage-metadata.md @@ -0,0 +1,85 @@ +--- +author: JinnyYi +status: draft +updated_at: 2021-06-17 +--- + +# GSP-111: Add System Metadata in Storage Metadata + +## Background + +In [GSP-6], we split storage meta and object meta. In [GSP-40], we split all metadata into four groups: `global metadata`, `standard metadata`, `service metadata` and `user metadata`. To avoid concept confusion, we decide to rename `service metadata` to `system metadata` in [Idea: Find a new word to represent Service]. For storage, there's no `standard metadata` and user defined metadata is not allowed. + +For now, storage related information carried in `StorageMeta`: + +```go +type StorageMeta struct { + location string + Name string + WorkDir string + + // bit used as a bitmap for object value, 0 means not set, 1 means set + bit uint64 + m map[string]interface{} +} +``` + +It only defines the `global metadata` for storage which could be used in all services, but no field to carry the system defined metadata in special services. + +## Proposal + +So I propose to add `system metadata` in storage metadata. + +The `global metadata` has been included in object type. `system metadata` will be `systemMetadata interface{}` in `StorageMeta`. + +`system metadata` for storage will be stored in a struct that defined by service. + +For services: + +Example of adding system pair for storage metadata in `service.toml`: + +```go +[infos.storage.meta.] +type = "" +description = "" +``` + +For `system metadata`, we will introduce `Strong Typed System Metadata`. We can generate following struct for specific service according to system pairs: + +```go +type StorageSystemMetadata struct { + + ... +} +``` + +And add following generated functions in service packages: + +```go +// Only be used in service to set SystemMetadata into StorageMeta. +func setStorageSystemMetadata(s *StorageMeta, sm StorageSystemMetadata) {} +// GetStorageSystemMetadata will get SystemMetadata from StorageMeta. +func GetStorageSystemMetadata(s *StorageMeta) StorageSystemMetadata {} +``` + +## Rationale + +This design is highly influenced by [GSP-40]. + +## Compatibility + +No breaking changes. + +## Implementation + +- `specs` + - Add `system-metadata` with type `any` in [info_storage_meta.toml]. +- `go-storage` + - Generate `systemMetadata` and corresponding `get/set` functions into `StorageMeta`. + - Support generate `StorageSystemMetadata` struct to set `system metadata` for storage. + + +[GSP-6]: ./6-normalize-metadata.md +[GSP-40]: ./40-unify-object-metadata.md +[Idea: Find a new word to represent Service]: https://github.com/beyondstorage/specs/issues/114 +[info_storage_meta.toml]: ../definitions/info_storage_meta.toml