This repository has been archived by the owner on Jul 9, 2021. It is now read-only.
-
-
Notifications
You must be signed in to change notification settings - Fork 1
GSP-111: Add System Metadata In Storage Metadata #111
Merged
Merged
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
b82b917
Proposal: Add Service Metadata In Storage Metadata
JinnyYi 42c7308
Assign number
JinnyYi e5bd8d9
Rename service metadata for storage and object
JinnyYi f400c15
merge master into service-metadata
JinnyYi cea54e4
Update comment
JinnyYi 1b2455b
Merge master into service-metadata
JinnyYi 4bfcdac
Rename service metadata to system metadata
JinnyYi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
--- | ||
author: JinnyYi <github.com/JinnyYi> | ||
status: draft | ||
updated_at: 2021-06-17 | ||
--- | ||
|
||
# GSP-111: Add Service Metadata In Storage Metadata | ||
|
||
## Background | ||
|
||
In [GSP-6], we split storage meta and object meta. In [GSP-40], we split all object metadata into four groups: `global` metadata, `standard` metadata, `service` metadata and `user` metadata. For storage, there's no `standard` metadata and user defined metadata is not allowed. | ||
|
||
For now, storage related information carried in `StorageMeta`: | ||
|
||
```go | ||
type StorageMeta struct { | ||
location string | ||
Name string | ||
WorkDir string | ||
|
||
// bit used as a bitmap for object value, 0 means not set, 1 means set | ||
bit uint64 | ||
m map[string]interface{} | ||
} | ||
``` | ||
|
||
It only defines the `global` metadata for storage which could be used in all services, but no field to carry the metadata used in special services. | ||
|
||
## Proposal | ||
|
||
So I propose to add service metadata in storage metadata. | ||
|
||
`service` metadata is provided by service, could only be used in current service. | ||
|
||
The `global` metadata has been included in object type. `service` metadata will be stored in a struct that defined by service: | ||
|
||
- service metadata will be `serviceMetadata interface{}` in `StorageMeta`. | ||
|
||
For services side: | ||
|
||
Example of adding service pair for storage metadata in `service.toml`: | ||
|
||
```go | ||
[infos.storage.meta.<service-meta>] | ||
type = "<type>" | ||
description = "<description>" | ||
``` | ||
|
||
For service metadata, we will introduce `Strong Typed Service Metadata` like `ObjectMetadata`. We will generate following struct for specific service according to service pairs: | ||
|
||
```go | ||
type ServiceMetadata struct { | ||
<service meta> <type> | ||
... | ||
} | ||
``` | ||
|
||
And add following generated functions in service packages: | ||
|
||
```go | ||
// Only be used in service to set ServiceMetadata into StorageMeta. | ||
func setServiceMetadata(s *StorageMeta, sm ServiceMetadata) {} | ||
// GetServiceMetadata will get ServiceMetadata from StorageMeta. | ||
func GetServiceMetadata(s *StorageMeta) ServiceMetadata {} | ||
``` | ||
|
||
## Rationale | ||
|
||
This design is highly influenced by [GSP-40]. | ||
|
||
## Compatibility | ||
|
||
No break changes. | ||
|
||
## Implementation | ||
|
||
- `specs` | ||
- Add `service-metadata` with type `any` in [info_storage_meta.toml]. | ||
- `go-storage` | ||
- Generate `serviceMetadata` and corresponding `get/set` functions into `StorageMeta`. | ||
- Support generate `ServiceMetadata` distinguish with `ObjectMetadata` by `Infos.Scope`. | ||
|
||
|
||
[GSP-6]: ./6-normalize-metadata.md | ||
[GSP-40]: ./40-unify-object-metadata.md | ||
[info_storage_meta.toml]: ../definitions/info_storage_meta.toml |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do I understand correctly?
go-storage/types
, there are typeObject
(has fieldserviceMetadata interface{}
), andStorageMeta
go-service-*
typeObjectMetadata
is used to setserviceMetadata
, and has getterGetObjectMetadata
Now, this proposal intends to also add the field
serviceMetadata interface{}
toStorageMeta
. Should thego-service-*
type be calledStorageMetadata
?And I'm wondering why type
Object
andStorageMeta
are at the same level 🤔There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to rename
go-service-*.ObjectMetadata
into something likego-service-*.ServiceObjectMetadata
to avoid confusion? (and correspondingly,ServiceStorageMeta
in this proposal.)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The root cause is
Object
is a struct butStorager
is an interface.If
Storage
is a struct that exposed to the user, we can setserviceMetadata
inStorage
struct just like we do onObject
.Well,
StorageMeta
andStorageMetadata
do look wired to me too. Do you have a better idea/name for that?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the most wired one is that both
go-storage.Object
andgo-storage.StorageMeta
will have fieldserviceMetadata
,but
go-service-*.ServiceMetadata
will be the one that belongs toStorageMeta
. This seems to be rather confusingThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I got it.
The problem is introduced by simplify
Service Level/Specified Object Metadata
toserviceMetadata
inObject
Service Level/Specified Storage Metadata
toserviceMetadata
inStorageMeta
But I don't have better ideas about the name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about
ServiceObjectMetadata
andServiceStorageMetadata
?Naming is so difficult!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we can find the opposite word of
global
to represent all those situations? So we can:Service Pair
andService's Pair
Service Metadata
andService's Metadata
And our API will turn into:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do I understand correctly?
Service Pair
could be all the pairs in the current service.Service's Pair
just means pairs related to the service implementations.Service Metadata
includesobject service metadata
andstorage service metadata
.Service's Metadata
could corresponds toStorageMeta
here.