Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add appender support #40

Merged
merged 7 commits into from
Apr 28, 2021
Merged

Add appender support #40

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a537bd1
Add appender support
Apr 23, 2021
1b20f44
modify newObject stat parameter and add optional parameters for append
JinnyYi Apr 25, 2021
39c09bd
add the XQSNextAppendPosition pointer nil check
JinnyYi Apr 25, 2021
bad705e
change the way to get storageClass
JinnyYi Apr 25, 2021
2e4e311
Implement commitAppend
Apr 27, 2021
a666278
modify append object metadata setting and add isAppend check
Apr 28, 2021
b2fa995
set append position
Apr 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
213 changes: 205 additions & 8 deletions generated.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.14
require (
bou.ke/monkey v1.0.2
github.com/aos-dev/go-integration-test/v3 v3.0.0
github.com/aos-dev/go-storage/v3 v3.6.0
github.com/aos-dev/go-storage/v3 v3.6.1-0.20210427074731-1b3b4e86c950
github.com/golang/mock v1.5.0
github.com/google/uuid v1.2.0
github.com/pengsrc/go-shared v0.2.1-0.20190131101655-1999055a4a14
Expand Down
8 changes: 4 additions & 4 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ github.com/Xuanwo/templateutils v0.1.0/go.mod h1:OdE0DJ+CJxDBq6psX5DPV+gOZi8bhuH
github.com/aos-dev/go-integration-test/v3 v3.0.0 h1:rxIc7YBfiw9JS5JiVU4BZAqQqoKJemIftFaeJpYCb5M=
github.com/aos-dev/go-integration-test/v3 v3.0.0/go.mod h1:woC3E9Ld1G/Cpo2tSEQ+iwIr4MMdgf+L/8UuyJOE11Q=
github.com/aos-dev/go-storage/v3 v3.5.0/go.mod h1:PZJT0Ta7YxVM5QoYoh8Q/X4I6e/z/7gOJqm85Aib4nY=
github.com/aos-dev/go-storage/v3 v3.6.0 h1:ywjMvh320+esJH81MqB9nyuMNLW97Krujz2UiprC2ZM=
github.com/aos-dev/go-storage/v3 v3.6.0/go.mod h1:ZQwybmoCcTWUOWg+G15gT/NQJoI8G8KH1pF41TuJqYk=
github.com/aos-dev/go-storage/v3 v3.6.1-0.20210427074731-1b3b4e86c950 h1:3ZsfgDAUF171rIrFdYjhr8P3WyxzR4IwxWGbGxkCRkA=
github.com/aos-dev/go-storage/v3 v3.6.1-0.20210427074731-1b3b4e86c950/go.mod h1:wzxhpUa73aCdTDRG0sHjNnnmXa9rC/ahJnk7hKOV6Pw=
github.com/aos-dev/specs/go v0.0.0-20210312090615-23109627848b/go.mod h1:XTNlLZtPA1inITyDH5hNnQXVjvvKUvo+lurs5GYB8NA=
github.com/aos-dev/specs/go v0.0.0-20210423110314-8361397c2bf3 h1:e65ozDhdfHfhnDpZF9SLcY5mwtAg/sAvNIUAkFd+4D0=
github.com/aos-dev/specs/go v0.0.0-20210423110314-8361397c2bf3/go.mod h1:gNah3KaPJEfysh7uCCX+sYjQC3g2yx2VgBkFlT945Ws=
github.com/aos-dev/specs/go v0.0.0-20210427074008-93183332d504 h1:vHxnQOXlz/X8/Co1ep57yc34Xso+f2+xt4Q2dMJVHA4=
github.com/aos-dev/specs/go v0.0.0-20210427074008-93183332d504/go.mod h1:gNah3KaPJEfysh7uCCX+sYjQC3g2yx2VgBkFlT945Ws=
github.com/dave/dst v0.26.2 h1:lnxLAKI3tx7MgLNVDirFCsDTlTG9nKTk7GcptKcWSwY=
github.com/dave/dst v0.26.2/go.mod h1:UMDJuIRPfyUCC78eFuB+SV/WI8oDeyFDvM/JR6NI3IU=
github.com/dave/gopackages v0.0.0-20170318123100-46e7023ec56e/go.mod h1:i00+b/gKdIDIxuLDFob7ustLAVqhsZRk2qVZrArELGQ=
Expand Down
5 changes: 4 additions & 1 deletion service.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ optional = ["location"]
optional = ["location"]

[namespace.storage]
implement = ["copier", "fetcher", "mover", "multiparter", "reacher"]
implement = ["appender", "copier", "fetcher", "mover", "multiparter", "reacher"]

[namespace.storage.new]
required = ["name"]
Expand All @@ -44,6 +44,9 @@ optional = ["offset", "io_callback", "size", "encryption_customer_algorithm", "e
[namespace.storage.op.write]
optional = ["content_md5", "content_type", "io_callback", "storage_class", "encryption_customer_algorithm", "encryption_customer_key"]

[namespace.storage.op.write_append]
optional = ["content_md5", "content_type", "storage_class"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can specify content_type and storage_class in write_append?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, those values should be specified in the first write. But it's not a good idea to introduce these value in write_append. Can we set them in create_append like append 0 byte to object?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can't do this, we need to save those value in the object and set them in the first time that user call write_append.

Something like the following:

offset, _ := o.GetAppendOffset()
if offset == 0 {
    value, _ = o.GetStorageClass()
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can set them in create_append through append 0 byte to object.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can't, we can set those metadata in create_append and use them while the user calls write_append the first time.


[namespace.storage.op.copy]
optional = ["encryption_customer_algorithm", "encryption_customer_key", "copy_source_encryption_customer_algorithm", "copy_source_encryption_customer_key"]

Expand Down
Loading