Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow generate signed url for upload objects #646

Closed
Tracked by #703
Xuanwo opened this issue Jun 24, 2021 · 9 comments
Closed
Tracked by #703

Allow generate signed url for upload objects #646

Xuanwo opened this issue Jun 24, 2021 · 9 comments
Assignees

Comments

@Xuanwo
Copy link
Contributor

Xuanwo commented Jun 24, 2021

We provide an API Reach for generating an URL for an object for the user to get.

Maybe we can provide like Reach but for uploading an object.

@Xuanwo
Copy link
Contributor Author

Xuanwo commented Jun 24, 2021

  • Reach -> generating url for Read
  • Xxxxx -> generating url for Write
  • ....

Maybe we can provide a higher-level abstraction for those operations?

Something like Generate(op, path string) (verb, url string) -> Generate("read", "abc.png") -> ("GET", "https://xxxx.com/abc.png")

@Xuanwo Xuanwo changed the title API works like Reach but for write Add general query sign support Jun 24, 2021
@Xuanwo
Copy link
Contributor Author

Xuanwo commented Jun 29, 2021

query sign is rarely used except for GetObject and PutObject, maybe we don't need to support general query sign support.

@Xuanwo Xuanwo transferred this issue from beyondstorage/specs Jul 9, 2021
@Xuanwo Xuanwo changed the title Add general query sign support Allow generate signed url for upload objects Jul 15, 2021
@Xuanwo
Copy link
Contributor Author

Xuanwo commented Jul 22, 2021

How about add API like: (maybe Signer / Sign)

type Accessor interface {
    Access(op, path string, pairs...) (req *http.Request, err error)
}

Or

type Accessor interface {
    AccessRead(path string, pairs...) (req *http.Request, err error)
    AccessWrite(path string, pairs...) (req *http.Request, err error)
}

@JinnyYi
Copy link
Contributor

JinnyYi commented Aug 12, 2021

We can implement it as most object support SignedURL.

For Accessor, I have two other doubts:

  • What's the difference between Reach() and AccessRead()? Both of them are used for reach an object, maybe AccessRead() is not needed?
  • If the service only support AccessRead(), like kodo, does it mean we can't implement Accessor for it?

From simplicity, maybe we can just support PutObjectWithURL. Conversely, maybe we can implement Generate(op, path string) (verb, url string) -> Generate("read", "abc.png") -> ("GET", "https://xxxx.com/abc.png") mentioned above, and services should maintain a list of supported authorized access operations and check whether op is supported.

@Xuanwo
Copy link
Contributor Author

Xuanwo commented Aug 13, 2021

Our users have a need to support the generation of a signed URL that allows their end-users to upload/download the corresponding data directly.

Operations that they need include: (take S3 as an example)

  • PutObject
  • GetObject
  • CreateMultipart
  • WriteMultipart
  • CompleteMultipart

@Xuanwo
Copy link
Contributor Author

Xuanwo commented Aug 13, 2021

Hi, I camp up a new idea that, we can define a new interface like:

type HttpSinger interface {
    QuerySignHttp(req *http.Request, ps ...types.Pair) (signedReq *http.Request, url string, err error)
//    HeaderSignHttp(req *http.Request, ps ...types.Pair) (signedReq *http.Request, auth string, err error)
}

For now, HeaderSignHttp is not required, we can add them later.

This interface can deprecated Reach API.

@JinnyYi
Copy link
Contributor

JinnyYi commented Aug 13, 2021

QuerySignHttp(req *http.Request, ps ...types.Pair) (signedReq *http.Request, url string, err error) is indeed more universality. We can generate request's signed URL in QuerySignHttp.

As external methods are provided for obtaining signed URL for most services, maybe the parameter req *http.Request complicates matters:

  • Users need to pass in the req.URL correctly.
  • We need to parse req.URL to get the wanted request resource(operation, path...).

How about one of the following methods:

  • QuerySignHttp(op, path string, ps ...types.Pair) (signedReq *http.Request, err error)
  • QuerySignHttp(req *Request, ps ...types.Pair) error
    type Request struct {
        Name        string  // "GetObject"
        HTTPPath    string  // "{Key+}"
        HTTPRequest  *http.Request
        ...
    }
    

@Xuanwo
Copy link
Contributor Author

Xuanwo commented Aug 16, 2021

req, err := store.QuerySignHttp(types.OpRead, "abc")

req, err := store.QuerySignHttp(types.OpWrite, "abc", pairs.WithContentType("application/json"))

req, err := store.QuerySignHttp(types.OpWriteMultipart, "abc", pairs.WithMultipartId("qqqq"))

Looks nice.

@JinnyYi
Copy link
Contributor

JinnyYi commented Aug 16, 2021

req, err := store.QuerySignHttp(types.OpRead, "abc")

req, err := store.QuerySignHttp(types.OpWrite, "abc", pairs.WithContentType("application/json"))

req, err := store.QuerySignHttp(types.OpWriteMultipart, "abc", pairs.WithMultipartId("qqqq"))

I will continue the GSP based on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants