Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

lightning: replace storage.Create by storage.New (#899) #919

Merged
merged 2 commits into from
Mar 25, 2021

Conversation

ti-srebot
Copy link
Contributor

@ti-srebot ti-srebot commented Mar 24, 2021

cherry-pick #899 to release-4.0
You can switch your code base to this Pull Request by using git-extras:

# In br repo:
git pr https://github.com/pingcap/br/pull/919

After apply modifications, you can push your change to this PR via:

git push [email protected]:ti-srebot/br.git pr/919:release-4.0-fea552d4101c

What problem does this PR solve?

  1. storage.Create has been deprecated... so let's replace the remaining two calls from Lightning.
  2. Lightning originally used storage.Create(sendCreds=true), but
    • the entire cloud access happens within Lightning, so we don't need to send credentials to anyone
    • this introduced skipCheckPath=false, which leads to #898.

What is changed and how it works?

Changed storage.Create to storage.New with sendCreds=false and skipCheckPath=true.

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Code changes

Side effects

Related changes

Release Note

  • When using Lightning to import from S3, we no longer require s3:ListBucket permission on the entire bucket, only the data source prefix itself.

@kennytm
Copy link
Collaborator

kennytm commented Mar 24, 2021

/lgtm

@ti-srebot ti-srebot added the status/LGT1 LGTM1 label Mar 24, 2021
Comment on lines +275 to +277
// we skip check path in favor of delaying the error to when we actually access the file.
// on S3, performing "check path" requires the additional "s3:ListBucket" permission.
SkipCheckPath: true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about BR? Does it need the change too?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #898 for details. Yes BR also needs to be fixed, but not via disabling SkipCheckPath everywhere.

@glorv
Copy link
Collaborator

glorv commented Mar 25, 2021

/lgtm

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • glorv

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

@ti-chi-bot ti-chi-bot added status/LGT2 LGTM2 and removed status/LGT1 LGTM1 labels Mar 25, 2021
@kennytm
Copy link
Collaborator

kennytm commented Mar 25, 2021

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 5e1d345

@ti-chi-bot ti-chi-bot merged commit a57cacc into pingcap:release-4.0 Mar 25, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants