Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job Launcher - Fix list objects in bucket function #1391

Merged
merged 1 commit into from
Dec 22, 2023
Merged

Conversation

portuu3
Copy link
Collaborator

@portuu3 portuu3 commented Dec 22, 2023

Description

The method was not properly listing if there were more than 1000 files in the bucket and/or the files were in a nested folder

Summary of changes

  • Add nested folder fix
  • Include pagination in the listing method
  • Move the function to utils (no dependency on a service initialization)
  • Modify and add tests

How test the changes

yarn test

@portuu3 portuu3 added the bug Something isn't working label Dec 22, 2023
@portuu3 portuu3 requested a review from Dzeranov December 22, 2023 09:26
@portuu3 portuu3 self-assigned this Dec 22, 2023
Copy link

vercel bot commented Dec 22, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
job-launcher-server ✅ Ready (Inspect) Visit Preview Dec 22, 2023 9:26am
8 Ignored Deployments
Name Status Preview Updated (UTC)
escrow-dashboard ⬜️ Ignored (Inspect) Dec 22, 2023 9:26am
faucet-server ⬜️ Ignored (Inspect) Dec 22, 2023 9:26am
hufi-exchange-oracle-server ⬜️ Ignored (Inspect) Dec 22, 2023 9:26am
hufi-job-launcher-server ⬜️ Ignored (Inspect) Dec 22, 2023 9:26am
hufi-recording-oracle-server ⬜️ Ignored (Inspect) Dec 22, 2023 9:26am
hufi-reputation-oracle ⬜️ Ignored (Inspect) Dec 22, 2023 9:26am
job-launcher-client ⬜️ Ignored (Inspect) Dec 22, 2023 9:26am
reputation-oracle-server ⬜️ Ignored (Inspect) Dec 22, 2023 9:26am

Comment on lines +63 to +71
const response = await axios.get(
`${url}?list-type=2${
nextContinuationToken
? `&continuation-token=${encodeURIComponent(
nextContinuationToken,
)}`
: ''
}${storageData.path ? `&prefix=${storageData.path}` : ''}`,
);

Check failure

Code scanning / CodeQL

Server-side request forgery

The [URL](1) of this request depends on a [user-provided value](2). The [URL](1) of this request depends on a [user-provided value](3).
@portuu3 portuu3 merged commit ecfb0bf into develop Dec 22, 2023
@portuu3 portuu3 deleted the fix/list-objects branch December 22, 2023 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants