Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider making the full download a little less aggressive #22

Closed
davep opened this issue Jan 3, 2025 · 2 comments · Fixed by #75
Closed

Consider making the full download a little less aggressive #22

davep opened this issue Jan 3, 2025 · 2 comments · Fixed by #75
Assignees
Labels
Enhancement New feature or request Priority: High I really want to get to this right away (ish) TODO A thing I'm planning on doing

Comments

@davep
Copy link
Owner

davep commented Jan 3, 2025

Right now the full download of all Raindrop data goes pretty hard. So far raindrop.io isn't complaining, and also it shouldn't happen too often if you're mostly using the Braindrop itself1.

Look at ensuring that the download code observes any throttling, and perhaps also provide a method of interrupting a download.

Footnotes

  1. Little less the case if you're a Pro user as Raindrop will do broken link scanning, etc, which will cause changes on the server that will trigger a fresh download the next time you run up the application.

@davep davep added Enhancement New feature or request TODO A thing I'm planning on doing labels Jan 3, 2025
@davep davep self-assigned this Jan 3, 2025
@davep
Copy link
Owner Author

davep commented Jan 10, 2025

Note the rate limits for the Raindrop API: https://developer.raindrop.io/#rate-limiting

It does say:

For requests using OAuth, you can make up to 120 requests per minute per authenticated user.

which would suggest that Raindrop wouldn't be subject to this because it's not using OAuth; but I suspect this is awkward wording and the same limits apply to calls using the test token too.

It's also worth noting that 120 requests per minute, given the scenario written about in #72, would mean it would take at least 30 seconds to download those 3,000 raindrops. It might be worth looking at estimating the number of pages that need downloading, calculating any pause that needs to be used based on that, and also letting the user know how long they can expect to wait (ish -- there's also the issue of download speed and stuff, of course).

It might make more sense to work around these headers:

X-RateLimit-Limit
X-RateLimit-Remaining
X-RateLimit-Reset

@davep davep added the Priority: High I really want to get to this right away (ish) label Jan 10, 2025
@davep
Copy link
Owner Author

davep commented Jan 10, 2025

Despite what the documentation says, it looks like there isn't a X-RateLimit-Reset header (or any of the other headers) when there's a 429. Instead you get a Retry-After.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request Priority: High I really want to get to this right away (ish) TODO A thing I'm planning on doing
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant