Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memoize usearch.sqlite_path() #833

Merged
merged 1 commit into from
Jan 20, 2025
Merged

memoize usearch.sqlite_path() #833

merged 1 commit into from
Jan 20, 2025

Conversation

skshetry
Copy link
Member

@skshetry skshetry commented Jan 20, 2025

If usearch fails to download the extension, it will keep retrying in the future. This adds significant cost - for example, in tests/func/test_pytorch.py run, it was invoked 111 times, taking ~30 seconds in total.

This PR caches the return value for the whole session.

Review with ?w=1 for better diff. :)

@skshetry skshetry requested a review from a team January 20, 2025 06:45
@skshetry skshetry force-pushed the memoize-usearch-ext branch from 4342707 to 187b591 Compare January 20, 2025 06:48
Copy link

cloudflare-workers-and-pages bot commented Jan 20, 2025

Deploying datachain-documentation with  Cloudflare Pages  Cloudflare Pages

Latest commit: d8c1a7a
Status: ✅  Deploy successful!
Preview URL: https://5bba9adb.datachain-documentation.pages.dev
Branch Preview URL: https://memoize-usearch-ext.datachain-documentation.pages.dev

View logs

Copy link

codecov bot commented Jan 20, 2025

Codecov Report

Attention: Patch coverage is 80.00000% with 5 lines in your changes missing coverage. Please review.

Project coverage is 87.42%. Comparing base (c6faa5f) to head (d8c1a7a).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/datachain/sql/sqlite/base.py 80.00% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #833      +/-   ##
==========================================
- Coverage   87.43%   87.42%   -0.01%     
==========================================
  Files         128      128              
  Lines       11392    11408      +16     
  Branches     1552     1553       +1     
==========================================
+ Hits         9961     9974      +13     
- Misses       1040     1043       +3     
  Partials      391      391              
Flag Coverage Δ
datachain 87.36% <76.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@skshetry skshetry force-pushed the memoize-usearch-ext branch from 8cb8aac to 72aff4b Compare January 20, 2025 07:18
@skshetry skshetry force-pushed the memoize-usearch-ext branch from 72aff4b to c28f65e Compare January 20, 2025 07:20
@skshetry skshetry force-pushed the memoize-usearch-ext branch from c28f65e to fa73f86 Compare January 20, 2025 07:30
@skshetry skshetry force-pushed the memoize-usearch-ext branch from fa73f86 to a2f7dbb Compare January 20, 2025 07:33
@skshetry skshetry force-pushed the memoize-usearch-ext branch from a2f7dbb to 77033ef Compare January 20, 2025 07:41
@skshetry skshetry force-pushed the memoize-usearch-ext branch from 77033ef to ad78349 Compare January 20, 2025 07:54
@skshetry skshetry force-pushed the memoize-usearch-ext branch from ad78349 to 8323a58 Compare January 20, 2025 07:56
@skshetry skshetry force-pushed the memoize-usearch-ext branch from 8323a58 to 9973ad7 Compare January 20, 2025 07:59
@skshetry skshetry force-pushed the memoize-usearch-ext branch from 9973ad7 to 375f789 Compare January 20, 2025 08:06
@skshetry skshetry force-pushed the memoize-usearch-ext branch from 375f789 to 64cfd6b Compare January 20, 2025 08:08
@skshetry skshetry force-pushed the memoize-usearch-ext branch from 64cfd6b to 14eb364 Compare January 20, 2025 08:11
If usearch fails to download the extension, it will keep retrying in the
future. This adds significant cost - for example, in `tests/func/test_pytorch.py`
run, it was invoked 111 times, taking ~30 seconds in total.

Now, we cache the return value for the whole session.
@skshetry
Copy link
Member Author

For the record, the failing "Studio tests" are also failing in the main branch. Merging ...

@skshetry skshetry merged commit d648134 into main Jan 20, 2025
31 of 38 checks passed
@skshetry skshetry deleted the memoize-usearch-ext branch January 20, 2025 08:49
dreadatour pushed a commit that referenced this pull request Jan 27, 2025
If usearch fails to download the extension, it will keep retrying in the
future. This adds significant cost - for example, in `tests/func/test_pytorch.py`
run, it was invoked 111 times, taking ~30 seconds in total.

Now, we cache the return value for the whole session.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants