Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use cache for snapshots even if refs does not exist #1306

Merged
merged 2 commits into from
Jan 24, 2023

Conversation

brycedrennan
Copy link
Contributor

@brycedrennan brycedrennan commented Jan 23, 2023

If one uses hf_hub_download only referencing specific commits the refs folder will not be created even though data will be cached via snapshots and blobs. Subsequent calls to try_to_load_from_cache will return None even though the desired data was in the cache.

Example:

# download something
hf_hub_download(repo_id=repo, revision=commit_hash, filename=filepath, token=token)
# returns None
try_to_load_from_cache(repo_id=repo, revision=commit_hash, filename=filepath)

If one uses `hf_hub_download` only referencing specific commits the `refs` folder will not be created even though data will be cached via `snapshots` and `blobs`.  Subsequent calls to `try_to_load_from_cache` were returning None even though the desired data was in the cache.

Example:

```python
# download something
hf_hub_download(repo_id=repo, revision=commit_hash, filename=filepath, token=token)
# returns None
try_to_load_from_cache(repo_id=repo, revision=commit_hash, filename=filepath)
```
brycedrennan added a commit to brycedrennan/imaginAIry that referenced this pull request Jan 24, 2023
If one uses `hf_hub_download` only referencing specific commits the `refs` folder will not be created even though data will be cached via `snapshots` and `blobs`.  Subsequent calls to `try_to_load_from_cache` will return None even though the desired data was in the cache.

Example:

```python
# download something
hf_hub_download(repo_id=repo, revision=commit_hash, filename=filepath, token=token)
# returns None
try_to_load_from_cache(repo_id=repo, revision=commit_hash, filename=filepath)
```

huggingface/huggingface_hub#1306
brycedrennan added a commit to brycedrennan/imaginAIry that referenced this pull request Jan 24, 2023
If one uses `hf_hub_download` only referencing specific commits the `refs` folder will not be created even though data will be cached via `snapshots` and `blobs`.  Subsequent calls to `try_to_load_from_cache` will return None even though the desired data was in the cache.

Example:

```python
# download something
hf_hub_download(repo_id=repo, revision=commit_hash, filename=filepath, token=token)
# returns None
try_to_load_from_cache(repo_id=repo, revision=commit_hash, filename=filepath)
```

huggingface/huggingface_hub#1306
@Wauplin
Copy link
Contributor

Wauplin commented Jan 24, 2023

Hi @brycedrennan, thanks for reporting the issue and opening a PR for it!
I reviewed it and realized we also had this issue with the .no_exists folder so I tweaked your code to fix both issues. I also added some regression tests. As soon as the checks are green, I'll merge it.

(and sorry for not having it merged under 12 hours 😛
The workaround in your repo is kinda hacky but completely valid :) )

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jan 24, 2023

The documentation is not available anymore as the PR was closed or merged.

@Wauplin Wauplin merged commit bdb9d06 into huggingface:main Jan 24, 2023
@brycedrennan brycedrennan deleted the bd/change-download-cache-check branch February 5, 2023 20:56
@brycedrennan
Copy link
Contributor Author

Thanks @Wauplin 😀

mattstern31 added a commit to mattstern31/imagin-AIry-Python that referenced this pull request Nov 11, 2023
If one uses `hf_hub_download` only referencing specific commits the `refs` folder will not be created even though data will be cached via `snapshots` and `blobs`.  Subsequent calls to `try_to_load_from_cache` will return None even though the desired data was in the cache.

Example:

```python
# download something
hf_hub_download(repo_id=repo, revision=commit_hash, filename=filepath, token=token)
# returns None
try_to_load_from_cache(repo_id=repo, revision=commit_hash, filename=filepath)
```

huggingface/huggingface_hub#1306
superlucky19971023 added a commit to superlucky19971023/imaginAIry that referenced this pull request Aug 18, 2024
If one uses `hf_hub_download` only referencing specific commits the `refs` folder will not be created even though data will be cached via `snapshots` and `blobs`.  Subsequent calls to `try_to_load_from_cache` will return None even though the desired data was in the cache.

Example:

```python
# download something
hf_hub_download(repo_id=repo, revision=commit_hash, filename=filepath, token=token)
# returns None
try_to_load_from_cache(repo_id=repo, revision=commit_hash, filename=filepath)
```

huggingface/huggingface_hub#1306
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants