Skip to content

Commit

Permalink
fix: bypass hf cache retrieval bug
Browse files Browse the repository at this point in the history
If one uses `hf_hub_download` only referencing specific commits the `refs` folder will not be created even though data will be cached via `snapshots` and `blobs`.  Subsequent calls to `try_to_load_from_cache` will return None even though the desired data was in the cache.

Example:

```python
# download something
hf_hub_download(repo_id=repo, revision=commit_hash, filename=filepath, token=token)
# returns None
try_to_load_from_cache(repo_id=repo, revision=commit_hash, filename=filepath)
```

huggingface/huggingface_hub#1306
  • Loading branch information
superlucky19971023 committed Jan 24, 2023
1 parent e790dfb commit fcca17f
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions imaginairy/model_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -346,6 +346,12 @@ def huggingface_cached_path(url):
dest_path = hf_hub_download(
repo_id=repo, revision=commit_hash, filename=filepath, token=token
)
# make a refs folder so caching works
# work-around for
# https://github.com/huggingface/huggingface_hub/pull/1306
# https://github.com/brycedrennan/imaginAIry/issues/171
refs_url = dest_path[: dest_path.index("/snapshots/")] + "/refs/"
os.makedirs(refs_url, exist_ok=True)
return dest_path


Expand Down

0 comments on commit fcca17f

Please sign in to comment.