Mitigate HuggingFace connection errors

[teetone]

Resolves #606

Problem

We infrequently get the following error when downloading from HuggingFace:

ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.

Solution

1. Try to use local files for the tokenizers unless we need to download
2. When we need to download, added additional retry, so _NUM_RETRIES = 6

[percyliang]

Doesn't this already check whether files are cached and would load them if they exist?

[teetone]

It's the same problem as huggingface/transformers#10901. It still tries to download and requires an internet connection even if the files are cached.

[teetone]

I will try to see if upgrading the library version fixes the issue.

[teetone]

I upgraded to the latest version of transformers and can still reproduce the issue:

With my laptop's wifi connection turned off and the tokenizer cached on my local disk:

>>> GPT2TokenizerFast.from_pretrained("gpt2")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/tonyhlee/research/mercury/benchmarking/venv/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1675, in from_pretrained
    resolved_config_file = get_file_from_repo(
  File "/Users/tonyhlee/research/mercury/benchmarking/venv/lib/python3.9/site-packages/transformers/utils/hub.py", line 687, in get_file_from_repo
    resolved_file = cached_path(
  File "/Users/tonyhlee/research/mercury/benchmarking/venv/lib/python3.9/site-packages/transformers/utils/hub.py", line 284, in cached_path
    output_path = get_from_cache(
  File "/Users/tonyhlee/research/mercury/benchmarking/venv/lib/python3.9/site-packages/transformers/utils/hub.py", line 554, in get_from_cache
    raise ValueError(
ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.

local_files_only=True works though:

>>> GPT2TokenizerFast.from_pretrained("gpt2", local_files_only=True)
PreTrainedTokenizerFast(name_or_path='gpt2', vocab_size=50257, model_max_len=1024, is_fast=True, padding_side='right', truncation_side='right', special_tokens={'bos_token': '<|endoftext|>', 'eos_token': '<|endoftext|>', 'unk_token': '<|endoftext|>'})

[percyliang]

I don't understand - if HF always tries to download, then when does it ever use the cache?

[teetone]

I don't understand - if HF always tries to download, then when does it ever use the cache?

I think it's a limitation with the library. I've seen a handful of GitHub issues regarding this issue. I don't think the local_files_only flag should even exist. I think my fix is the right thing to do, and we should merge it to fix #606, but I can take another look.