How can I convert a model that doesn't have a config.json file

[hamdoudhakem]

Lately after the PR that added the convert-hf-to-gguf-update.py and convert-hf-to-gguf.py scripts i've had a problem of when I try to convert a model it needs to have a certain 'structure' or the convertion wouldn't work.

for instance I downloaded the bloom-3b model using transformers the resulting folder couldn't get converted but when I downloaded the same model using huggingface snapshot it worked. And this time I'm stuck downloading Llama-2-7B-chat using snapshots but the resulting folder couldn't get converted. every time I have this problem I get this error FileNotFoundError: [Errno 2] No such file or directory: 'Llama-2-7b-chat\\config.json'.

So this means that each time I download a model I have to hope that it has the config.json file and that it can get converted which is really frustrating.

Is there a way I can fix this behavior? I want a permanent solution not just one that works this time.

[Galunid]

config.json is part of the huggingface standard. If you don't have it, then you must have made a mistake somewhere in your download script (assuming you download from huggingface). bloom-3b does have this file. Please note that convert-hf-to-gguf.py stands for convert huggingface (models) to gguf. If you get your models elsewhere, then it may fail. The permanent solution would be to simply download models from huggingface.

[hamdoudhakem]

Here is what my downloaded folder looks like :

and here is the script I used to download it :

from huggingface_hub import snapshot_download

model_id="meta-llama/Llama-2-7b-chat"
snapshot_download(repo_id=model_id, local_dir="Llama-2-7b-chat",
                  local_dir_use_symlinks=False, revision="main")

also like I said when I downloaded bloom-3b using the snapshot method I got the config.json file but using the transformers didn't seem to have it. This time I used snapshot directly to download Llama-2-7b-chat but I still don't have the file.

EDIT: it seems like `` doesn't have the config.json file :

but what can I do then?

[Galunid]

For llama/llama2 models you can use examples/convert-legacy-llama.py. It's weird they published it like that though.

[hamdoudhakem]

For llama/llama2 models you can use examples/convert-legacy-llama.py. It's weird they published it like that though.

Thanks, I was finally able to convert it!
But I want to know in case the model that I want to convert doesn't have config.json and isn't a llama model, can I still use this convert script on it?

[Galunid]

No, in general you shouldn't use it for anything other than llama/llama2 based models.

It may or may not with other models, but there are no guarantees.

[ioresult77]

I failed to locate convert-legacy-llama.py in current repo. Please, help.

[ioresult77]

I managed to extract the script from the repo's history, yet it fails after creating exactly 2 Gb of output with the following message:

Traceback (most recent call last):
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 1416, in <module>
    main()
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 1410, in main
    OutputFile.write_all(outfile, ftype, params, model, vocab, special_vocab,
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 1003, in write_all
    of.write_tensor_data(ftype, model, concurrency)
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 931, in write_tensor_data
    for i, ((name, lazy_tensor), ndarray) in enumerate(zip(model.items(), ndarrays)):
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 759, in bounded_parallel_map
    result = futures.pop(0).result()
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/iovchinnikov/anaconda3/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/iovchinnikov/anaconda3/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/iovchinnikov/anaconda3/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 965, in do_item
    tensor = lazy_tensor.load().to_ggml()
             ^^^^^^^^^^^^^^^^^^
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 475, in load
    ret = self._load()
          ^^^^^^^^^^^^
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 485, in load
    return self.load().astype(data_type)
           ^^^^^^^^^^^
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 475, in load
    ret = self._load()
          ^^^^^^^^^^^^
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 530, in load
    ndarrays = [load_unquantized(tensor) for tensor in lazy_tensors]
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 449, in load_unquantized
    tensor = lazy_tensor.load()
             ^^^^^^^^^^^^^^^^^^
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 475, in load
    ret = self._load()
          ^^^^^^^^^^^^
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 644, in load
    return UnquantizedTensor(storage.load(storage_offset, elm_count).reshape(size))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/e/Igor/Immortal/AI/GPT/llama.cpp/examples/convert-legacy-llama.py", line 628, in load
    with self.zip_file.open(info) as fp:
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/iovchinnikov/anaconda3/lib/python3.12/zipfile/__init__.py", line 1652, in open
    raise BadZipFile(f"Overlapped entries: {zinfo.orig_filename!r} (possible zip bomb)")
zipfile.BadZipFile: Overlapped entries: 'consolidated.02/data/3' (possible zip bomb)