Added support for multiple LoRA adapters

[LLukas22]

Closes #227.

Moved the loading logic into the LoraPatches struct and added the LoraAdapter struct which abstracts multiple LoRA patches.

Changed "loar_adapter" to "lora_adapters" in the ModelParameters.

[philpax]

This looks good, but I think LoraPatches should be renamed to LoraAdapter and the current LoraAdapter's logic should be moved back into the loader.

LoraAdapter's current logic doesn't add much (and unwraps - you can use .collect::<Result<Vec<_>>>()? to avoid that), and the name is confusing (since it can apply multiple LoRAs).

Apart from that, good to go from me.

[LLukas22]

Alright i will move the loading logic back over and just iterate over the LoraAdapter's while loading the tensors.

[LLukas22]

This should be ready, but i havent tested it yet.

[LLukas22]

Old LoRA adapters fail with the following error:

✗ Failed to load model    
Error:
   0: Could not load model
   1: invariant broken: 256 <= 2 in Some("E:\\GGML_Models\\LoRA\\alpaca-7B.bin")

Is this a side-effect of the new quantization formats?

[philpax]

Can you provide more information about which models you tried it with? That sounds like something's misaligned because it comes from

llm/crates/ggml/src/format/loader.rs

Line 209 in 923a09b

return Err(LoadError::InvariantBroken(format!("{n_dims} <= {ne_len}")));

[LLukas22]

Sure i used this LLama base model in q4_0 and these adapters:

• https://huggingface.co/LLukas22/vigogne-instruct-7b-lora-ggml
• https://huggingface.co/LLukas22/alpaca-lora-adapter-ggml

[LLukas22]

Alright, i passed the wrong paths to the LoRA adapter 🤦 . Its now working and tested it with the model and the two adapters linked above.