Implement LoRA patching via the Loader

[LLukas22]

This pull request adds the ability to apply a LoRA adapter while loading the model. The adapter can be defined via the optional lora_adapter field in ModelParameters.

The patches are then lazily applied while the tensors are loaded via the MmapCompatibleLoader.

This currently produces garbled outputs. Probably because the weights aren't copied correctly from the patched_tensor to the base_tensor.

Currently the weights are moved by setting the tensor data directly, this probably doesn't work as i expected it to.

unsafe {
    tensor.set_data(target_tensor.data());
}

To use a LoRA adapter via the cli simply add a --lora-path flag.

Example: llama infer --model-path "path\to\model" --lora-path "path\to\adapter" -p "The meaning of life is"

[LLukas22]

Alright the output is now correct (at least not gibberish anymore). I just moved the memory regions correctly and its working now.

[philpax]

Will look at soon! Can you mention what you tested with, and make Clippy happy?

[LLukas22]

Sure i used this Alpaca adapter and this LLama base model.

[LLukas22]

Maybe the vicuna-chat should be removed from the examples 🤔

[philpax]

That'd be a question for @danforbes 🤔 What would be your reasoning for removing it?

[LLukas22]

Its fine, just had a brain-lag. Thought i was another inference example but its an actual chat implementation.😅