Inference error: size mismatch for model.mm_projector.weight

[gameveloster]

After doing a lora finetuning on the pretrained model using finetune_lora.sh, I tried to do inference using CLI

python -m llava.serve.cli \
    --model-path ./checkpoints/llava-Mistral-7B-v0.1-finetune_lora_1 \
    --model-base /models/llm/Mistral-7B-v0.1 \
    --image-file "https://llava-vl.github.io/static/images/view.jpg"

and received this error when loading non_lora_trainables.bin into the model

RuntimeError: Error(s) in loading state_dict for LlavaMistralForCausalLM:
	size mismatch for model.mm_projector.weight: copying a param with shape torch.Size([4096, 1024]) from checkpoint, the shape in current model is torch.Size([2097152, 1]).

There is no problem loading the Bakkala-1 checkpoint by running

python -m llava.serve.cli \
    --model-path /models/lmm/BakLLaVA-1 \
    --image-file "https://llava-vl.github.io/static/images/view.jpg"

Can I have some help to solve this crash when trying to use the LoRA created from the finetuning please? Thank you

[zhyhome]

How did you solve this problem?