Format of dataset when fine-tuning with LoRA

[coffeeBeansz]

Trying to use the lora.py script for fine-tuning. I have prepared a .json file with the following format:

{
    "images": [<image_path_1>, ...]
    "messages": [
            [
                    {
                          "role": "system", "content": [{"text": <system prompt>, "type": "text"}]
                    },
                    {
                           "role": "user", "content":
                                  [
                                         {"image": <image_path_1>, "type": "image"},
                                         {"text": <question>, "type": "text"}
                                  ]
                    },
                    {
                            "role": "assistant", "content": [{"text": <desired response>, "type": "text"}]
                    }
            ],
            ...
     ]
}

And then I have imported Dataset from dataset and used Dataset.from_dict() to convert it to a huggingface dataset.

My question is if this is the correct format of the data? Should I pass the image in the "messages" as well as in the "images"? Or do I only need to pass it in the "images" field? Also should I pass the local path or a PIL image?