The current generator interface expects to receive prompts as str see: https://github.com/leondz/garak/blob/4127ae5092ad3acaba680a32011018fc564cc92a/garak/generators/base.py#L66

This initial simple submission process has worked to date; however #587 show an example of a query prompt that needs a more complex structure. In this case the Multi-modal model accepts both text and image data to generate a response.

I propose an added abstraction layer by implementing a Prompt base interface class that be extended to model these more complex prompts to be processed by each generator.

def generate(self, prompt: Prompt) -> List[str]:

or possibly also abstracting the response as well:

def generate(self, prompt: Prompt) -> List[PromptResponse]:

Prompts can then be further segmented into things like TextPrompt, MultiStepTextPrompt, VisualPrompt, VisualTextPrompt and other such constructs to that on the base functions available to allow use with different and even mixed prompt modalities for models that can accept various input patterns.

Rough example:

class Prompt:
    text = None

    def str(self)
        return self.text

class TextPrompt(Prompt):
    def __init__(self, text: str):
        self.text = text

class VisualTextPrompt(Prompt):
    image
    def __init__(self, text: str, image_path: str):
        self.text = text
        try:
            Image.open(image_path)
         except Exception:
             logger.error(f"No image found at: {image_path}")