How to use locally hosted LLM's ?

[notV3NOM]

Hi

The code snippet provided in more llms: section of Readme, fails with this error
raise OpenAIError( openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

from autollm import AutoQueryEngine
from autollm.utils.document_reading import read_files_as_documents

llm_model = "ollama/llama2"
llm_api_base = "http://localhost:11434"

documents = read_files_as_documents(input_dir="./data", required_exts=[".md"])

AutoQueryEngine.from_defaults(
     documents=documents,
     llm_model=llm_model,
     llm_api_base=llm_api_base,
)

I assume using OpenAI models is not mandatory

[fcakyon]

You'll need to provide a local embedding model as well.

Example:

import os
from autollm import AutoQueryEngine

os.environ['HUGGINGFACE_API_KEY'] = ""

AutoQueryEngine.from_defaults(
    ...,
    embed_model='huggingface/BAAI/bge-large-zh',
)

[notV3NOM]

Thank you !

I found the other parameters as well in the quickstart colab notebook