Add Support for Ollama as an LLM Provider in RAGGENIE

[agberoz]

Enhance RAGGENIE by integrating Ollama as a new LLM provider, enabling users to perform inferences with self-hosted language models.

Task:

• Develop an Ollama loader to facilitate inference generation.
• Ensure the integration is compatible with RAGGENIE's existing low-code architecture for building RAG solutions.

Additional Information:

• Refer existing loaders for creating this integration

[kart2004]

Hi @agberoz, could you please assign it to me, I would love to work on it.

[agberoz]

hi @kart2004 ,

Assigning this issue to you, happy coding

If any assistance is required meet our team here: https://join.slack.com/t/theailounge/shared_invite/zt-2ogkrruyf-FPOHuPr5hdqXl34bDWjHjw

[shravankshenoy]

Hi @agberoz , could you assign this issue to me, I would love to work on this.

I have installed ollama and pulled the llama3.2 model into my local system using ollama pull llama3.2 . Post that I was able to run the following Python code to use the Ollama REST API and generate a response

import ollama 
import requests
import json

url = "http://localhost:11434/api/generate"

headers = {
    "Content-Type" : "application/json"
}

data = {
    "model":"llama3.2",
    "prompt": "Why are leaves green?",
    "stream": False
}

response = requests.post(url, headers=headers, data=json.dumps(data))

if response.status_code == 200:
    response_text = response.text
    data = json.loads(response_text)
    actual_response = data["response"]
    print(actual_response)
else:
    print(response.status_code)

The output of the api call is the following format (reference https://github.com/ollama/ollama/blob/main/docs/api.md)

{
  "model": "llama3.2",
  "created_at": "2024-12-17T19:22:45.499127Z",
  "response": "The leaves are green because.....",
  "done": true,
  "context": [1, 2, 3],
  "total_duration": 5043500667,
  "load_duration": 5025959,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 325953000,
  "eval_count": 290,
  "eval_duration": 4709213000
}

With this information and by looking at the other loaders like the open ai loader, I hope to be able to solve this issue