Configuring local Ollama instance for use with LLM honeypots.

[JeffMcGeoff]

Hi all, for context I am using T-Pot with Debian 12.9.0, an i9-13900K, RTX 3080 10 GB, 128 GBs of DDR5 RAM and a 4 TB SSD.

I have installed Ollama as per the Read Me states via curl -fsSL https://ollama.com/install.sh | sh and have downloaded the models llama3.3 and phi4.

I would like to use the LLM model llama3.3 with Gallah and phi4 with Beelzebub. I have confirmed that both models are downloaded correctly and work when I interact with them via terminal prompts.

I have configured my .env file as so...

BEELZEBUB_LLM_MODEL: "ollama"
BEELZEBUB_LLM_HOST: "http://127.0.0.1:11434/api/chat"
BEELZEBUB_OLLAMA_MODEL: "phi4"
......
GALAH_LLM_PROVIDER: "ollama"
GALAH_LLM_SERVER_URL: "http://127.0.0.1:11434"
GALAH_LLM_MODEL: "llama3.3"

...and my docker-compose.yml with the following:

    beelzebub:
    container_name: beelzebub
    restart: always
    depends_on:
      tpotinit:
        condition: service_healthy
    networks:
    - beelzebub_local
    ports:
    - 2022:22
    image: ${TPOT_REPO}/beelzebub:${TPOT_VERSION}
    pull_policy: ${TPOT_PULL_POLICY}
    environment:
      LLM_MODEL: ${BEELZEBUB_LLM_MODEL}
      LLM_HOST: ${BEELZEBUB_LLM_HOST}
      OLLAMA_MODEL: ${BEELZEBUB_OLLAMA_MODEL}
    read_only: true
    volumes:
    - ${TPOT_DATA_PATH}/beelzebub/key:/opt/beelzebub/configurations/key
    - ${TPOT_DATA_PATH}/beelzebub/log:/opt/beelzebub/configurations/log
    ......
      galah:
    container_name: galah
    restart: always
    depends_on:
      tpotinit:
        condition: service_healthy
    networks:
    - galah_local
    ports:
    - 80:80
    - 443:443
    - 8443:8443
    - 8080:8080
    image: ${TPOT_REPO}/galah:${TPOT_VERSION}
    pull_policy: ${TPOT_PULL_POLICY}
    environment:
      LLM_PROVIDER: ${GALAH_LLM_PROVIDER}
      LLM_SERVER_URL: ${GALAH_LLM_SERVER_URL}
      LLM_MODEL: ${GALAH_LLM_MODEL}
    read_only: true
    volumes:
    - ${TPOT_DATA_PATH}/galah/cache:/opt/galah/config/cache
    - ${TPOT_DATA_PATH}/galah/cert:/opt/galah/config/cert
    - ${TPOT_DATA_PATH}/galah/log:/opt/galah/log

However, when interacting with the Beelzebub on port 2022 as per my docker-compose file, for every ssh input I get Command not found as the response back, so I believe obviously that the honeypot is not communicating correctly with the Ollama server, and this will also be the case for Galah on its publicly exposed ports.

Per the discussion within the Issue #1729, I had changed from initially using BEELZEBUB_LLM_HOST: "http://localhost:11434/api/chat" and GALAH_LLM_SERVER_URL: "http://localhost:11434" within the .env file, however I am not keen on running Ollama on a separate PC given the hardware specification of my TPot Server. Further, I have not yet tried using Ollama within a docker container. The ReadMe does not make it clear if Ollama should be setup locally, within a docker container or a seperate host, so maybe this can be revised?

Any advice or guidance would be appreciated.

Thanks all.

[t3chn0m4g3]

127.0.0.1 or localhost will not work, since the container simply tries to contact itself. You need to use the host's IP address.
Otherwise do not change the API endpoint defaults or the honeypot / llm access will not work.

[JeffMcGeoff]

Thank you for the reply @t3chn0m4g3, after reading Ollama's faq.md, particularly Setting environment variables on Linux and adding the line Environment="OLLAMA_HOST=0.0.0.0" within the ollama.service file, I got both honeypots communicating with Ollama.

I have shared relevant snippets of my .env file, should anyone else have a similar issue running Ollama locally.

BEELZEBUB_LLM_MODEL: "ollama"
BEELZEBUB_LLM_HOST: "http://192.168.0.1:11434/api/chat"
BEELZEBUB_OLLAMA_MODEL: "phi4"
......
GALAH_LLM_PROVIDER: "ollama"
GALAH_LLM_SERVER_URL: "http://192.168.0.1:11434"
GALAH_LLM_MODEL: "phi4"

Obviously, replace 192.168.0.1 with the inet IP address displayed in the output of the command ip addr show docker0.

Note: For some reason the model llama3.3 was not working with Galah, switching the model to phi4 Galah was working as intended, maybe two LLM models cannot be used concurrently? - I have not tested this though.

[t3chn0m4g3]

Great to hear it works for you.

Using multiple models in parallel here without any isues.

Setting OLLAMA_KEEP_ALIVE=-1 and OLLAMA_SCHED_SPREAD=true (if having multiple GPUs) should help here.