Cassandra (#254)

* start local cassandra is not specified

* executable script and fixes

* fix readme

* fix entrypoint for docker

* wording

* perm-storage -> refact-perm-storage

---------

Co-authored-by: Oleg Klimov <[email protected]>

Dockerfile

@@ -43,6 +43,16 @@ ENV PATH="${PATH}:/tmp/linguist/bin"
 
 RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y python3-packaging
 
+# cassandra
+RUN apt-get install -y \
+    default-jdk \
+    wget \
+    sudo
+RUN echo "deb https://debian.cassandra.apache.org 41x main" | tee -a /etc/apt/sources.list.d/cassandra.sources.list
+RUN curl https://downloads.apache.org/cassandra/KEYS | apt-key add -
+RUN apt-get update
+RUN apt-get install cassandra -y
+
 # refact lsp requisites
 RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | bash -s -- -y
 ENV PATH="${PATH}:/root/.cargo/bin"
@@ -65,4 +75,7 @@ ENV RDMAV_HUGEPAGES_SAFE 0
 
 EXPOSE 8008
 
-CMD ["python", "-m", "self_hosting_machinery.watchdog.docker_watchdog"]
+COPY docker-entrypoint.sh /
+RUN chmod +x docker-entrypoint.sh
+
+CMD ./docker-entrypoint.sh

README.md

@@ -5,7 +5,7 @@
   </picture>
 </p>
 
-This repo consists Refact WebUI for fine-tuning and self-hosting of code models, that you can later use inside Refact plugins for code completion and chat. 
+This repo consists Refact WebUI for fine-tuning and self-hosting of code models, that you can later use inside Refact plugins for code completion and chat.
 
 ---
 
@@ -16,12 +16,12 @@ This repo consists Refact WebUI for fine-tuning and self-hosting of code models,
 [![JetBrains](https://img.shields.io/jetbrains/plugin/d/com.smallcloud.codify?label=JetBrains)](https://plugins.jetbrains.com/plugin/20647-codify)
 
 - [x] Fine-tuning of open-source code models
-- [x] Self-hosting of open-source code models 
+- [x] Self-hosting of open-source code models
 - [x] Download and upload Lloras
-- [x] Use models for code completion and chat inside Refact plugins 
+- [x] Use models for code completion and chat inside Refact plugins
 - [x] Model sharding
 - [x] Host several small models on one GPU
-- [x] Use OpenAI keys to connect GPT-models for chat 
+- [x] Use OpenAI keys to connect GPT-models for chat
 
 ![self-hosting-refact](https://github.com/smallcloudai/refact/assets/5008686/18e48b42-b638-4606-bde0-cadd47fd26e7)
 
@@ -32,16 +32,17 @@ The easiest way to run the self-hosted server is a pre-build Docker image.
 Install [Docker with NVidia GPU support](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker).
 On Windows you need to install WSL 2 first, [one guide to do this](https://docs.docker.com/desktop/install/windows-install).
 
-Run server with following command within this repo:
+Run docker container with following command:
 ```commandline
-docker compose up -d
-```
-To upgrade the docker, you need to run:
-```commandline
-docker compose kill
-docker compose pull
+docker run -d --rm --gpus all -p 8008:8008 -v refact-perm-storage:/perm_storage -v refact-database:/var/lib/cassandra smallcloud/refact_self_hosting:latest
 ```
 
+`perm-storage` is a volume that is mounted inside the container. All the configuration files, downloaded weights and logs are stored here.
+`refact-database` is a volume for database where server stores statistics from your users.
+
+To upgrade the docker, delete it using `docker kill XXX` (the volume `perm-storage` will retain your
+data), run `docker pull smallcloud/refact_self_hosting` and run it again.
+
 Now you can visit http://127.0.0.1:8008 to see the server Web GUI.
 
 

docker-entrypoint.sh

@@ -0,0 +1,6 @@
+#!/bin/sh
+if [ -z "$REFACT_DATABASE_HOST" ]; then
+    sudo service cassandra start
+    echo "cassandra database started on localhost"
+fi
+python -m self_hosting_machinery.watchdog.docker_watchdog