Skip to content

Commit

Permalink
Merge pull request #55 from MeetKai/server_py_removal
Browse files Browse the repository at this point in the history
Deleted server.py and updated README and requirements
  • Loading branch information
musab-mk authored Nov 20, 2023
2 parents 9a2862c + 590205c commit 24134b4
Show file tree
Hide file tree
Showing 3 changed files with 5 additions and 87 deletions.
7 changes: 3 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,13 @@ Based on [Llama 2](https://arxiv.org/abs/2307.09288).

### Setup

Make sure you have [PyTorch](https://pytorch.org/get-started/locally/) installed. Then:
Make sure you have [PyTorch](https://pytorch.org/get-started/locally/) installed. Then to install the required dependencies, run:

```shell
pip install -r requirements.txt
python3 server.py --model "meetkai/functionary-7b-v1.1"
```

Or start blazing fast [vLLM](https://vllm.readthedocs.io/en/latest/getting_started/installation.html) server:
Now you can start a blazing fast [vLLM](https://vllm.readthedocs.io/en/latest/getting_started/installation.html) server:

```shell
python3 server_vllm.py --model "meetkai/functionary-7b-v1.1" --host 0.0.0.0
Expand Down Expand Up @@ -61,7 +60,7 @@ sudo docker run --gpus all -it --shm-size=8g --name functionary -v ${PWD}/functi
```

### Call Real Python Function
To call the real python function, get the result and extract the result to respond, you can use [chatlab](https://github.com/rgbkrk/chatlab).
To call the real python function, get the result and extract the result to respond, you can use [chatlab](https://github.com/rgbkrk/chatlab). The following example uses chatlab==0.16.0:

```python
from chatlab import Conversation
Expand Down
3 changes: 2 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,5 @@ requests~=2.31.0
PyYAML~=6.0.1
typer==0.9.0
protobuf==3.20.0
tokenizers==0.14.1
tokenizers==0.14.1
vllm==0.2.1
82 changes: 0 additions & 82 deletions server.py

This file was deleted.

0 comments on commit 24134b4

Please sign in to comment.