Skip to content

Commit

Permalink
new: deepseek integration (closes #24)
Browse files Browse the repository at this point in the history
  • Loading branch information
evilsocket committed Dec 3, 2024
1 parent 2d74fa6 commit 883e78c
Show file tree
Hide file tree
Showing 3 changed files with 59 additions and 1 deletion.
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ docker build . -t nerve

## LLM Support

Nerve features integrations for any model accessible via the [ollama](https://github.com/ollama/ollama), [groq](https://groq.com), [OpenAI](https://openai.com/index/openai-api/), [Anthropic](https://www.anthropic.com/), [Fireworks](https://fireworks.ai/), [Huggingface](https://huggingface.co/blog/tgi-messages-api#using-inference-endpoints-with-openai-client-libraries), [Nvidia NIM](https://www.nvidia.com/en-us/ai/) and [NovitaAI](https://novita.ai/model-api/product/llm-api) APIs.
Nerve features integrations for any model accessible via the [ollama](https://github.com/ollama/ollama), [groq](https://groq.com), [OpenAI](https://openai.com/index/openai-api/), [Anthropic](https://www.anthropic.com/), [Fireworks](https://fireworks.ai/), [Huggingface](https://huggingface.co/blog/tgi-messages-api#using-inference-endpoints-with-openai-client-libraries), [Nvidia NIM](https://www.nvidia.com/en-us/ai/), [DeepSeek](https://deepseek.com/) and [NovitaAI](https://novita.ai/model-api/product/llm-api) APIs.

**The tool will automatically detect if the selected model natively supports function calling. If not, it will provide a compatibility layer that empowers older models to perform function calling anyway.**

Expand Down Expand Up @@ -118,6 +118,12 @@ For **Nvidia NIM**:
NIM_API_KEY=you-api-key nerve -G "nim://nvidia/nemotron-4-340b-instruct" ...
```

For **DeepSeek**:

```sh
DEEPSEEK_API_KEY=you-api-key nerve -G "deepseek://deepseek-chat" ...
```

For **Novita**:

```sh
Expand Down
45 changes: 45 additions & 0 deletions src/agent/generator/deepseek.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
use anyhow::Result;
use async_trait::async_trait;

use crate::agent::state::SharedState;

use super::{openai::OpenAIClient, ChatOptions, ChatResponse, Client};

pub struct DeepSeekClient {
client: OpenAIClient,
}

#[async_trait]
impl Client for DeepSeekClient {
fn new(_: &str, _: u16, model_name: &str, _: u32) -> anyhow::Result<Self>
where
Self: Sized,
{
let client = OpenAIClient::custom(
model_name,
"DEEPSEEK_API_KEY",
"https://api.deepseek.com/v1/",
)?;

Ok(Self { client })
}

async fn check_native_tools_support(&self) -> Result<bool> {
self.client.check_native_tools_support().await
}

async fn chat(
&self,
state: SharedState,
options: &ChatOptions,
) -> anyhow::Result<ChatResponse> {
self.client.chat(state, options).await
}
}

#[async_trait]
impl mini_rag::Embedder for DeepSeekClient {
async fn embed(&self, text: &str) -> Result<mini_rag::Embeddings> {
self.client.embed(text).await
}
}
7 changes: 7 additions & 0 deletions src/agent/generator/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ use serde::{Deserialize, Serialize};
use super::{state::SharedState, Invocation};

mod anthropic;
mod deepseek;
mod fireworks;
mod groq;
mod huggingface;
Expand Down Expand Up @@ -194,6 +195,12 @@ macro_rules! factory_body {
$model_name,
$context_window,
)?)),
"deepseek" => Ok(Box::new(deepseek::DeepSeekClient::new(
$url,
$port,
$model_name,
$context_window,
)?)),
"http" => Ok(Box::new(openai_compatible::OpenAiCompatibleClient::new(
$url,
$port,
Expand Down

0 comments on commit 883e78c

Please sign in to comment.