Custom LLM model integration (like chatgpt)

[blancsw]

Hi everyone
I would like to use a custom LLM model (like chatGPT) to suggest an answer to a ticket

I add a button on the reply UI:

Now I want to link this button to a http request call with the full ticket conversation to call the openAI api.
What is recomande, create an automation script?

Thanks a lot in advance for help !

[jstanden]

Hi @blancsw!

We've actually done a lot of work on ChatGPT examples for the upcoming 10.4 update. Most of that is still possible in 10.3 and earlier (we've just added conveniences to sheets & interactions). We also have a lot of new examples for using self-hosted fine-tuned Huggingface/SentenceTransformer models (e.g. BART, RoBERTa).

Cerb can integrate with most third-party APIs using keys, OAuth, OIDC, etc. OpenAI's API auth is really easy -- it's just a "bearer" token without the 3-legged OAuth consent.

In Cerb, go to Search >> Connected Services >> (+) >> Build and add a new Token Bearer service. Leave the token name at the default of "Bearer". The URI doesn't matter.

Then go to Search >> Connected Accounts >> (+) and add choose your new "OpenAI" service from above. The URI here is what you will refer to in automations using http.request:. You can use openai, or something more specific if you have multiple accounts (e.g. openai-cerb). Paste your token from: https://platform.openai.com/account/api-keys

Now you can use that connected account to authenticate HTTP requests from automations.

Go to Search >> Automations >> (+) and choose the Trigger: of automation.function.

The code would be something like:

inputs:
  text/prompt:
    type: freeform
    required@bool: yes

start:
  http.request/chatCompletion:
    output: http_response
    inputs:
      method: POST
      url: https://api.openai.com/v1/chat/completions
      headers:
        Content-Type: application/json
      body:
        model: gpt-3.5-turbo
        temperature@key,optional,float: inputs:temperature
        messages:
          0:
            role: user
            content@key: inputs:prompt
      timeout: 20
      authentication: cerb:connected_account:openai
    on_success:
      return:
        response@json: {{http_response.body}}
    on_error:

In the Policy tab at the bottom, allow outgoing HTTP requests to OpenAI's API:

commands:
  http.request:
    deny/url@bool: {{inputs.url is not prefixed ('https://api.openai.com/')}}
    allow@bool: yes

Then in the Run tab, set the test Input: to:

inputs:
  prompt: |
    You are support bot for Cerb helpdesk by Webgroup Media LLC. Answer the following question without making up answers. If you do not know the answer, say "Sorry, I'm not sure."
    Question: What is Cerb?

You should get output like:

_exit: return
__return:
  response:
    id: chatcmpl-a1b2c3d4e5f6
    object: chat.completion
    created: 1688585636
    model: gpt-3.5-turbo-0613
    choices:
    - index: 0
      message:
        role: assistant
        content: Cerb is a comprehensive and flexible helpdesk software developed
          by Webgroup Media LLC. It provides a platform for managing customer support
          inquiries, tickets, and tasks efficiently. It offers features such as email
          integration, workflow automation, team collaboration, analytics, and customizable
          templates. Cerb aims to streamline customer communication and enhance productivity
          for support teams.
      finish_reason: stop

That output is what you'd use for a message draft in Cerb.

You can pass the entire conversation history as user/assistant messages to the OpenAI endpoint. Just be aware of the input token limit. I recommend stripping quoted lines and other non-essential content.

An even better strategy is to use OpenAI (or another casual/generative model) to extract the top intents from an inbound message. Then use those as queries to a knowledgebase/FAQ/Q&A using semantic search. You can either return the the articles to a user directly (e.g. as suggestions in your auto reply), or send the articles to ChatGPT and have it summarize a single answer from the sources. That's also handy as a worker interaction.

For the use case here, you could use https://cerb.ai/docs/automations/events/mail.draft/ to modify the draft being created, or you could add an interaction button to the read (https://cerb.ai/docs/toolbars/interactions/mail.read/) or reply (https://cerb.ai/docs/toolbars/interactions/mail.reply/) toolbars. The latter can autocomplete for you and insert text at the cursor.

The Cerb approach toward AI/ML is very open-ended. Let me know if you need help with anything. I'm happy to provide more examples.

[blancsw]

Hello @jstanden

Sorry for the delay!

Thank you so much for this great answer! Is very helpful!
I'm going to make a little guide to integrating Huggingface on cerbeurs.

Respect for all you work on Cerberus!

[blancsw]

Hi @jstanden

One Last question

I creat this toolbar in mail.reply

interaction/autoreply:
  label: Auto-Reply
  uri: cerb:automation:falcon
  icon: magic
  inputs:
    message: {{message_id}}

And I creat this automation

Trigger: cerb.trigger.interaction.worker

Name: falcon

inputs:
  record/message:
    record_type: message
    required@bool: yes

start:
  set:
    request_body:
      stream@bool: false
      inputs: {{"<|system|>You are a technical support agent who responds to customers, answer in the same language of the input text<|system|><|prompter|>"}}{{inputs.message.content|strip_lines(prefixes='>')|truncate(2000)}}{{"<|endoftext|><|assistant|>"}}
      parameters@json:
        {
          "do_sample": true,
          "max_new_tokens": 1024,
          "repetition_penalty": 1.2,
          "return_full_text": false,
          "stop": [
              "<|endoftext|>"
          ],
          "seed": null,
          "temperature": 0.75,
          "top_k": 50,
          "top_p": 0.95,
          "truncate": null,
          "typical_p": null,
          "best_of": null,
          "watermark": false,
          "details": true,
          "decoder_input_details": false
        }
  http.request:
    output: http_response
    inputs:
      method: POST
      url: https://falcon.t4-1.ik8s.ch
      headers:
        Content-Type: application/json
        Accept: application/json
      body: {{request_body|json_encode}}
    on_success:
      set:
        body@key,json: http_response:body
      return:
        snippet@text:
          Suggestion:
          {{body[0].generated_text}}

Is there a way to get all the messages of the conversations in this automation script?

Thanks a lot in advance for help!

Have a great day!

[jstanden]

Is there a way to get all the messages of the conversations in this automation script?

Sure! The easiest way is to just run a record.search:. That gives you full control over filtering, sorting, and paging.

inputs:
  record/message:
    record_type: message
    required@bool: yes

start:
  record.search:
    output: conversation
    inputs:
      record_type: message
      record_query: ticket.id:${ticket_id} limit:100 sort:-id
      record_query_params:
        ticket_id@int: {{inputs.message.ticket_id}}

Add the following rule to the Policy tab:

  record.search:
    deny/type@bool: {{inputs.record_type is not record type ('message')}}
    allow@bool: yes

You then have a new {{conversations}} placeholder containing an array of messages from the ticket (most recent 100 in descending order).

For instance, you can concatenate the messages into a single string:

  return:
    output: {{conversation|column('content')|join('\n\n')}}

Or you can use |map and format the items however you like:

  return:
    output@text:
      {{conversation|map((msg) => 
        'On '
        ~ msg.created|date('r')
        ~ ', '
        ~ msg.sender__label
        ~ ' wrote:\n'
        ~ msg.content
      )|join('\n\n')}}

[blancsw]

Perfect!

Thank you so much for all your help!

I will share the final code soon.

Have a great day!