Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KoboldCpp client tweaks #109

Closed
vegu-ai-tools opened this issue May 13, 2024 · 0 comments · Fixed by #113
Closed

KoboldCpp client tweaks #109

vegu-ai-tools opened this issue May 13, 2024 · 0 comments · Fixed by #113
Labels
bug Something isn't working Client: KoboldCpp
Milestone

Comments

@vegu-ai-tools
Copy link
Contributor

vegu-ai-tools commented May 13, 2024

  • some issues with repetition breaking when open ai api is selected: rep_pen key error
  • default visualizer to koboldcpp A1111 if koboldcpp is the client and has a sd model loaded
  • default port should be 5001
  • fix token count
  • fix whatever the issue is with
  File "C:\AI\talemate\src\talemate\agents\base.py", line 85, in wrapper
    return await fn(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\talemate\src\talemate\agents\narrator.py", line 46, in narration_wrapper
    response = await fn(self, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\talemate\src\talemate\agents\narrator.py", line 321, in narrate_query
    response = await Prompt.request(
               ^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\talemate\src\talemate\prompts\base.py", line 278, in request
    return await prompt.send(client, kind)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\talemate\src\talemate\prompts\base.py", line 818, in send
    response = await client.send_prompt(str(self), kind=kind)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\talemate\src\talemate\client\base.py", line 546, in send_prompt
    response, finalized_prompt = await self.auto_break_repetition(
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\talemate\src\talemate\client\base.py", line 662, in auto_break_repetition
    prompt_param["max_tokens"] += pad_max_tokens
    ~~~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: 'max_tokens'
 unhandled=True
@vegu-ai-tools vegu-ai-tools added bug Something isn't working Client: KoboldCpp labels May 13, 2024
@vegu-ai-tools vegu-ai-tools added this to the 0.25.3 milestone May 13, 2024
@vegu-ai-tools vegu-ai-tools mentioned this issue May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Client: KoboldCpp
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant