Local GPT compatible with text-generation-webui

[meistaken]

Hi Pavel, thank you for your plugin. I'm pleased to share that your plugin works with the widely used AI frontend text-generation-webui, which is also compatible with Windows. Previously, I attempted to use the "Text Generator" plugin for Obsidian, but unfortunately, I couldn't get it to work with this frontend.

For anyone who might find this information useful a small guide how to get it works:

1. Start the text-generation-webui and select the desired model (in my case, it's openhermes-2.5-neural-chat-7b-v3-2-7b.Q5_K_M)
   

2. Configure a session

• Select the OpenAI extension and choose API mode (the gallery isn't necessary).
• Click "Apply flags and restart". I usually don't save settings, as I use text-generation in various ways, but you can save these settings as a default if you prefer
  

3. Configure the Local GPT plugin in Obsidian:
   

• Set 'AI provider' to 'OpenAI compatible server'.
• Use the address from the text-generation-webui console, the "OpenAI-compatible API URL" line. In my case, it is http://127.0.0.1:5000/ (note: do not copy the local URL as it won't work, and do not add 'v1' or anything similar to the address).
  
• For 'API Key', use the one provided in the official official plugin documentation: sk-111111111111111111111111111111111111111111111111

That's it. You can now access the Local GPT menu and see the results. I've tested it with a small 7b model (openhermes-2.5-neural-chat-7b-v3-2-7b.Q5_K_M), and on an RTX3090, I achieved 60.98 tokens/s. The quality is quite decent.