0.4.0

Release 0.4.0

• OpenAI API support (/v1/completions, /v1/chat/completions, /v1/models).
• Fixed crashes on race conditions (#20, #7).
• Fixed that if a service was in healthcheck stage when large-model-proxy received an interrupt signal, that service was never killed.
• Improved test coverage.
• At least Go 1.18 is now required to build.

Release 0.3.0

Release 0.3.0

• Add HealthcheckCommand and HealthcheckIntervalMilliseconds config options, allowing to not proxy the first connection to the service until the healthcheck command returns 0 exit code.
• Fixed deadlocks in certain cases, improving overall stability.
• Start services in a separate process group and use process groups for killing them, to kill the child processes too.
• Add tests for some of the functionality.

BC break: The 2-second sleep after opening first connection was removed, if you were relying on it, add

"HealthcheckCommand": "sleep 2"

to the service config, but configuring a proper healthcheck instead is recommended for better performance.

Release 0.2.1

Release 0.2.1

• Fix an issue where not specifying idleTimeout value in config would cause a crash
• Fix specific concurrency workloads causing a sync issue and a crash
• Fix trying to stop service if it's already stopped on connection failure

Thank you @lun-4 for the first 2 fixes!

0.2.0

Release 0.2.0

• New feature - idle timeout. ShutDownAfterInactivitySeconds variable in config, on global and config level allows to set time after which the service will be shutdown. The timer is restarted any time connection is opened or closed or the timer is reached and there are still open connections.
• Improved behavior of forwarding multiple connections when service is starting or stopping
• Additional logging when opening service connection
• AUR package

0.1.3

Release 0.1.3

• Increase default timeout when initially connecting to service from 2 minutes to 10 minutes.
• Fix message when this timeout is reached showing incorrect timeout value.

0.1.2

Release 0.1.2

• Fix that multiple connections to the same service were handled synchronously, leading to multiple connections at the same not working.

0.1.1

Release 0.1.1

• Improve handling SIGTERM and SIGINT signals, now the started services are shut down before exiting (thank you @lun-4 for the Pull Request).
• Send SIGTERM first instead of immediate SIGKILL when stopping processes (thank you @lun-4 for the Pull Request).
• Change binary name to large-model-proxy to match project name.
• Avoid use of a function not available in Go 1.19.
• Fix last use time tracking not working.
• Script for making Ubuntu packages.

Full Changelog: 0.1.0...0.1.1