Web server doesn't work

[SunixLiu]

System Info

GPT4ALL 2.4.6
Platform: Windows 10
Python 3.10.9

After checking the enable web server box, and try to run server access code here https://docs.gpt4all.io/gpt4all_chat.html#server-mode, I got error message as below:
openai.error.APIConnectionError: Error communicating with OpenAI: HTTPConnectionPool(host='localhost', port=4891): Max retries exceeded with url: /v1/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1427fa0>: Failed to establish a new connection: [Errno 111] Connection refused'))

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• backend
• bindings
• python-bindings
• chat-ui
• models
• circleci
• docker
• api

Reproduction

import openai

openai.api_base = "http://localhost:4891/v1"
#openai.api_base = "https://api.openai.com/v1"

openai.api_key = "not needed for a local LLM"

# Set up the prompt and other parameters for the API request
prompt = "Who is Michael Jordan?"

# model = "gpt-3.5-turbo"
#model = "mpt-7b-chat"
model = "gpt4all-j-v1.3-groovy"

# Make the API request
response = openai.Completion.create(
    model=model,
    prompt=prompt,
    max_tokens=50,
    temperature=0.28,
    top_p=0.95,
    n=1,
    echo=True,
    stream=False
)

# Print the generated completion
print(response)

Expected behavior

{
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"text": "Who is Michael Jordan?\nMichael Jordan is a former professional basketball player who played for the Chicago Bulls in the NBA. He was born on December 30, 1963, and retired from playing basketball in 1998."
}
],
"created": 1684260896,
"id": "foobarbaz",
"model": "gpt4all-j-v1.3-groovy",
"object": "text_completion",
"usage": {
"completion_tokens": 35,
"prompt_tokens": 39,
"total_tokens": 74
}
}

[K-Max-Me]

Failed to establish a new connection: [Errno 111] Connection refused

@SunixLiu - I tried the web server and it works for me. So it may be something else.

Just to make sure, do you do have the "Enable Web Server" option on? (I am 99.999% sure you did, but it doesn't hurt to ask)

If you do, Try the CURL example (or use any http REST client to mimic the HTTP headers/data) as mentioned here - https://platform.openai.com/docs/api-reference/completions/create

So, you'd send a test http POST request:

ie: curl https://localhost:4891/v1/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer Any Key Will Work Locally" \ -d '{ "model": "gpt4all-j-v1.3-groovy", "prompt": "Hello there, how are you?", "max_tokens": 256, "temperature": 0.7 }'

(Each -H item is a HTTP heading, and the -d is the data payload)

If it works, you should get a JSON response back like:

'{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"references":[],"text":"I'm doing well, thank you! How can I assist you today?"}],"created":1686413902,"id":"foobarbaz","model":"gpt4all-j-v1.3-groovy","object":"text_completion","usage":{"completion_tokens":15,"prompt_tokens":16,"total_tokens":31}}'

(Here's a screenshot from a REST Client I used)

If you're still getting the connection refused error then windows firewall might be blocking it.

Check your settings to make sure it isn't blocking the app:

1. Settings >> Windows Security >> Firewall & Network Protection >> Allow a app through firewall.
2. Click Change Settings
3. Click Allow Another App
4. Find and select where chat.exe is.
5. Click OK

Try the REST request again to see if that works.

Good luck!

[SunixLiu]

Thank you K ! This helps!
I finally figured it out that it's because I'm using GPT4All in Windows, but tried to call the API from within WSL, and somehow the network configuration blocks the access.

[bmarcall]

Failed to establish a new connection: [Errno 111] Connection refused

@SunixLiu - I tried the web server and it works for me. So it may be something else.

Just to make sure, do you do have the "Enable Web Server" option on? (I am 99.999% sure you did, but it doesn't hurt to ask)

If you do, Try the CURL example (or use any http REST client to mimic the HTTP headers/data) as mentioned here - https://platform.openai.com/docs/api-reference/completions/create

So, you'd send a test http POST request:

ie: curl https://localhost:4891/v1/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer Any Key Will Work Locally" \ -d '{ "model": "gpt4all-j-v1.3-groovy", "prompt": "Hello there, how are you?", "max_tokens": 256, "temperature": 0.7 }'

(Each -H item is a HTTP heading, and the -d is the data payload)

If it works, you should get a JSON response back like:

'{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"references":[],"text":"I'm doing well, thank you! How can I assist you today?"}],"created":1686413902,"id":"foobarbaz","model":"gpt4all-j-v1.3-groovy","object":"text_completion","usage":{"completion_tokens":15,"prompt_tokens":16,"total_tokens":31}}'

(Here's a screenshot from a REST Client I used)

If you're still getting the connection refused error then windows firewall might be blocking it.

Check your settings to make sure it isn't blocking the app:

1. Settings >> Windows Security >> Firewall & Network Protection >> Allow a app through firewall.
2. Click Change Settings
3. Click Allow Another App
4. Find and select where chat.exe is.
5. Click OK

Try the REST request again to see if that works.

Good luck!

This was very, very useful, Thanks a lot.