[BUG] simultaneous requests causing memory leak

[Aleksandar1932]

Describe the bug
The current implementation of the HTTP server relies on FastAPI using async handlers and uvicorn as ASGI server. The performance is blazing fast however there are certain issues when scaling the app, and moreover when the number of simultaneous requests increases.

More specifically the result of a significant number of simultaneous requests explodes the memory, thus introducing a memory leak which is handled by the OS by killing the main process. This can cause reliability issues when scaling to production.

Issues: tiangolo/fastapi/1624, tiangolo/fastapi/4649 present the problem and in the evolved discussion in tiangolo/fastapi/4221 a possible solution is presented.

To Reproduce
Steps to reproduce the behavior:

1. Spin up http server, rembg s
2. Send parallel requests, with the tool of your choice. (wrk used at the moment with wrk -c 400 -t 400 http://localhost:5000/\?url\=https://upload.wikimedia.org/wikipedia/commons/thumb/2/2f/Google_2015_logo.svg/2560px-Google_2015_logo.svg.png --latency)
3. Analyze the memory usage
4. Depending on the machine, after some time the memory usage will explode and the FastAPI server will be killed by the OS. (some time in our case on 16GB is after ~30 requests).

Expected behavior
Response time increases.

Images
Any image, Google Logo is provided for example.

OS Version:
Pop!_OS 22.04 LTS x86_64

Rembg version:
v2.0.24

[Aleksandar1932]

Fix provided in !290, and merged.

[onlinebizsoft]

For anyone who has similar issue, I had same issue and it was OK when I use machine with GPU and install this with GPU option.