plaintext bench

[zah]

@nitely has worked on a benchmark for asyncdispatch2, aiming to compare it to the older asyncdispatch and also to some well-known fast frameworks in other languages.

I'm creating this pull request, so we can provide feedback and eventually merge the code.

[zah]

Thanks Esteban,

My expectation was that the TechEmperor benchmark has published their own testing tool. Can we get a comparison of running this tool with your servers, so we can compare Nim's performance now to that of the best results in other languages? You'll have to spend a little bit of time in getting some of these other implementations up and running.

Sorry for not explaining my expectations well enough upfront. I've re-read my email now and I realize that my wording was not very clear.

[zah]

I think you need to add bench to the skipDirs list of the nimble package. Otherwise nimble check will complain that you have violated the rules for allowed package structure.

Alternatively, "bench" can put as a sub-directory inside "tests"

[nitely]

My expectation was that the TechEmperor benchmark has published their own testing tool.

Indeed they have. It just wrk with a custom lua script to allow pipelining. Setting up the frameworks at the top (rust, go, c++) won't be a problem.

[zah]

asyncdispatch2 doesn't support multi-threaded servers yet, so please try to see if you can also force these other frameworks in single-threaded mode. We'll eventually add multi-threading support here, so we'll need to repeat the tests in the future. Some instructions or scripts for this will be appreciated.

[nitely]

I'll do that. But I wonder if multi-threading will be faster than multiple processes + REUSEPORT, I guess I can check how multi-process performs too.

[cheatfate]

First of all thanks for your work @nitely.
Just a remark:
For asyncnet version you not using procedure readHeaders() which internally performs one more allocation for asyncdispatch2.

Also i'm pretty impressed in 57% of boost in speed for single-threaded version, so i think we will have even more boost for multi-threaded version.

[nitely]

Also i'm pretty impressed in 57% of boost in speed for single-threaded version

I forgot to update the asyncnet benchmark after I optimized the server code. It's closer to 60k requests/s now.

@cheatfate btw, I get a Error: unhandled exception: (24) Too many open files [TransportOsError] after running wrk a few times. It seems asyncdispatch2 is not closing the tansport socket.

[cheatfate]

Could you please show me optimized version? because asyncdispatch2 can load all headers in one step too (next step of optimization) and also uses unnecessary readHeaders().

I will check file descriptors leak.

[cheatfate]

But i can't find any descriptors leakage, but found a problem inside of your code.
server.join() is waiting for somebody to call server.close().

So to stop and close server properly you need to use https://github.com/status-im/nim-asyncdispatch2/blob/master/asyncdispatch2/asyncloop.nim#L521 this function to handle CTRL+C. And in handler you can call stop/close sequence.

[nitely]

It seems atEof never returns true (even after the client has closed the connection).

server.join() is waiting for somebody to call server.close()

I never stop the server so that shouldn't be an issue.

[cheatfate]

@nitely you will never get EOF if you will not perform read operation on closed socket, in such way EOF marker will be set.

[nitely]

It works when I call readLine instead of readUntil. I'll dig the issue later.

[cheatfate]

@nitely i have found a problem, leakage happens because readUntil() generated exception IncompleteError, and because tasks are started without asyncCheck this exceptions got hidden.

[nitely]

@cheatfate yeah, that's it. I also found the exception is being swallowed 😄

[cheatfate]

@nitely what do you think about this benchmark https://gist.github.com/cheatfate/2a46181ab03f1a2dc2c943b3e1a56ca5

[arnetheduck]

This PR would need a full rewrite to stay relevant - closing since there has been no activity