perf: Reduce allocations

[mhmd-azeez]

This PR aims to improve the performance by:

• reduce allocations
• implement some of the extism functions in the host itself instead of delegating it to the kernel
• adding caching to function calls?

[mhmd-azeez]

Before:

Mo in D:\dylibso\go-sdk on perf/allocations λ go test -bench . -benchmem -run x
goos: windows
goarch: amd64
pkg: github.com/extism/go-sdk
cpu: 12th Gen Intel(R) Core(TM) i7-1270P
BenchmarkNoop/noop-16      19446             61054 ns/op           27135 B/op         39 allocs/op
BenchmarkReplace/4096-16                     727           1383910 ns/op           2.96 MB/s     4649654 B/op       5177 allocs/op
BenchmarkReplace/8192-16                     424           2817234 ns/op           2.91 MB/s     9258105 B/op      10297 allocs/op
BenchmarkReplace/16383-16                    223           5524193 ns/op           2.97 MB/s    18474918 B/op      20539 allocs/op
BenchmarkReplace/32768-16                    100          10907597 ns/op           3.00 MB/s    36907012 B/op      41020 allocs/op
BenchmarkReplace/empty-16                  10000            115713 ns/op           40624 B/op         54 allocs/op
BenchmarkReplace/2048-16                    1398            868701 ns/op           2.36 MB/s     2345005 B/op       2615 allocs/op
PASS
ok      github.com/extism/go-sdk        10.443s

After:

Mo in D:\dylibso\go-sdk on perf/allocations λ go test -bench . -benchmem -run x
goos: windows
goarch: amd64
pkg: github.com/extism/go-sdk
cpu: 12th Gen Intel(R) Core(TM) i7-1270P
BenchmarkNoop/noop-16      19431             61042 ns/op           27191 B/op         41 allocs/op
BenchmarkReplace/2048-16                    7986            142137 ns/op          14.41 MB/s       29739 B/op         51 allocs/op
BenchmarkReplace/4096-16                    5066            217107 ns/op          18.87 MB/s       31766 B/op         51 allocs/op
BenchmarkReplace/8192-16                    3202            360491 ns/op          22.72 MB/s       35862 B/op         51 allocs/op
BenchmarkReplace/16383-16                   1744            734839 ns/op          22.30 MB/s       44058 B/op         51 allocs/op
BenchmarkReplace/32768-16                    822           1419127 ns/op          23.09 MB/s       60446 B/op         51 allocs/op
BenchmarkReplace/empty-16                  10000            122935 ns/op           27669 B/op         50 allocs/op
PASS
ok      github.com/extism/go-sdk        9.771s

[zshipko]

It looks like RuntimeConfig.WithCloseOnContextDone could be causing some slow-down as well: https://github.com/tetratelabs/wazero/blob/009ee7056c1f7afb597781851cc8b0f660c5bbc5/config.go#L151

Maybe we could try adding a PluginConfig.EnableCancellation field that needd to be set to make cancellation work? Then only if EnableCancellation is set then we set WithCloseOnContextDone, otherwise we skip it.

[mhmd-azeez]

The difference is huge!

Before:

Mo in D:\dylibso\go-sdk on perf/allocations λ go test -bench . -benchmem -run x
goos: windows
goarch: amd64
pkg: github.com/extism/go-sdk
cpu: 12th Gen Intel(R) Core(TM) i7-1270P
BenchmarkNoop/noop-16      19370             61954 ns/op           27190 B/op         41 allocs/op
BenchmarkReplace/empty-16                  19249             62648 ns/op           27670 B/op         50 allocs/op
BenchmarkReplace/2048-16                    8539            143100 ns/op          14.31 MB/s       29718 B/op         51 allocs/op
BenchmarkReplace/4096-16                    5859            219129 ns/op          18.69 MB/s       31766 B/op         51 allocs/op
BenchmarkReplace/8192-16                    3568            357805 ns/op          22.90 MB/s       35863 B/op         51 allocs/op
BenchmarkReplace/16383-16                   1718            732431 ns/op          22.37 MB/s       44057 B/op         51 allocs/op
BenchmarkReplace/32768-16                    787           1774815 ns/op          18.46 MB/s       60446 B/op         51 allocs/op
PASS
ok      github.com/extism/go-sdk        11.214s

After

Mo in D:\dylibso\go-sdk on perf/allocations ● ~1 λ go test -bench . -benchmem -run x
goos: windows
goarch: amd64
pkg: github.com/extism/go-sdk
cpu: 12th Gen Intel(R) Core(TM) i7-1270P
BenchmarkNoop/noop-16      70303             16123 ns/op           26229 B/op         23 allocs/op
BenchmarkReplace/empty-16                  72595             16778 ns/op           26228 B/op         23 allocs/op
BenchmarkReplace/2048-16                   18588             64010 ns/op          31.99 MB/s       28277 B/op         24 allocs/op
BenchmarkReplace/4096-16                   10000            113073 ns/op          36.22 MB/s       30325 B/op         24 allocs/op
BenchmarkReplace/8192-16                    5282            222642 ns/op          36.79 MB/s       34422 B/op         24 allocs/op
BenchmarkReplace/16383-16                   2607            496502 ns/op          33.00 MB/s       42616 B/op         24 allocs/op
BenchmarkReplace/32768-16                    745           1496126 ns/op          21.90 MB/s       59003 B/op         24 allocs/op
PASS
ok      github.com/extism/go-sdk        10.074s

Since we are already exposing RuntimeConfig via PluginConfig.RuntimeConfig, Maybe we don't need a separate option, we can just highlight the use case in docs

[nilslice]

Awesome! I suggest we merge this so it's available to anyone trying it out & figure out the docs as you suggest as a follow up.

[bhelx]

Nice work!

…

On Aug 9, 2023 at 2:11 AM -0500, Muhammad Azeez ***@***.***>, wrote: Merged #12 into main. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because your review was requested.Message ID: ***@***.***>