Improve memory and cpu performance of restperf collector

[cgrinds]

No description provided.

[cgrinds]

Steps to benchmark and analyze

Baseline

git clone https://github.com/NetApp/harvest.git
cd cmd/collectors/restperf
go test -run=XXX -bench . -benchmem -count 20 -cpuprofile cpu-baseline.prof -memprofile heap-baseline.prof

Baseline Results

Legend

Column	Unit	Want	Note
1			Name
2	scalar		Number of times function was called
3	ns/op	duration, lower is better	Average number of nanoseconds per operation
4	B/op	speed, higher is better	Average number of bytes allocated per operation
5	allocs/op	allocs, lower is better	Number of allocations per operation

goos: darwin
goarch: amd64
pkg: github.com/netapp/harvest/v2/cmd/collectors/restperf
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkRestPerf_PollData-12                  7         163618966 ns/op        53356683 B/op     107093 allocs/op
BenchmarkRestPerf_PollData-12                  7         162988088 ns/op        53368083 B/op     107096 allocs/op
BenchmarkRestPerf_PollData-12                  7         162208384 ns/op        53389025 B/op     107099 allocs/op
BenchmarkRestPerf_PollData-12                  7         162761720 ns/op        53340061 B/op     107093 allocs/op
BenchmarkRestPerf_PollData-12                  7         161598725 ns/op        53334995 B/op     107093 allocs/op
BenchmarkRestPerf_PollData-12                  7         169443129 ns/op        53385154 B/op     107097 allocs/op
BenchmarkRestPerf_PollData-12                  7         162592634 ns/op        53386448 B/op     107098 allocs/op
BenchmarkRestPerf_PollData-12                  7         162747055 ns/op        53372768 B/op     107096 allocs/op
BenchmarkRestPerf_PollData-12                  7         161722839 ns/op        53349408 B/op     107093 allocs/op
BenchmarkRestPerf_PollData-12                  6         173581256 ns/op        53374161 B/op     107096 allocs/op
BenchmarkRestPerf_PollData-12                  7         163321643 ns/op        53392473 B/op     107096 allocs/op
BenchmarkRestPerf_PollData-12                  7         163201108 ns/op        53370452 B/op     107095 allocs/op
BenchmarkRestPerf_PollData-12                  7         168883004 ns/op        53339926 B/op     107092 allocs/op
BenchmarkRestPerf_PollData-12                  7         163765035 ns/op        53323748 B/op     107088 allocs/op
BenchmarkRestPerf_PollData-12                  7         162575892 ns/op        53354563 B/op     107091 allocs/op
BenchmarkRestPerf_PollData-12                  7         161546697 ns/op        53414387 B/op     107104 allocs/op
BenchmarkRestPerf_PollData-12                  7         165637754 ns/op        53328480 B/op     107092 allocs/op
BenchmarkRestPerf_PollData-12                  7         162599737 ns/op        53322353 B/op     107088 allocs/op
BenchmarkRestPerf_PollData-12                  7         162095090 ns/op        53371297 B/op     107094 allocs/op
BenchmarkRestPerf_PollData-12                  7         162261167 ns/op        53402083 B/op     107099 allocs/op
PASS
ok      github.com/netapp/harvest/v2/cmd/collectors/restperf    28.569s

Improvement Results

goos: darwin
goarch: amd64
pkg: github.com/netapp/harvest/v2/cmd/collectors/restperf
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkRestPerf_PollData-12                 66          17037661 ns/op         1121902 B/op      13953 allocs/op
BenchmarkRestPerf_PollData-12                 67          17174233 ns/op         1121729 B/op      13952 allocs/op
BenchmarkRestPerf_PollData-12                 68          17227058 ns/op         1121563 B/op      13953 allocs/op
BenchmarkRestPerf_PollData-12                 69          17997218 ns/op         1121680 B/op      13952 allocs/op
BenchmarkRestPerf_PollData-12                 70          16783147 ns/op         1121456 B/op      13952 allocs/op
BenchmarkRestPerf_PollData-12                 70          16967229 ns/op         1121447 B/op      13952 allocs/op
BenchmarkRestPerf_PollData-12                 69          17227538 ns/op         1121366 B/op      13952 allocs/op
BenchmarkRestPerf_PollData-12                 67          17681042 ns/op         1121449 B/op      13952 allocs/op
BenchmarkRestPerf_PollData-12                 69          16894417 ns/op         1121454 B/op      13951 allocs/op
BenchmarkRestPerf_PollData-12                 70          16715543 ns/op         1121412 B/op      13952 allocs/op
BenchmarkRestPerf_PollData-12                 63          16826054 ns/op         1121248 B/op      13951 allocs/op
BenchmarkRestPerf_PollData-12                 70          17959803 ns/op         1121133 B/op      13951 allocs/op
BenchmarkRestPerf_PollData-12                 70          16912855 ns/op         1121494 B/op      13951 allocs/op
BenchmarkRestPerf_PollData-12                 69          16916882 ns/op         1121228 B/op      13951 allocs/op
BenchmarkRestPerf_PollData-12                 73          16940379 ns/op         1121089 B/op      13951 allocs/op
BenchmarkRestPerf_PollData-12                 70          18218486 ns/op         1121368 B/op      13952 allocs/op
BenchmarkRestPerf_PollData-12                 70          16880036 ns/op         1121139 B/op      13950 allocs/op
BenchmarkRestPerf_PollData-12                 66          16771323 ns/op         1121407 B/op      13952 allocs/op
BenchmarkRestPerf_PollData-12                 72          16708908 ns/op         1121271 B/op      13952 allocs/op
BenchmarkRestPerf_PollData-12                 69          17853464 ns/op         1121374 B/op      13951 allocs/op
PASS
ok      github.com/netapp/harvest/v2/cmd/collectors/restperf    25.422s

Compare baseline to improved version

Each column left to right

• Name of benchmark function
• Time per operation. How long it took to execute each operation. Units are: µ microseconds, m milliseconds, etc.
• % difference between runs. p value is 95ile confidence interval between runs. Lower is better, a value of 0.05 or lower typically indicates statistical significance. Negative values mean the second benchmark is faster.
• B/op Average number of bytes allocated per operation. Units are base 2, IEC units: Ki=1024, Mi=1024², etc.

go install golang.org/x/perf/cmd/benchstat@latest

benchstat restperf_baseline.txt restperf_improve.txt
goos: darwin
goarch: amd64
pkg: github.com/netapp/harvest/v2/cmd/collectors/restperf
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
                     │ restperf_baseline.txt │        restperf_improve.txt         │
                     │        sec/op         │   sec/op     vs base                │
RestPerf_PollData-12            162.75m ± 1%   16.95m ± 2%  -89.58% (p=0.000 n=20)

                     │ restperf_baseline.txt │         restperf_improve.txt         │
                     │         B/op          │     B/op      vs base                │
RestPerf_PollData-12           50.897Mi ± 0%   1.069Mi ± 0%  -97.90% (p=0.000 n=20)

                     │ restperf_baseline.txt │        restperf_improve.txt         │
                     │       allocs/op       │  allocs/op   vs base                │
RestPerf_PollData-12            107.09k ± 0%   13.95k ± 0%  -86.97% (p=0.000 n=20)

Resources

• https://pkg.go.dev/testing#hdr-Benchmarks
• https://dave.cheney.net/2013/06/30/how-to-write-benchmarks-in-go
• https://pkg.go.dev/golang.org/x/perf/cmd/benchstat