Tracking Issue: reduce allocations during bulk data transfers

[marten-seemann]

A 1 GB transfer using v0.29.0 currently creates (as measured by pprof's alloc tool, which measures total allocations over the lifetime of the process) about 500 - 600 MB of allocations, both on the sender and on the receiver side.

This is a problem for performance, because allocating memory consumes resources, and more importantly, all of this memory has to be garbage-collected, putting a lot of pressure on the GC.

We need to drastically reduce the amount of allocations. Target is a reduction of an order of magnitude.

The worst offenders at the moment seem to be:

• allocations of wire.{Extended}Header. This is a struct representing a QUIC packet header
• allocations of ackhandler.Packet structs (used to track sent packets until they're acknowledged / declared lost): reduce allocations of ackhandler.Packet #3525
• allocations of wire.AckFrame: use a sync.Pool for ACK frames #3547
• parsing the control message via unix.ParseSocketControlMessage. This will require us to either rewrite that code ourselves, or upstream a fix to golang.org/x/net (both?)
  Go issue: x/sys/unix: add ParseOneSocketControlMessage golang/go#54714
  PR: use the new zero-allocation control message parsing function from x/sys #3609
• use a more efficient type for connection IDs: use a more efficient type for Connection IDs #3527
• stop using a bytes.Buffer when packing packets. Allocating new bytes.Buffers is responsible 32 MB of allocations. We can just append to a []byte
  serialize frames by appending to a byte slice, not to a bytes.Buffer #3530
• don’t allocate a new bytes.Reader for every packet when parsing frames. This is responsible for 32 MB of allocations on the receiver side.
  use a single bytes.Reader for frame parsing #3536
• list allocations, especially when sending packets: use a sync.Pool to reduce allocation of linked list elements #3655

Traces

Server

Client

[zllovesuki]

On v0.30.0 the allocations look way better:

File: specter-linux-amd64
Type: alloc_space
Time: Oct 30, 2022 at 4:30am (PDT)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top5
Showing nodes accounting for 356.03MB, 51.02% of 697.81MB total
Dropped 309 nodes (cum <= 3.49MB)
Showing top 5 nodes out of 109
      flat  flat%   sum%        cum   cum%
  119.51MB 17.13% 17.13%   119.51MB 17.13%  github.com/lucas-clemente/quic-go.(*packetPacker).getShortHeader
   89.51MB 12.83% 29.95%    89.51MB 12.83%  golang.org/x/sys/unix.ParseSocketControlMessage
   69.50MB  9.96% 39.91%    95.51MB 13.69%  github.com/lucas-clemente/quic-go.(*packetPacker).composeNextPacket
   49.50MB  7.09% 47.01%   159.52MB 22.86%  github.com/lucas-clemente/quic-go.(*oobConn).ReadPacket
      28MB  4.01% 51.02%       28MB  4.01%  kon.nect.sh/specter/chord.(*LocalNode).fixK

[marten-seemann]

Interesting article on how to reduce allocations using a sync.Pool in combination with finalizers: https://web.archive.org/web/20220525085959/http://www.golangdevops.com/2019/12/31/autopool/

[marten-seemann]

pprof update, running in a branch that includes the above PRs (#3644, #3646, #3648, #3655). Again, the scenario is the transfer of a single 1 GB file on a single stream.

Server

We're down from 540 MB to 128 MB, that's 76%.

Client

We're down from 514 MB to 147MB, that's 72%. I believe we can get rid of the allocation in ReadPacket as well, by using a sync.Pool for the receivedPacket struct.

[mholt]

Impressive work, Marten!!

[bt90]

#3655 looks fixed

[zllovesuki]

Do we know that allocation overhead (if any) for opening/closing streams? That would be common in the RPC use case

[marten-seemann]

Do we know that allocation overhead (if any) for opening/closing streams? That would be common in the RPC use case

I just added a new benchmark test that opens and accepts streams in #3697. Looks like we're allocating almost 4 kB per stream:

goos: darwin
goarch: arm64
pkg: github.com/quic-go/quic-go/integrationtests/self
BenchmarkStreamChurn-10           561832              2662 ns/op            3696 B/op         42 allocs/op

[marten-seemann]

@zllovesuki If you have any ideas how to optimize that, I'd be happy to review a PR :)