Enable HTTP Keepalive's to reduce number of TLS handshakes

[takeyourhatoff]

We found that when using Credhub with Concourse, there was a massive spike in
CPU usage. We profiled ATC (Concourse component that interacts with Credhub) and
found that >90% of samples were in TLS handshake methods. We did not profile
credhub, but presumably it was much the same.

This commit ensures HTTP response body's are read to completion, which is
required for net/http to reuse the connection and avoid a costly TLS handshake.
It also increases the maximum number of idle HTTP connections to 100, from the
default of 2, which is required to work around limitations in net/http:

golang/go#13801

Signed-off-by: Ife Runsewe [email protected]

[cfdreddbot]

Hey takeyourhatoff!

Thanks for submitting this pull request! I'm here to inform the recipients of the pull request that you and the commit authors have already signed the CLA.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/158865415

The labels on this github issue will be updated when the story is started.

[DanielJonesEB]

Hi CredHubbers! Just so you can see the impact, here's some profiling data before and after this PR:

Here you can see that most of the time is spent doing TLS handshakes:

...and here it's basically invisible because the ATC is now able to get on with it's business:

Here's a (under-represented, because it was previously CPU-starved) demonstration of the impact on the VM:

[mdelillo]

Looks good to me, thanks for submitting this!