basic tracing support draft

[guygrigsby]

Here we have a draft PR for Open Telemetry support. It includes span propagation from the client through the proxy and adds the events below. While initially I intended to implement all of them, time restraints have limited what I can do. That being said, I wanted to get this out there so Trickster has minimal tracing support for 1.0 and we can build it up as we go.

The requested support for initial tracing is as follows: (pulled from #36 )

• upstream request time
• fast forward request time
• upstream revalidation request time
• cache query time (also recording the query result in the span)
• cache write time (recording # bytes written into the span)

As I was testing the tracing, I noticed that some traces are happening twice. The more I dug into it, the more I had a suspicion that it wasn't the tracing, but the tracing was exposing that we are making requests twice. This may be expected behavior that I don't understand, but I thought it was worth pointing out. Resolved.

[jranson]

What a Guy! Thank you, sir. Do you happen to know which spans are doubling up? In the delta proxy cache, this would be possible (or probable?), depending upon the spans and how they are observed. For example, if Trickster needs to make parallel requests to the upstream origin, for fast forward, or for reconstituting disparate ranges in the cache to fulfill the client request, that may register the spans instrumented during the upstream request process multiple times.

[guygrigsby]

I was sending multiple queries in a single request. Disregard duplicate comment. It will be hidden.

[jranson]

One general change I would prefer is that we scope a TracingConfig to an Origin, rather than to the Trickster process, as there is a possibility different Origins will need different TracingConfigs (or none at all). I think we can easily change this out for the same map[string]*__Config approach we use for mapping caches and negative caches to an origin. There may be implications with the context to address as a result, but should be minimal effort.

[claassistantio]

All committers have signed the CLA.