Exposing trace context to python backend

[oandreeva-nv]

The purpose of this PR:
This PR is a Stage 1 towards adding an ability for users to create custom traces in python backend.
This set of PRs is heavily focused on OpenTelemetry trace mode.

Functionality:

• Prepares OpenTelemetry trace context via PrepareTraceContext and attaches it to TRITONSERVER_InferenceTrace instance through TRITONSERVER_InferenceTraceSetContex at the moment in time, when REQUEST_START callback happens. That one is happening from the core side, so when PBE model starts execution, it should have context attached to its trace.
• Context is passed as a string and for OpenTelemetry is essentially 2 headers traceparent : "..." and tracestate: "...". Some of you may know about them from OpenTelemetry context propagation discussions. It's the same.
• We do nothing with trace Context in Triton mode. On the core side TRITONSERVER_InferenceTrace is initiated with an empty string as a context, since it is never updated, pbe will get an empty string and return None with request.trace().get_context()

Below is a POC illustrated.

mode=triton tests are on the way.

Related PRs:
python_be: triton-inference-server/python_backend#346
core: triton-inference-server/core#334

POC:

created with model.py:

import time
import json

import numpy as np
import triton_python_backend_utils as pb_utils

from opentelemetry import trace
from opentelemetry.propagate import inject
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
    BatchSpanProcessor
)
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter



class TritonPythonModel:

    def initialize(self, args):
    	trace.set_tracer_provider(TracerProvider())
    	self.tracer = trace.get_tracer_provider().get_tracer("pbe")
    	trace.get_tracer_provider().add_span_processor(
	    BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces"))
	)

    def execute(self, requests):
        logger = pb_utils.Logger
        logger.log_info(f"got {len(requests)} requests")

        responses = []
        for request in requests:
            inp = pb_utils.get_input_tensor_by_name(request, "INPUT0").as_numpy()
            context = request.trace().get_context()
            if context == None:
            	context = "{}"
            ctx = TraceContextTextMapPropagator().extract(carrier=json.loads(context))

            # emulate some work
            with self.tracer.start_as_current_span('child', context=ctx) as span:
            	time.sleep(3)
            
            time.sleep(3)

            output_tensors = [pb_utils.Tensor("OUTPUT0", inp.astype(np.float32))]
            inference_response = pb_utils.InferenceResponse(output_tensors=output_tensors)
            responses.append(inference_response)

        return responses

[rmccorm4]

Nice work 🚀

[kthui]

@oandreeva-nv do we need a copyright statement for this file?

[oandreeva-nv]

Good catch, will follow up shortly