Fix W3CTraceContextPropagator for non-standard array types #6654

bouwkast · 2025-02-11T14:01:43Z

Summary of changes

The W3CTraceContextPropagator would check whether there was a single traceparent header value to determine whether or not it should extract W3C headers further. For non-standard array types it would always return false meaning that headers wouldn't be extracted. This pull request fixes that so that non-standard array types will return true to allow extraction to happen.

Reason for change

If we have valid headers we should extract them.

Implementation details

Swapped to GetEnumerator as it seems more readable than the foreach route.
If we have one value return true instead of false.
Swapped the TryGetSingle and TryGetSingleRare to internal to add direct unit tests to them.

Test coverage

Added tests for TryGetSingle and TryGetSingleRare testing fundamental behavior.
Added tests for Extract to test possible behavior we could see.

Other details

Fixes #6596

andrewlock

Nice catch 👍

andrewlock · 2025-02-11T14:36:13Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6654) - mean (69ms)  : 66, 72
     .   : milestone, 69,
    master - mean (70ms)  : 64, 76
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (6654) - mean (990ms)  : 967, 1013
     .   : milestone, 990,
    master - mean (994ms)  : 968, 1019
     .   : milestone, 994,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6654) - mean (102ms)  : 100, 105
     .   : milestone, 102,
    master - mean (102ms)  : 100, 104
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (6654) - mean (671ms)  : 656, 686
     .   : milestone, 671,
    master - mean (675ms)  : 662, 688
     .   : milestone, 675,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6654) - mean (89ms)  : 87, 91
     .   : milestone, 89,
    master - mean (90ms)  : 87, 92
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (6654) - mean (626ms)  : 610, 641
     .   : milestone, 626,
    master - mean (635ms)  : 608, 662
     .   : milestone, 635,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6654) - mean (190ms)  : 185, 195
     .   : milestone, 190,
    master - mean (190ms)  : 186, 195
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6654) - mean (1,101ms)  : 1076, 1127
     .   : milestone, 1101,
    master - mean (1,104ms)  : 1076, 1131
     .   : milestone, 1104,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6654) - mean (270ms)  : 266, 275
     .   : milestone, 270,
    master - mean (270ms)  : 266, 273
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (6654) - mean (861ms)  : 834, 888
     .   : milestone, 861,
    master - mean (864ms)  : 834, 895
     .   : milestone, 864,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6654) - mean (262ms)  : 258, 267
     .   : milestone, 262,
    master - mean (261ms)  : 257, 266
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (6654) - mean (846ms)  : 804, 887
     .   : milestone, 846,
    master - mean (845ms)  : 814, 877
     .   : milestone, 845,

datadog-ddstaging · 2025-02-11T14:46:33Z

Datadog Report

Branch report: steven/w3c-extraction-bug
Commit report: dafd3c8
Test service: dd-trace-dotnet

✅ 0 Failed, 241699 Passed, 2379 Skipped, 18h 25m 46.66s Total Time

andrewlock · 2025-02-11T15:28:28Z

Benchmarks Report for tracer 🐌

Benchmarks for #6654 compared to master:

1 benchmarks are faster, with geometric mean 1.189
2 benchmarks are slower, with geometric mean 1.146
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.86μs	37.4ns	191ns	0.0155	0.00777	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10μs	53.3ns	277ns	0.0145	0.00483	0	5.8 KB
master	`StartStopWithChild`	net472	15.8μs	35.3ns	132ns	1.03	0.293	0.0952	6.21 KB
#6654	`StartStopWithChild`	net6.0	8.04μs	41ns	201ns	0.0164	0.00818	0	5.61 KB
#6654	`StartStopWithChild`	netcoreapp3.1	9.95μs	54.7ns	310ns	0.0237	0.0142	0.00474	5.8 KB
#6654	`StartStopWithChild`	net472	16.1μs	51.5ns	200ns	1.02	0.29	0.0806	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	519μs	350ns	1.36μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	653μs	607ns	2.27μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	851μs	292ns	1.09μs	0.422	3.3 KB
#6654	`WriteAndFlushEnrichedTraces`	net6.0	483μs	326ns	1.26μs	0	2.7 KB
#6654	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	697μs	509ns	1.97μs	0	2.7 KB
#6654	`WriteAndFlushEnrichedTraces`	net472	841μs	883ns	3.3μs	0.422	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	131μs	529ns	2.05μs	0.132	14.47 KB
master	`SendRequest`	netcoreapp3.1	150μs	238ns	920ns	0.226	17.27 KB
master	`SendRequest`	net472	0.00515ns	0.00186ns	0.00721ns	0	0 b
#6654	`SendRequest`	net6.0	128μs	532ns	2.06μs	0.131	14.47 KB
#6654	`SendRequest`	netcoreapp3.1	143μs	378ns	1.41μs	0.209	17.27 KB
#6654	`SendRequest`	net472	0.00252ns	0.000857ns	0.00332ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	591μs	3.09μs	16μs	0.631	0	0	41.46 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	658μs	3.12μs	11.7μs	0.355	0	0	41.75 KB
master	`WriteAndFlushEnrichedTraces`	net472	851μs	3.13μs	12.1μs	8.81	2.52	0.419	53.32 KB
#6654	`WriteAndFlushEnrichedTraces`	net6.0	583μs	3.41μs	30.7μs	0.628	0	0	41.62 KB
#6654	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	659μs	3.51μs	21.4μs	0.331	0	0	41.62 KB
#6654	`WriteAndFlushEnrichedTraces`	net472	849μs	3.26μs	12.6μs	8.28	2.48	0.414	53.33 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.33μs	2.62ns	10.1ns	0.0146	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.73μs	3.43ns	13.3ns	0.0129	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.15μs	2.1ns	7.84ns	0.156	0.00107	987 B
#6654	`ExecuteNonQuery`	net6.0	1.28μs	0.968ns	3.75ns	0.0142	0	1.02 KB
#6654	`ExecuteNonQuery`	netcoreapp3.1	1.75μs	1.49ns	5.76ns	0.014	0	1.02 KB
#6654	`ExecuteNonQuery`	net472	2.08μs	2.84ns	11ns	0.157	0.00105	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.28μs	0.527ns	2.04ns	0.0135	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.58μs	0.729ns	2.73ns	0.0128	976 B
master	`CallElasticsearch`	net472	2.66μs	1.78ns	6.89ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.22μs	0.825ns	3.2ns	0.0134	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.65μs	2.08ns	8.06ns	0.0138	1.02 KB
master	`CallElasticsearchAsync`	net472	2.64μs	1.2ns	4.48ns	0.166	1.05 KB
#6654	`CallElasticsearch`	net6.0	1.29μs	0.337ns	1.26ns	0.0135	976 B
#6654	`CallElasticsearch`	netcoreapp3.1	1.63μs	0.631ns	2.36ns	0.013	976 B
#6654	`CallElasticsearch`	net472	2.64μs	4.3ns	16.6ns	0.157	995 B
#6654	`CallElasticsearchAsync`	net6.0	1.32μs	1.06ns	3.96ns	0.0133	952 B
#6654	`CallElasticsearchAsync`	netcoreapp3.1	1.62μs	1.54ns	5.96ns	0.0138	1.02 KB
#6654	`CallElasticsearchAsync`	net472	2.63μs	1.38ns	5.33ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.3μs	0.677ns	2.62ns	0.013	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.64μs	0.678ns	2.63ns	0.0123	952 B
master	`ExecuteAsync`	net472	1.87μs	0.755ns	2.93ns	0.145	915 B
#6654	`ExecuteAsync`	net6.0	1.34μs	0.876ns	3.28ns	0.0133	952 B
#6654	`ExecuteAsync`	netcoreapp3.1	1.63μs	1.96ns	7.6ns	0.0132	952 B
#6654	`ExecuteAsync`	net472	1.84μs	0.724ns	2.8ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.51μs	2.1ns	7.85ns	0.0312	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.35μs	1.63ns	6.33ns	0.0374	2.85 KB
master	`SendAsync`	net472	7.46μs	2.18ns	8.46ns	0.493	3.12 KB
#6654	`SendAsync`	net6.0	4.36μs	2.34ns	8.74ns	0.0327	2.31 KB
#6654	`SendAsync`	netcoreapp3.1	5.34μs	1.39ns	5.19ns	0.0374	2.85 KB
#6654	`SendAsync`	net472	7.41μs	1.45ns	5.43ns	0.495	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.51μs	0.417ns	1.56ns	0.0227	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.25μs	0.478ns	1.66ns	0.0227	1.64 KB
master	`EnrichedLog`	net472	2.54μs	1.02ns	3.97ns	0.249	1.57 KB
#6654	`EnrichedLog`	net6.0	1.46μs	0.862ns	3.23ns	0.0228	1.64 KB
#6654	`EnrichedLog`	netcoreapp3.1	2.16μs	1.8ns	6.95ns	0.0214	1.64 KB
#6654	`EnrichedLog`	net472	2.64μs	0.833ns	3.12ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	111μs	130ns	487ns	0.0559	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	116μs	191ns	741ns	0	0	4.28 KB
master	`EnrichedLog`	net472	151μs	84.5ns	327ns	0.674	0.225	4.46 KB
#6654	`EnrichedLog`	net6.0	112μs	109ns	406ns	0.0561	0	4.28 KB
#6654	`EnrichedLog`	netcoreapp3.1	116μs	158ns	613ns	0.0581	0	4.28 KB
#6654	`EnrichedLog`	net472	150μs	50.4ns	195ns	0.676	0.225	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3μs	1.37ns	5.32ns	0.0299	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.28μs	2.96ns	11.5ns	0.0297	2.2 KB
master	`EnrichedLog`	net472	4.97μs	1.79ns	6.92ns	0.32	2.02 KB
#6654	`EnrichedLog`	net6.0	3.16μs	2.22ns	8.59ns	0.0302	2.2 KB
#6654	`EnrichedLog`	netcoreapp3.1	4.03μs	1.87ns	6.98ns	0.0282	2.2 KB
#6654	`EnrichedLog`	net472	4.98μs	0.93ns	3.48ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.29μs	1.22ns	4.73ns	0.0162	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.71μs	2.64ns	10.2ns	0.0155	1.14 KB
master	`SendReceive`	net472	1.98μs	1.68ns	6.49ns	0.183	1.16 KB
#6654	`SendReceive`	net6.0	1.37μs	1.23ns	4.75ns	0.0159	1.14 KB
#6654	`SendReceive`	netcoreapp3.1	1.76μs	1.26ns	4.9ns	0.0151	1.14 KB
#6654	`SendReceive`	net472	2.06μs	1.36ns	5.26ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.76μs	1.18ns	4.55ns	0.0221	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.91μs	3.11ns	12.1ns	0.0215	1.65 KB
master	`EnrichedLog`	net472	4.39μs	1.99ns	6.89ns	0.322	2.04 KB
#6654	`EnrichedLog`	net6.0	2.72μs	1.43ns	5.54ns	0.0217	1.6 KB
#6654	`EnrichedLog`	netcoreapp3.1	3.86μs	0.708ns	2.65ns	0.0232	1.65 KB
#6654	`EnrichedLog`	net472	4.4μs	2.63ns	10.2ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6654

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.156	670.15	775.02
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1	1.135	547.10	620.82

Faster 🎉 in #6654

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0	1.189	575.79	484.27

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	482ns	0.868ns	3.36ns	0.00798	576 B
master	`StartFinishSpan`	netcoreapp3.1	547ns	0.762ns	2.85ns	0.00786	576 B
master	`StartFinishSpan`	net472	630ns	1.17ns	4.54ns	0.0918	578 B
master	`StartFinishScope`	net6.0	575ns	0.882ns	3.42ns	0.00966	696 B
master	`StartFinishScope`	netcoreapp3.1	669ns	0.458ns	1.71ns	0.0093	696 B
master	`StartFinishScope`	net472	805ns	1.48ns	5.75ns	0.105	658 B
#6654	`StartFinishSpan`	net6.0	461ns	0.508ns	1.97ns	0.00804	576 B
#6654	`StartFinishSpan`	netcoreapp3.1	620ns	0.631ns	2.36ns	0.00771	576 B
#6654	`StartFinishSpan`	net472	577ns	0.822ns	3.07ns	0.0916	578 B
#6654	`StartFinishScope`	net6.0	483ns	0.886ns	3.43ns	0.0099	696 B
#6654	`StartFinishScope`	netcoreapp3.1	773ns	1.87ns	7.23ns	0.00952	696 B
#6654	`StartFinishScope`	net472	820ns	1.35ns	5.23ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	650ns	0.737ns	2.85ns	0.00982	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	984ns	1.68ns	6.49ns	0.00924	696 B
master	`RunOnMethodBegin`	net472	1.01μs	2.88ns	11.2ns	0.105	658 B
#6654	`RunOnMethodBegin`	net6.0	672ns	0.733ns	2.84ns	0.00966	696 B
#6654	`RunOnMethodBegin`	netcoreapp3.1	919ns	1.09ns	4.22ns	0.00919	696 B
#6654	`RunOnMethodBegin`	net472	1.05μs	1.3ns	5.05ns	0.104	658 B

lucaspimentel · 2025-02-21T18:52:06Z

tracer/src/Datadog.Trace/Propagators/W3CTraceContextPropagator.cs

            }

-            // there were no items
-            return false;


It took me a while to figure out the underlying bug. In short, this last line should have been

return hasValue;

instead of

return false;

otherwise there's never any way for this method to return true!

Fix W3CTraceContextPropagator for non-standard array types

dafd3c8

bouwkast requested a review from a team as a code owner February 11, 2025 14:01

andrewlock approved these changes Feb 11, 2025

View reviewed changes

bouwkast merged commit b8967f9 into master Feb 21, 2025
138 of 141 checks passed

bouwkast deleted the steven/w3c-extraction-bug branch February 21, 2025 14:02

github-actions bot added this to the vNext-v3 milestone Feb 21, 2025

lucaspimentel reviewed Feb 21, 2025

View reviewed changes

andrewlock added type:bug area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) identified-by:customer labels Feb 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix W3CTraceContextPropagator for non-standard array types #6654

Fix W3CTraceContextPropagator for non-standard array types #6654

bouwkast commented Feb 11, 2025

andrewlock left a comment

andrewlock commented Feb 11, 2025

datadog-ddstaging bot commented Feb 11, 2025 •

edited

Loading

andrewlock commented Feb 11, 2025

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6654

Faster 🎉 in #6654

Raw results

Raw results

lucaspimentel Feb 21, 2025

Fix W3CTraceContextPropagator for non-standard array types #6654

Fix W3CTraceContextPropagator for non-standard array types #6654

Conversation

bouwkast commented Feb 11, 2025

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

andrewlock left a comment

Choose a reason for hiding this comment

andrewlock commented Feb 11, 2025

Execution-Time Benchmarks Report ⏱️

datadog-ddstaging bot commented Feb 11, 2025 • edited Loading

Datadog Report

andrewlock commented Feb 11, 2025

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6654

Faster 🎉 in #6654

Raw results

Raw results

lucaspimentel Feb 21, 2025

Choose a reason for hiding this comment

datadog-ddstaging bot commented Feb 11, 2025 •

edited

Loading