Support instrumentation of repackaged libraries #8153

mcculls · 2025-01-04T15:10:54Z

What Does This Do

Introduces the concept of an advice "shader" that can relocate type references before applying the advice or injecting advice helpers into the library/application class-loader.

This makes it possible to share and re-use advice for libraries which may appear under different namespaces, for example javax.jms vs. jakarta.jms. (Reminder that advice is never loaded directly, instead its bytecode is parsed and patched into the target class.)

Motivation

We can already account for relocated packages in matchers by providing a namespace when creating Instrumenters from an InstrumenterModule, but doing the same for advice bytecode requires using an ASM based remapper.

Additional Notes

This is tested by #8155 which uses it to share the javax JMS instrumentation with the jakarta namespace.

An alternative approach would be to create a complete copy of the InstrumenterModule class, its Instrumenters, and all its advice/helpers at build-time. These classes would be pre-processed to use the relocated namespace - an additional relocation would be required to move these copies to a different instrumenter package to avoid conflicting with the original classes. The pre-processing would need to happen before we create the various instrumenter indices.

Doing the relocation at build-time would require no runtime changes, but would use more space in the final jar. It is also not possible to support ad-hoc relocations when processing types at build-time. This limitation led us to prefer a runtime solution, once we'd checked that the relocation overhead was acceptable (and only applied for relocated advice.)

Note: relocating matchers at runtime by passing in the relevant namespace from the InstrumenterModule to its Instrumenters is also consistent with how OpenTelemetry handles relocated libraries.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: APMAPI-858

dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/AdviceShader.java

pr-commenter · 2025-01-04T16:04:35Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mcculls/support-instrumenting-repackaged-libraries
git_commit_date	1735941408	1736114200
git_commit_sha	`886169b`	`e4d0c6f`
release_version	1.45.0-SNAPSHOT~886169bb0a	1.45.0-SNAPSHOT~e4d0c6fb0e

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1736116618	1736116618
ci_job_id	754897109	754897109
ci_pipeline_id	52192265	52192265
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 2 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.45.0-SNAPSHOT~e4d0c6fb0e, baseline=1.45.0-SNAPSHOT~886169bb0a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.051 s) : 0, 1051122
Total [baseline] (10.382 s) : 0, 10382154
Agent [candidate] (1.055 s) : 0, 1055192
Total [candidate] (10.426 s) : 0, 10426206
section appsec
Agent [baseline] (1.188 s) : 0, 1187818
Total [baseline] (10.731 s) : 0, 10731489
Agent [candidate] (1.194 s) : 0, 1193579
Total [candidate] (10.685 s) : 0, 10685256
section iast
Agent [baseline] (1.176 s) : 0, 1176167
Total [baseline] (10.933 s) : 0, 10932608
Agent [candidate] (1.179 s) : 0, 1179281
Total [candidate] (10.98 s) : 0, 10980062
section profiling
Agent [baseline] (1.271 s) : 0, 1270767
Total [baseline] (10.761 s) : 0, 10760656
Agent [candidate] (1.281 s) : 0, 1280791
Total [candidate] (10.797 s) : 0, 10797085

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.051 s	-
Agent	appsec	1.188 s	136.696 ms (13.0%)
Agent	iast	1.176 s	125.045 ms (11.9%)
Agent	profiling	1.271 s	219.645 ms (20.9%)
Total	tracing	10.382 s	-
Total	appsec	10.731 s	349.335 ms (3.4%)
Total	iast	10.933 s	550.454 ms (5.3%)
Total	profiling	10.761 s	378.502 ms (3.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.055 s	-
Agent	appsec	1.194 s	138.386 ms (13.1%)
Agent	iast	1.179 s	124.088 ms (11.8%)
Agent	profiling	1.281 s	225.599 ms (21.4%)
Total	tracing	10.426 s	-
Total	appsec	10.685 s	259.05 ms (2.5%)
Total	iast	10.98 s	553.855 ms (5.3%)
Total	profiling	10.797 s	370.879 ms (3.6%)

gantt
    title petclinic - break down per module: candidate=1.45.0-SNAPSHOT~e4d0c6fb0e, baseline=1.45.0-SNAPSHOT~886169bb0a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (712.345 ms) : 0, 712345
BytebuddyAgent [candidate] (713.098 ms) : 0, 713098
GlobalTracer [baseline] (255.724 ms) : 0, 255724
GlobalTracer [candidate] (255.685 ms) : 0, 255685
AppSec [baseline] (55.05 ms) : 0, 55050
AppSec [candidate] (55.851 ms) : 0, 55851
Remote Config [baseline] (729.781 µs) : 0, 730
Remote Config [candidate] (714.244 µs) : 0, 714
Telemetry [baseline] (12.246 ms) : 0, 12246
Telemetry [candidate] (14.825 ms) : 0, 14825
section appsec
BytebuddyAgent [baseline] (730.294 ms) : 0, 730294
BytebuddyAgent [candidate] (735.346 ms) : 0, 735346
GlobalTracer [baseline] (253.203 ms) : 0, 253203
GlobalTracer [candidate] (253.442 ms) : 0, 253442
AppSec [baseline] (170.858 ms) : 0, 170858
AppSec [candidate] (171.34 ms) : 0, 171340
Remote Config [baseline] (653.749 µs) : 0, 654
Remote Config [candidate] (670.439 µs) : 0, 670
Telemetry [baseline] (8.037 ms) : 0, 8037
Telemetry [candidate] (7.953 ms) : 0, 7953
IAST [baseline] (19.472 ms) : 0, 19472
IAST [candidate] (19.428 ms) : 0, 19428
section iast
BytebuddyAgent [baseline] (826.965 ms) : 0, 826965
BytebuddyAgent [candidate] (829.853 ms) : 0, 829853
GlobalTracer [baseline] (246.234 ms) : 0, 246234
GlobalTracer [candidate] (246.277 ms) : 0, 246277
AppSec [baseline] (57.673 ms) : 0, 57673
AppSec [candidate] (58.095 ms) : 0, 58095
Remote Config [baseline] (646.679 µs) : 0, 647
Remote Config [candidate] (641.45 µs) : 0, 641
Telemetry [baseline] (8.555 ms) : 0, 8555
Telemetry [candidate] (8.504 ms) : 0, 8504
IAST [baseline] (21.22 ms) : 0, 21220
IAST [candidate] (20.903 ms) : 0, 20903
section profiling
ProfilingAgent [baseline] (95.396 ms) : 0, 95396
ProfilingAgent [candidate] (96.236 ms) : 0, 96236
BytebuddyAgent [baseline] (699.906 ms) : 0, 699906
BytebuddyAgent [candidate] (706.472 ms) : 0, 706472
GlobalTracer [baseline] (371.441 ms) : 0, 371441
GlobalTracer [candidate] (373.044 ms) : 0, 373044
AppSec [baseline] (53.555 ms) : 0, 53555
AppSec [candidate] (54.519 ms) : 0, 54519
Remote Config [baseline] (659.214 µs) : 0, 659
Remote Config [candidate] (660.754 µs) : 0, 661
Telemetry [baseline] (7.799 ms) : 0, 7799
Telemetry [candidate] (7.84 ms) : 0, 7840
Profiling [baseline] (95.423 ms) : 0, 95423
Profiling [candidate] (96.264 ms) : 0, 96264

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.45.0-SNAPSHOT~e4d0c6fb0e, baseline=1.45.0-SNAPSHOT~886169bb0a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1053927
Total [baseline] (8.59 s) : 0, 8589948
Agent [candidate] (1.056 s) : 0, 1056006
Total [candidate] (8.588 s) : 0, 8588494
section iast
Agent [baseline] (1.179 s) : 0, 1178957
Total [baseline] (9.2 s) : 0, 9199616
Agent [candidate] (1.189 s) : 0, 1188688
Total [candidate] (9.203 s) : 0, 9203442
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.192 s) : 0, 1191756
Total [baseline] (9.25 s) : 0, 9250041
Agent [candidate] (1.18 s) : 0, 1180467
Total [candidate] (9.163 s) : 0, 9162864
section iast_TELEMETRY_OFF
Agent [baseline] (1.181 s) : 0, 1180678
Total [baseline] (9.157 s) : 0, 9156825
Agent [candidate] (1.173 s) : 0, 1173188
Total [candidate] (9.172 s) : 0, 9172056

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.054 s	-
Agent	iast	1.179 s	125.03 ms (11.9%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.192 s	137.829 ms (13.1%)
Agent	iast_TELEMETRY_OFF	1.181 s	126.752 ms (12.0%)
Total	tracing	8.59 s	-
Total	iast	9.2 s	609.667 ms (7.1%)
Total	iast_HARDCODED_SECRET_DISABLED	9.25 s	660.093 ms (7.7%)
Total	iast_TELEMETRY_OFF	9.157 s	566.876 ms (6.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	iast	1.189 s	132.683 ms (12.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.18 s	124.462 ms (11.8%)
Agent	iast_TELEMETRY_OFF	1.173 s	117.182 ms (11.1%)
Total	tracing	8.588 s	-
Total	iast	9.203 s	614.948 ms (7.2%)
Total	iast_HARDCODED_SECRET_DISABLED	9.163 s	574.37 ms (6.7%)
Total	iast_TELEMETRY_OFF	9.172 s	583.562 ms (6.8%)

gantt
    title insecure-bank - break down per module: candidate=1.45.0-SNAPSHOT~e4d0c6fb0e, baseline=1.45.0-SNAPSHOT~886169bb0a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.091 ms) : 0, 716091
BytebuddyAgent [candidate] (714.529 ms) : 0, 714529
GlobalTracer [baseline] (255.562 ms) : 0, 255562
GlobalTracer [candidate] (256.296 ms) : 0, 256296
AppSec [baseline] (55.025 ms) : 0, 55025
AppSec [candidate] (55.2 ms) : 0, 55200
Remote Config [baseline] (731.558 µs) : 0, 732
Remote Config [candidate] (719.101 µs) : 0, 719
Telemetry [baseline] (11.386 ms) : 0, 11386
Telemetry [candidate] (14.239 ms) : 0, 14239
section iast
BytebuddyAgent [baseline] (829.203 ms) : 0, 829203
BytebuddyAgent [candidate] (837.242 ms) : 0, 837242
GlobalTracer [baseline] (246.263 ms) : 0, 246263
GlobalTracer [candidate] (247.594 ms) : 0, 247594
AppSec [baseline] (57.847 ms) : 0, 57847
AppSec [candidate] (58.139 ms) : 0, 58139
IAST [baseline] (21.484 ms) : 0, 21484
IAST [candidate] (21.405 ms) : 0, 21405
Remote Config [baseline] (660.995 µs) : 0, 661
Remote Config [candidate] (651.394 µs) : 0, 651
Telemetry [baseline] (8.548 ms) : 0, 8548
Telemetry [candidate] (8.562 ms) : 0, 8562
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (838.649 ms) : 0, 838649
BytebuddyAgent [candidate] (829.744 ms) : 0, 829744
GlobalTracer [baseline] (248.593 ms) : 0, 248593
GlobalTracer [candidate] (246.194 ms) : 0, 246194
AppSec [baseline] (58.347 ms) : 0, 58347
AppSec [candidate] (58.739 ms) : 0, 58739
IAST [baseline] (21.732 ms) : 0, 21732
IAST [candidate] (21.528 ms) : 0, 21528
Remote Config [baseline] (670.613 µs) : 0, 671
Remote Config [candidate] (678.887 µs) : 0, 679
Telemetry [baseline] (8.597 ms) : 0, 8597
Telemetry [candidate] (8.565 ms) : 0, 8565
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (830.833 ms) : 0, 830833
BytebuddyAgent [candidate] (825.571 ms) : 0, 825571
GlobalTracer [baseline] (247.229 ms) : 0, 247229
GlobalTracer [candidate] (245.623 ms) : 0, 245623
AppSec [baseline] (57.675 ms) : 0, 57675
AppSec [candidate] (57.304 ms) : 0, 57304
IAST [baseline] (20.836 ms) : 0, 20836
IAST [candidate] (20.724 ms) : 0, 20724
Remote Config [baseline] (659.066 µs) : 0, 659
Remote Config [candidate] (641.073 µs) : 0, 641
Telemetry [baseline] (8.406 ms) : 0, 8406
Telemetry [candidate] (8.336 ms) : 0, 8336

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-01-05T22:07:23	2025-01-05T22:14:22
git_branch	master	mcculls/support-instrumenting-repackaged-libraries
git_commit_date	1735941408	1736114200
git_commit_sha	`886169b`	`e4d0c6f`
release_version	1.45.0-SNAPSHOT~886169bb0a	1.45.0-SNAPSHOT~e4d0c6fb0e
start_time	2025-01-05T22:07:09	2025-01-05T22:14:08

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1736115617	1736115617
ci_job_id	754897110	754897110
ci_pipeline_id	52192265	52192265
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~e4d0c6fb0e, baseline=1.45.0-SNAPSHOT~886169bb0a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.358 ms) : 1338, 1378
.   : milestone, 1358,
appsec (1.773 ms) : 1750, 1796
.   : milestone, 1773,
appsec_no_iast (1.734 ms) : 1711, 1757
.   : milestone, 1734,
iast (1.501 ms) : 1478, 1523
.   : milestone, 1501,
profiling (1.498 ms) : 1475, 1520
.   : milestone, 1498,
tracing (1.496 ms) : 1472, 1520
.   : milestone, 1496,
section candidate
no_agent (1.356 ms) : 1337, 1375
.   : milestone, 1356,
appsec (1.745 ms) : 1721, 1769
.   : milestone, 1745,
appsec_no_iast (1.752 ms) : 1729, 1775
.   : milestone, 1752,
iast (1.514 ms) : 1491, 1537
.   : milestone, 1514,
profiling (1.509 ms) : 1486, 1531
.   : milestone, 1509,
tracing (1.481 ms) : 1457, 1505
.   : milestone, 1481,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.358 ms [1.338 ms, 1.378 ms]	-
appsec	1.773 ms [1.75 ms, 1.796 ms]	415.161 µs (30.6%)
appsec_no_iast	1.734 ms [1.711 ms, 1.757 ms]	376.318 µs (27.7%)
iast	1.501 ms [1.478 ms, 1.523 ms]	142.616 µs (10.5%)
profiling	1.498 ms [1.475 ms, 1.52 ms]	139.738 µs (10.3%)
tracing	1.496 ms [1.472 ms, 1.52 ms]	138.151 µs (10.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.356 ms [1.337 ms, 1.375 ms]	-
appsec	1.745 ms [1.721 ms, 1.769 ms]	389.131 µs (28.7%)
appsec_no_iast	1.752 ms [1.729 ms, 1.775 ms]	395.978 µs (29.2%)
iast	1.514 ms [1.491 ms, 1.537 ms]	157.992 µs (11.7%)
profiling	1.509 ms [1.486 ms, 1.531 ms]	152.652 µs (11.3%)
tracing	1.481 ms [1.457 ms, 1.505 ms]	124.945 µs (9.2%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~e4d0c6fb0e, baseline=1.45.0-SNAPSHOT~886169bb0a
    dateFormat X
    axisFormat %s
section baseline
no_agent (386.533 µs) : 366, 407
.   : milestone, 387,
iast (497.397 µs) : 476, 519
.   : milestone, 497,
iast_FULL (666.1 µs) : 645, 688
.   : milestone, 666,
iast_GLOBAL (522.319 µs) : 499, 545
.   : milestone, 522,
iast_HARDCODED_SECRET_DISABLED (498.733 µs) : 477, 520
.   : milestone, 499,
iast_INACTIVE (455.209 µs) : 434, 476
.   : milestone, 455,
iast_TELEMETRY_OFF (482.385 µs) : 461, 504
.   : milestone, 482,
tracing (449.469 µs) : 429, 470
.   : milestone, 449,
section candidate
no_agent (375.605 µs) : 356, 395
.   : milestone, 376,
iast (495.698 µs) : 474, 517
.   : milestone, 496,
iast_FULL (662.682 µs) : 641, 684
.   : milestone, 663,
iast_GLOBAL (527.683 µs) : 505, 550
.   : milestone, 528,
iast_HARDCODED_SECRET_DISABLED (490.342 µs) : 469, 512
.   : milestone, 490,
iast_INACTIVE (458.981 µs) : 438, 480
.   : milestone, 459,
iast_TELEMETRY_OFF (487.984 µs) : 466, 510
.   : milestone, 488,
tracing (450.62 µs) : 430, 472
.   : milestone, 451,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	386.533 µs [366.403 µs, 406.663 µs]	-
iast	497.397 µs [476.07 µs, 518.725 µs]	110.864 µs (28.7%)
iast_FULL	666.1 µs [644.545 µs, 687.655 µs]	279.567 µs (72.3%)
iast_GLOBAL	522.319 µs [499.312 µs, 545.327 µs]	135.786 µs (35.1%)
iast_HARDCODED_SECRET_DISABLED	498.733 µs [477.07 µs, 520.395 µs]	112.2 µs (29.0%)
iast_INACTIVE	455.209 µs [433.98 µs, 476.437 µs]	68.676 µs (17.8%)
iast_TELEMETRY_OFF	482.385 µs [460.887 µs, 503.882 µs]	95.852 µs (24.8%)
tracing	449.469 µs [428.652 µs, 470.286 µs]	62.936 µs (16.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	375.605 µs [356.038 µs, 395.173 µs]	-
iast	495.698 µs [474.101 µs, 517.295 µs]	120.093 µs (32.0%)
iast_FULL	662.682 µs [640.974 µs, 684.391 µs]	287.077 µs (76.4%)
iast_GLOBAL	527.683 µs [505.433 µs, 549.932 µs]	152.077 µs (40.5%)
iast_HARDCODED_SECRET_DISABLED	490.342 µs [468.776 µs, 511.908 µs]	114.737 µs (30.5%)
iast_INACTIVE	458.981 µs [437.798 µs, 480.164 µs]	83.376 µs (22.2%)
iast_TELEMETRY_OFF	487.984 µs [466.271 µs, 509.698 µs]	112.379 µs (29.9%)
tracing	450.62 µs [429.511 µs, 471.729 µs]	75.015 µs (20.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mcculls/support-instrumenting-repackaged-libraries
git_commit_date	1735941408	1736114200
git_commit_sha	`886169b`	`e4d0c6f`
release_version	1.45.0-SNAPSHOT~886169bb0a	1.45.0-SNAPSHOT~e4d0c6fb0e

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1736116187	1736116187
ci_job_id	754897111	754897111
ci_pipeline_id	52192265	52192265
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~e4d0c6fb0e, baseline=1.45.0-SNAPSHOT~886169bb0a
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.143 s) : 15143000, 15143000
.   : milestone, 15143000,
appsec (15.294 s) : 15294000, 15294000
.   : milestone, 15294000,
iast (19.027 s) : 19027000, 19027000
.   : milestone, 19027000,
iast_GLOBAL (18.132 s) : 18132000, 18132000
.   : milestone, 18132000,
profiling (15.191 s) : 15191000, 15191000
.   : milestone, 15191000,
tracing (15.049 s) : 15049000, 15049000
.   : milestone, 15049000,
section candidate
no_agent (15.427 s) : 15427000, 15427000
.   : milestone, 15427000,
appsec (14.91 s) : 14910000, 14910000
.   : milestone, 14910000,
iast (18.69 s) : 18690000, 18690000
.   : milestone, 18690000,
iast_GLOBAL (17.716 s) : 17716000, 17716000
.   : milestone, 17716000,
profiling (15.647 s) : 15647000, 15647000
.   : milestone, 15647000,
tracing (14.994 s) : 14994000, 14994000
.   : milestone, 14994000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.143 s [15.143 s, 15.143 s]	-
appsec	15.294 s [15.294 s, 15.294 s]	151.0 ms (1.0%)
iast	19.027 s [19.027 s, 19.027 s]	3.884 s (25.6%)
iast_GLOBAL	18.132 s [18.132 s, 18.132 s]	2.989 s (19.7%)
profiling	15.191 s [15.191 s, 15.191 s]	48.0 ms (0.3%)
tracing	15.049 s [15.049 s, 15.049 s]	-94.0 ms (-0.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.427 s [15.427 s, 15.427 s]	-
appsec	14.91 s [14.91 s, 14.91 s]	-517.0 ms (-3.4%)
iast	18.69 s [18.69 s, 18.69 s]	3.263 s (21.2%)
iast_GLOBAL	17.716 s [17.716 s, 17.716 s]	2.289 s (14.8%)
profiling	15.647 s [15.647 s, 15.647 s]	220.0 ms (1.4%)
tracing	14.994 s [14.994 s, 14.994 s]	-433.0 ms (-2.8%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~e4d0c6fb0e, baseline=1.45.0-SNAPSHOT~886169bb0a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.479 ms) : 1467, 1491
.   : milestone, 1479,
appsec (2.367 ms) : 2325, 2409
.   : milestone, 2367,
iast (2.1 ms) : 2046, 2153
.   : milestone, 2100,
iast_GLOBAL (2.141 ms) : 2087, 2194
.   : milestone, 2141,
profiling (1.956 ms) : 1914, 1998
.   : milestone, 1956,
tracing (1.946 ms) : 1905, 1987
.   : milestone, 1946,
section candidate
no_agent (1.477 ms) : 1466, 1489
.   : milestone, 1477,
appsec (2.358 ms) : 2316, 2400
.   : milestone, 2358,
iast (2.085 ms) : 2032, 2138
.   : milestone, 2085,
iast_GLOBAL (2.158 ms) : 2104, 2212
.   : milestone, 2158,
profiling (1.967 ms) : 1924, 2009
.   : milestone, 1967,
tracing (1.947 ms) : 1906, 1988
.   : milestone, 1947,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.479 ms [1.467 ms, 1.491 ms]	-
appsec	2.367 ms [2.325 ms, 2.409 ms]	888.419 µs (60.1%)
iast	2.1 ms [2.046 ms, 2.153 ms]	620.712 µs (42.0%)
iast_GLOBAL	2.141 ms [2.087 ms, 2.194 ms]	661.747 µs (44.7%)
profiling	1.956 ms [1.914 ms, 1.998 ms]	476.95 µs (32.3%)
tracing	1.946 ms [1.905 ms, 1.987 ms]	467.478 µs (31.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.477 ms [1.466 ms, 1.489 ms]	-
appsec	2.358 ms [2.316 ms, 2.4 ms]	880.868 µs (59.6%)
iast	2.085 ms [2.032 ms, 2.138 ms]	607.756 µs (41.1%)
iast_GLOBAL	2.158 ms [2.104 ms, 2.212 ms]	681.058 µs (46.1%)
profiling	1.967 ms [1.924 ms, 2.009 ms]	489.253 µs (33.1%)
tracing	1.947 ms [1.906 ms, 1.988 ms]	469.586 µs (31.8%)

dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/AdviceShader.java

amarziali · 2025-01-06T08:00:19Z

dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/AdviceShader.java

+
+/** Shades advice bytecode by applying relocations to all references. */
+public final class AdviceShader extends Remapper {
+  private final DDCache<String, String> cache = DDCaches.newFixedSizeCache(64);


Can we just consider 'safe' to leave it unbounded not to forget to raise the limits if needed in a near future?

The general recommendation would be to always use bounded collections.
What's the worst case according to you if the cache is too small? Loosing CPU cycle at startup or having unreclaimed memory? 🤷

FYI this cache was tuned by running it over some sample advice and optimising for both space and reducing duplicate requests - as Bruce says if we ever have advice whose bytecode contains type references that overflow the cache (and repeat over the same references) then the worst case is a small amount of CPU needed for pattern matching (also this cache is only used when the main relocated matcher applies to a class-loader and byte-buddy resolves the advice for transformation)

amarziali

Thanks having this implemented and for the related example. This is a real game changer for reusing code in instrumentations

PerfectSlayer

Looks good 👍
Few question / request for comments along the review.

One general question though with this new piece of code: do we foresee an opportunity for customers to be able to remap matching library package? (thinking of shaded libraries) Or is it still too unstable / unpredictable / not worth it?

PerfectSlayer · 2025-01-06T08:00:43Z

dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/AdviceShader.java

+      if (text.isEmpty()) {
+        return text;
+      } else if (text.indexOf('.') > 0) {
+        return shadeDottedName(text);


Quick question: when do you expect having dotted names?

Needed for #8157 - we want to relocate the context store string, which is in an inner advice class which we cannot use the namespace field, because the advice must remain decoupled from the instrumenter that declares it (because byte-buddy doesn't actually run the advice, it just resolves it and lifts the bytecode out)

The context store key here is a string literal containing a dotted class-name.

(Also the reason the AWS instrumentation uses a string literal for the context store key is so it can access the context-store without needing a compile-time dependency, for reasons that come down to how the AWS client may be deployed)

PerfectSlayer · 2025-01-06T08:03:40Z

dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/InstrumenterModule.java

@@ -156,6 +156,11 @@ public ElementMatcher<? super MethodDescription> methodIgnoreMatcher() {
    return isSynthetic();
  }

+  /** Override this to apply shading to method advice and injected helpers. */


Can we document here what the expected "string"?
Like the from-package / to-package couple, . vs / format, leading . support, etc
Not sure it will be trivial for the instrumentation author 😓

I'll do that in the same PR where I'm updating the instrumentation documentation.

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.api.grpc:proto-google-common-protos](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.50.0` -> `2.50.1` | | [com.google.cloud:google-cloud-core-http](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.49.0` -> `2.49.1` | | [com.google.cloud:google-cloud-core](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.49.0` -> `2.49.1` | | [com.google.api:gax](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.59.0` -> `2.59.1` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.44.1` -> `1.45.0` | | [com.datadoghq:dd-trace-ot](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.44.1` -> `1.45.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | --- ### Release Notes <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.45.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.45.0): 1.45.0 ##### Breaking changes > \[!WARNING]\ > Support for custom scope manager using OpenTelemetry tracer artifact (`dd-trace-ot`) is dropped. > Tracing with OpenTracing API and custom scope manager will continue to work on 1.44.x releases. ##### Components ##### Application Security Management (IAST) - ✨ Add propagation to URI#toURL method ([#8146](DataDog/dd-trace-java#8146) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Increase IAST propagation to StringBuilder setLength ([#8119](DataDog/dd-trace-java#8119) - [@Mariovido](https://github.com/Mariovido)) - ✨ Increase IAST propagation to StringBuffer append ([#8082](DataDog/dd-trace-java#8082) - [@Mariovido](https://github.com/Mariovido)) - ✨ Handle IAST security controls custom validation and sanitization methods ([#7997](DataDog/dd-trace-java#7997) - [@jandro996](https://github.com/jandro996)) ##### Application Security Management (WAF) - ✨ Update user lifecycle tracking to V3 ([#8108](DataDog/dd-trace-java#8108) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Exploit prevention for Shell Injection / Command Injection ([#7615](DataDog/dd-trace-java#7615) - [@jandro996](https://github.com/jandro996)) ##### Build & Tooling - 💡 Support instrumentation of repackaged libraries ([#8153](DataDog/dd-trace-java#8153) - [@mcculls](https://github.com/mcculls)) - ✨ Configure native image build setting for JDK-22 based GraalVM ([#8092](DataDog/dd-trace-java#8092) - [@MattAlp](https://github.com/MattAlp)) ##### Database Monitoring - ✨ Add full APM/DBM mode for Oracle ([#8090](DataDog/dd-trace-java#8090) - [@nenadnoveljic](https://github.com/nenadnoveljic)) ##### Dynamic Instrumentation - 🐛 make local var hoisting disabled by default ([#8158](DataDog/dd-trace-java#8158) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Fix var hoisting issue when no previous store ([#8122](DataDog/dd-trace-java#8122) - [@jpbempel](https://github.com/jpbempel)) - ✨ Only decorate spans without code origin information ([#8105](DataDog/dd-trace-java#8105) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Fix suspend Kotlin methods instrumentation ([#8080](DataDog/dd-trace-java#8080) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Fix class file version detection ([#8057](DataDog/dd-trace-java#8057) - [@jpbempel](https://github.com/jpbempel)) ##### GraalVM native-image - ✨ Configure native image build setting for JDK-22 based GraalVM ([#8092](DataDog/dd-trace-java#8092) - [@MattAlp](https://github.com/MattAlp)) ##### ML Observability (LLMObs) - ✨🧪 Add LLMObs configuration ([#8076](DataDog/dd-trace-java#8076) - [@gary-huang](https://github.com/gary-huang)) ##### Metrics - Bump integrations-core submodule to 7.60.0 ([#8098](DataDog/dd-trace-java#8098) - [@mcculls](https://github.com/mcculls)) - Upgrade to java-dogstatsd-client v4.4.3 ([#8096](DataDog/dd-trace-java#8096) - [@mcculls](https://github.com/mcculls)) ##### OpenTracing - ⚠️🧹 Remove custom scope manager support ([#8164](DataDog/dd-trace-java#8164) - [@PerfectSlayer](https://github.com/PerfectSlayer)) ##### Telemetry - ✨ Retry telemetry requests if CI Visibility is enabled ([#8147](DataDog/dd-trace-java#8147) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add configurable Dependency service resolution period ([#8079](DataDog/dd-trace-java#8079) - [@jandro996](https://github.com/jandro996)) ##### Testing - 🐛 Remove restriction to not run vertx4 latest tests on java 17 ([#8133](DataDog/dd-trace-java#8133) - [@vandonr](https://github.com/vandonr)) ##### Tracer core - ✨ Defer remote components to avoid OkHttp class-loading side-effects ([#8131](DataDog/dd-trace-java#8131) - [@mcculls](https://github.com/mcculls)) - ✨ Improve Context API null handling and Javadoc ([#8129](DataDog/dd-trace-java#8129) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - 🐛⚡ Avoid performing blocking I/O operation on application thread ([#8120](DataDog/dd-trace-java#8120) - [@mcculls](https://github.com/mcculls)) - 💡 Introduce a shared context component, independent of tracing ([#8117](DataDog/dd-trace-java#8117) - [@mcculls](https://github.com/mcculls)) - ✨ Improves ServiceNameCollector ([#8109](DataDog/dd-trace-java#8109) - [@amarziali](https://github.com/amarziali)) - Upgrade to ASM 9.7.1 (adds new constant for Java 24) ([#8097](DataDog/dd-trace-java#8097) - [@mcculls](https://github.com/mcculls)) - 🐛 Dynamically evaluate service name for message consumers ([#8088](DataDog/dd-trace-java#8088) - [@amarziali](https://github.com/amarziali)) ##### Serverless - 🐛 Add avoid double instrumenting lambda non-streaming handlers. ([#8073](DataDog/dd-trace-java#8073) - [@purple4reina](https://github.com/purple4reina)) ##### Instrumentations ##### AWS SDK instrumentation - 💡 Instrument EMR's relocated AWS SDK ([#8157](DataDog/dd-trace-java#8157) - [@mcculls](https://github.com/mcculls)) ##### Eclipse Vert.x instrumentation - 🐛 Remove restriction to not run vertx4 latest tests on java 17 ([#8133](DataDog/dd-trace-java#8133) - [@vandonr](https://github.com/vandonr)) ##### JDBC instrumentation - ✨ Add full APM/DBM mode for Oracle ([#8090](DataDog/dd-trace-java#8090) - [@nenadnoveljic](https://github.com/nenadnoveljic)) ##### Jetty instrumentation - 🐛 Ensure jetty 12 has servlet.path starting with / ([#8093](DataDog/dd-trace-java#8093) - [@github-actions](https://github.com/github-actions)\[bot]) ##### JMS instrumentation - 🧹 Re-use `javax` JMS module for `jakarta` namespace ([#8155](DataDog/dd-trace-java#8155) - [@mcculls](https://github.com/mcculls)) - 🧹 Group `javax.jms` instrumentations under a single module ([#8154](DataDog/dd-trace-java#8154) - [@mcculls](https://github.com/mcculls)) ##### Reactor instrumentation - 🐛 Reactor: early propagate span in context when subscribing ([#8166](DataDog/dd-trace-java#8166) - [@amarziali](https://github.com/amarziali)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: ba2355aa4e2e39ab1fee27319cc4176238efd90b

mcculls added type: feature request comp: tooling Build & Tooling labels Jan 4, 2025

datadog-datadog-prod-us1 bot reviewed Jan 4, 2025

View reviewed changes

dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/AdviceShader.java Outdated Show resolved Hide resolved

mcculls force-pushed the mcculls/support-instrumenting-repackaged-libraries branch from 4e674e3 to 7391ebd Compare January 4, 2025 15:21

mcculls added the tag: no release notes Changes to exclude from release notes label Jan 4, 2025

mcculls marked this pull request as ready for review January 4, 2025 16:33

mcculls requested a review from a team as a code owner January 4, 2025 16:33

mcculls requested review from ygree and amarziali January 4, 2025 16:33

mcculls mentioned this pull request Jan 4, 2025

Group javax.jms instrumentations under a single module #8154

Merged

mcculls removed the tag: no release notes Changes to exclude from release notes label Jan 4, 2025

mcculls mentioned this pull request Jan 4, 2025

Re-use javax JMS module for jakarta namespace #8155

Merged

mcculls force-pushed the mcculls/support-instrumenting-repackaged-libraries branch 2 times, most recently from 11f0727 to c39b811 Compare January 5, 2025 15:56

mcculls added tag: no release notes Changes to exclude from release notes and removed tag: no release notes Changes to exclude from release notes labels Jan 5, 2025

Support instrumentation of repackaged libraries

e4d0c6f

mcculls force-pushed the mcculls/support-instrumenting-repackaged-libraries branch from c39b811 to e4d0c6f Compare January 5, 2025 21:57

datadog-datadog-prod-us1 bot reviewed Jan 5, 2025

View reviewed changes

dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/AdviceShader.java Show resolved Hide resolved

mcculls mentioned this pull request Jan 5, 2025

Instrument EMR's relocated AWS SDK #8157

Merged

mcculls added tag: no release notes Changes to exclude from release notes and removed tag: no release notes Changes to exclude from release notes labels Jan 5, 2025

amarziali reviewed Jan 6, 2025

View reviewed changes

amarziali approved these changes Jan 6, 2025

View reviewed changes

PerfectSlayer approved these changes Jan 6, 2025

View reviewed changes

mcculls merged commit 6bfef97 into master Jan 6, 2025
150 checks passed

mcculls deleted the mcculls/support-instrumenting-repackaged-libraries branch January 6, 2025 08:37

github-actions bot added this to the 1.45.0 milestone Jan 6, 2025

mcculls mentioned this pull request Jan 13, 2025

Ensure shaded helpers have unique names when injected into class-loaders #8192

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support instrumentation of repackaged libraries #8153

Support instrumentation of repackaged libraries #8153

mcculls commented Jan 4, 2025 •

edited

Loading

pr-commenter bot commented Jan 4, 2025 •

edited

Loading

amarziali Jan 6, 2025

PerfectSlayer Jan 6, 2025

mcculls Jan 6, 2025 •

edited

Loading

amarziali left a comment

PerfectSlayer left a comment

PerfectSlayer Jan 6, 2025

mcculls Jan 6, 2025 •

edited

Loading

PerfectSlayer Jan 6, 2025

mcculls Jan 6, 2025

Support instrumentation of repackaged libraries #8153

Support instrumentation of repackaged libraries #8153

Conversation

mcculls commented Jan 4, 2025 • edited Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

pr-commenter bot commented Jan 4, 2025 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

amarziali Jan 6, 2025

Choose a reason for hiding this comment

PerfectSlayer Jan 6, 2025

Choose a reason for hiding this comment

mcculls Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

amarziali left a comment

Choose a reason for hiding this comment

PerfectSlayer left a comment

Choose a reason for hiding this comment

PerfectSlayer Jan 6, 2025

Choose a reason for hiding this comment

mcculls Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

PerfectSlayer Jan 6, 2025

Choose a reason for hiding this comment

mcculls Jan 6, 2025

Choose a reason for hiding this comment

mcculls commented Jan 4, 2025 •

edited

Loading

pr-commenter bot commented Jan 4, 2025 •

edited

Loading

mcculls Jan 6, 2025 •

edited

Loading

mcculls Jan 6, 2025 •

edited

Loading