Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add test to check absence of client computed stats #3812

Merged
merged 5 commits into from
Feb 6, 2025
Merged

Add test to check absence of client computed stats #3812

merged 5 commits into from
Feb 6, 2025

Conversation

estringana
Copy link
Contributor

@estringana estringana commented Jan 14, 2025 •
edited
Loading

Motivation

The added test is a regression test which caused a incident on PHP. When ASM Standalone was implemented, it was required to send the header Datadog-Client-Computed-Stats as yes when it was a ASM Standalone request. However the way it was implemented in PHP was to add the header Datadog-Client-Computed-Stats always. Then the value of the header was yes when it was a ASM Standalone request and no when not. This was generating the agent to never compute stats as it was interpreting the value no as if it was yes.

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • If PR title starts with [<language>], double-check that only <language> is impacted by the change
  • No system-tests internal is modified. Otherwise, I have the approval from R&P team
  • CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added (or removed)?

@estringana estringana force-pushed the estringana/test-asm-standalone-not-enabled branch 8 times, most recently from ebd5622 to ef1e175 Compare January 15, 2025 10:56
jandro996
jandro996 reviewed Jan 16, 2025
View reviewed changes
tests/appsec/test_asm_standalone.py
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if there is a current test in another scenario that checks this datadog-client-computed-stats header, but IMHO we need one more test case that checks datadog-client-computed-stats=true when standalone billing is disabled and metrics enabled, to ensure the previous is working as before the asm feature was implemented.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if I'm following you correctly. How would you make APM to send the header datadog-client-computed-stats=true? In PHP that header was not sent

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's something that only happens in java, but the header is set with empty value, but to keep it simple we can continue with the PR as it is

@estringana estringana force-pushed the estringana/test-asm-standalone-not-enabled branch from 06e9ddb to 39a1b00 Compare January 17, 2025 15:09
@estringana estringana marked this pull request as ready for review January 17, 2025 16:36
@estringana estringana requested review from a team as code owners January 17, 2025 16:36
@estringana estringana force-pushed the estringana/test-asm-standalone-not-enabled branch from 4dacb21 to 9d85bd9 Compare January 22, 2025 17:01
cbeauchesne
cbeauchesne requested changes Jan 23, 2025
View reviewed changes
Copy link
Collaborator

@cbeauchesne cbeauchesne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add new scenario in the CI

@estringana estringana force-pushed the estringana/test-asm-standalone-not-enabled branch 2 times, most recently from 3500a27 to e954ad2 Compare January 24, 2025 15:53
@estringana estringana force-pushed the estringana/test-asm-standalone-not-enabled branch from a76ca84 to edb3d81 Compare February 5, 2025 15:32
cbeauchesne
cbeauchesne approved these changes Feb 5, 2025
View reviewed changes
Copy link
Collaborator

@cbeauchesne cbeauchesne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failures are not related

@estringana estringana merged commit 5216e27 into main Feb 6, 2025
433 of 436 checks passed
@estringana estringana deleted the estringana/test-asm-standalone-not-enabled branch February 6, 2025 09:54
@iunanua iunanua mentioned this pull request Feb 7, 2025
Merged
7 tasks
@cbeauchesne cbeauchesne mentioned this pull request Feb 10, 2025
Merged
7 tasks
sezen-datadog added a commit that referenced this pull request Feb 10, 2025
@sezen-datadog
Extend vulnerability location data with class
41aa93f 
re-enable aws integrations tests (#3733)

feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961)

Activate ruff rules on tests/ folder (#3999)

[python] Avoid passing global tracer to pin in weblog apps (#4004)

All classes must declare feature ids (#4003)

Extend mypy scope (#4002)

Onboarding: bug marker profiling (#4005)

Docker SSI: fix scenario (#4006)

[ruby] Enable IP blocking tests for Ruby (#3937)

Activate ruff rules on tests/ folder (#4007)

[nodejs] remove auto login event skip (#3998)

Email HTML Injection detection in IAST Java (#3906)

Co-authored-by: Mario Vidal Domínguez <[email protected]>

Add test to check absence of client computed stats (#3812)

[java] Skip payara/CROSSED_TRACING_LIBRARIES/prod (#4009)

Add GraphQL error extension tests (#3986)

Co-authored-by: William Conti <[email protected]>

Use prod agent, dev agent broke lot  of tracers (#4011)

update xpassing baggage tests for unimplemented languages (#3773)

[NodeJS] skip failing baggage tests (#4015)

[python] fix 500 errors in sql queries (#3997)

Activate ruff rules on tests/ folder (#4010)

Hotfix

Fix fuzzer

[Nodejs] Enable untrusted deserialization stack trace test for Node.js (#3995)

[python] use main again for dev branch (#4008)

Co-authored-by: erikayasuda <[email protected]>
Co-authored-by: Charles de Beauchesne <[email protected]>

Revert agent dev fix (#4013)

[PHP] Enable rasp telemetry tests for PHP (#3972)

[skipci] Update CODEOWNERS for static files (#4012)

[Java] Enable more easy wins (#4018)

[java] Bump GraalVM system test to JDK 22 (#4001)

[NodeJS] skip more failing baggage tests (#4021)

[Debugger] Update dotnet Exception Replay tests (#3974)

Test multiple rasp during one request (#3989)

Add test for location extended data (#3978)

Fix APPSEC_NO_STATS scenario name (#4019)

Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029)

[java] Enable Test_Blocking_strip_response_headers in some variants (#4033)

[java] Remove some outdated manifest entries (#4039)

[java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038)

Consolidate remote config tests into same directory/file (#4031)

[python] use last patch version of python for django weblogs (#4025)

crashtracking: assert si_signo is set to 11 (#4023)
sezen-datadog added a commit that referenced this pull request Feb 10, 2025
@sezen-datadog
# This is the 1st commit message:
d2064eb 
Extend vulnerability location data with class

re-enable aws integrations tests (#3733)

feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961)

Activate ruff rules on tests/ folder (#3999)

[python] Avoid passing global tracer to pin in weblog apps (#4004)

All classes must declare feature ids (#4003)

Extend mypy scope (#4002)

Onboarding: bug marker profiling (#4005)

Docker SSI: fix scenario (#4006)

[ruby] Enable IP blocking tests for Ruby (#3937)

Activate ruff rules on tests/ folder (#4007)

[nodejs] remove auto login event skip (#3998)

Email HTML Injection detection in IAST Java (#3906)

Co-authored-by: Mario Vidal Domínguez <[email protected]>

Add test to check absence of client computed stats (#3812)

[java] Skip payara/CROSSED_TRACING_LIBRARIES/prod (#4009)

Add GraphQL error extension tests (#3986)

Co-authored-by: William Conti <[email protected]>

Use prod agent, dev agent broke lot  of tracers (#4011)

update xpassing baggage tests for unimplemented languages (#3773)

[NodeJS] skip failing baggage tests (#4015)

[python] fix 500 errors in sql queries (#3997)

Activate ruff rules on tests/ folder (#4010)

Hotfix

Fix fuzzer

[Nodejs] Enable untrusted deserialization stack trace test for Node.js (#3995)

[python] use main again for dev branch (#4008)

Co-authored-by: erikayasuda <[email protected]>
Co-authored-by: Charles de Beauchesne <[email protected]>

Revert agent dev fix (#4013)

[PHP] Enable rasp telemetry tests for PHP (#3972)

[skipci] Update CODEOWNERS for static files (#4012)

[Java] Enable more easy wins (#4018)

[java] Bump GraalVM system test to JDK 22 (#4001)

[NodeJS] skip more failing baggage tests (#4021)

[Debugger] Update dotnet Exception Replay tests (#3974)

Test multiple rasp during one request (#3989)

Add test for location extended data (#3978)

Fix APPSEC_NO_STATS scenario name (#4019)

Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029)

[java] Enable Test_Blocking_strip_response_headers in some variants (#4033)

[java] Remove some outdated manifest entries (#4039)

[java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038)

Consolidate remote config tests into same directory/file (#4031)

[python] use last patch version of python for django weblogs (#4025)

crashtracking: assert si_signo is set to 11 (#4023)

# This is the commit message #2:

[nodejs] Fix get_target_branch - "quotes, parens and # (#3952)"" (#3953)

Co-authored-by: Charles de Beauchesne <[email protected]>
# This is the commit message #3:

Fix Standalone Billing values (#3965)

# This is the commit message #4:

Add success to valid WAF telemetry tags (#3966)

# This is the commit message #5:

Ensure tracer release coherence (#3967)

[Nodejs] Code injection telemetry metric (#3959)

Do not use special chars in "redacted" string used in scrubber (#3977)

fix Context Propagation: Extraction feature_id (#3970)

[dotnet] Enable Custom Blocking Response tests (#3971)

[python] update flask base image (#3979)

[ruby] Skip failing test for APMAPI-1141 (#3980)

update weblog sampling_test use sampling rules instead of deprecated envvar (#3984)

[python] enable Python IAST+SSRF tests (#2512)

Signed-off-by: Juanjo Alvarez <[email protected]>

[python] IAST Header source in werkzeug 3.1 (#3991)

[python] switch flaky decorator for bug (#3990)

[python] update versions for major release (#3993)

[test optimization] Update config shape sent as telemetry (#3992)

Add profiling scenarios to Python onboarding tests (#3002)

Update changelog

re-enable aws integrations tests (#3733)

feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961)

[python] Avoid passing global tracer to pin in weblog apps (#4004)

All classes must declare feature ids (#4003)

Extend mypy scope (#4002)

Onboarding: bug marker profiling (#4005)

Docker SSI: fix scenario (#4006)

[ruby] Enable IP blocking tests for Ruby (#3937)

[nodejs] remove auto login event skip (#3998)

[NodeJS] skip failing baggage tests (#4015)

[python] fix 500 errors in sql queries (#3997)

Hotfix

Fix fuzzer

[python] use main again for dev branch (#4008)

Co-authored-by: erikayasuda <[email protected]>
Co-authored-by: Charles de Beauchesne <[email protected]>

Revert agent dev fix (#4013)

[skipci] Update CODEOWNERS for static files (#4012)

[Java] Enable more easy wins (#4018)

[java] Bump GraalVM system test to JDK 22 (#4001)

[NodeJS] skip more failing baggage tests (#4021)

[Debugger] Update dotnet Exception Replay tests (#3974)

Test multiple rasp during one request (#3989)

Add test for location extended data (#3978)

Fix APPSEC_NO_STATS scenario name (#4019)

Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029)

[java] Enable Test_Blocking_strip_response_headers in some variants (#4033)

[java] Remove some outdated manifest entries (#4039)

[java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038)

Consolidate remote config tests into same directory/file (#4031)

[python] use last patch version of python for django weblogs (#4025)

crashtracking: assert si_signo is set to 11 (#4023)

class vs path
sezen-datadog added a commit that referenced this pull request Feb 10, 2025
@sezen-datadog
# This is the 1st commit message:
d4259f6 
Extend vulnerability location data with class

re-enable aws integrations tests (#3733)

feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961)

Activate ruff rules on tests/ folder (#3999)

[python] Avoid passing global tracer to pin in weblog apps (#4004)

All classes must declare feature ids (#4003)

Extend mypy scope (#4002)

Onboarding: bug marker profiling (#4005)

Docker SSI: fix scenario (#4006)

[ruby] Enable IP blocking tests for Ruby (#3937)

Activate ruff rules on tests/ folder (#4007)

[nodejs] remove auto login event skip (#3998)

Email HTML Injection detection in IAST Java (#3906)

Co-authored-by: Mario Vidal Domínguez <[email protected]>

Add test to check absence of client computed stats (#3812)

[java] Skip payara/CROSSED_TRACING_LIBRARIES/prod (#4009)

Add GraphQL error extension tests (#3986)

Co-authored-by: William Conti <[email protected]>

Use prod agent, dev agent broke lot  of tracers (#4011)

update xpassing baggage tests for unimplemented languages (#3773)

[NodeJS] skip failing baggage tests (#4015)

[python] fix 500 errors in sql queries (#3997)

Activate ruff rules on tests/ folder (#4010)

Hotfix

Fix fuzzer

[Nodejs] Enable untrusted deserialization stack trace test for Node.js (#3995)

[python] use main again for dev branch (#4008)

Co-authored-by: erikayasuda <[email protected]>
Co-authored-by: Charles de Beauchesne <[email protected]>

Revert agent dev fix (#4013)

[PHP] Enable rasp telemetry tests for PHP (#3972)

[skipci] Update CODEOWNERS for static files (#4012)

[Java] Enable more easy wins (#4018)

[java] Bump GraalVM system test to JDK 22 (#4001)

[NodeJS] skip more failing baggage tests (#4021)

[Debugger] Update dotnet Exception Replay tests (#3974)

Test multiple rasp during one request (#3989)

Add test for location extended data (#3978)

Fix APPSEC_NO_STATS scenario name (#4019)

Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029)

[java] Enable Test_Blocking_strip_response_headers in some variants (#4033)

[java] Remove some outdated manifest entries (#4039)

[java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038)

Consolidate remote config tests into same directory/file (#4031)

[python] use last patch version of python for django weblogs (#4025)

crashtracking: assert si_signo is set to 11 (#4023)

# This is the commit message #2:

[nodejs] Fix get_target_branch - "quotes, parens and # (#3952)"" (#3953)

Co-authored-by: Charles de Beauchesne <[email protected]>
# This is the commit message #3:

Fix Standalone Billing values (#3965)

# This is the commit message #4:

Add success to valid WAF telemetry tags (#3966)

# This is the commit message #5:

Ensure tracer release coherence (#3967)

[Nodejs] Code injection telemetry metric (#3959)

Do not use special chars in "redacted" string used in scrubber (#3977)

fix Context Propagation: Extraction feature_id (#3970)

[dotnet] Enable Custom Blocking Response tests (#3971)

[python] update flask base image (#3979)

[ruby] Skip failing test for APMAPI-1141 (#3980)

update weblog sampling_test use sampling rules instead of deprecated envvar (#3984)

[python] enable Python IAST+SSRF tests (#2512)

Signed-off-by: Juanjo Alvarez <[email protected]>

[python] IAST Header source in werkzeug 3.1 (#3991)

[python] switch flaky decorator for bug (#3990)

[python] update versions for major release (#3993)

[test optimization] Update config shape sent as telemetry (#3992)

Add profiling scenarios to Python onboarding tests (#3002)

Update changelog

re-enable aws integrations tests (#3733)

feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961)

[python] Avoid passing global tracer to pin in weblog apps (#4004)

All classes must declare feature ids (#4003)

Extend mypy scope (#4002)

Onboarding: bug marker profiling (#4005)

Docker SSI: fix scenario (#4006)

[ruby] Enable IP blocking tests for Ruby (#3937)

[nodejs] remove auto login event skip (#3998)

[NodeJS] skip failing baggage tests (#4015)

[python] fix 500 errors in sql queries (#3997)

Hotfix

Fix fuzzer

[python] use main again for dev branch (#4008)

Co-authored-by: erikayasuda <[email protected]>
Co-authored-by: Charles de Beauchesne <[email protected]>

Revert agent dev fix (#4013)

[skipci] Update CODEOWNERS for static files (#4012)

[Java] Enable more easy wins (#4018)

[java] Bump GraalVM system test to JDK 22 (#4001)

[NodeJS] skip more failing baggage tests (#4021)

[Debugger] Update dotnet Exception Replay tests (#3974)

Test multiple rasp during one request (#3989)

Add test for location extended data (#3978)

Fix APPSEC_NO_STATS scenario name (#4019)

Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029)

[java] Enable Test_Blocking_strip_response_headers in some variants (#4033)

[java] Remove some outdated manifest entries (#4039)

[java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038)

Consolidate remote config tests into same directory/file (#4031)

[python] use last patch version of python for django weblogs (#4025)

crashtracking: assert si_signo is set to 11 (#4023)

class vs path
robertomonteromiguel pushed a commit that referenced this pull request Mar 17, 2025
@estringana @robertomonteromiguel
Add test to check absence of client computed stats (#3812)
fc5f7b0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jandro996 jandro996 jandro996 left review comments

@iunanua iunanua iunanua left review comments

@cbeauchesne cbeauchesne cbeauchesne approved these changes

Assignees
No one assigned
Labels
run-appsec-scenarios
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@estringana @jandro996 @cbeauchesne @iunanua