Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce request per iteration in IAST benchmarks #3260

Merged
merged 3 commits into from
Jun 20, 2023

Conversation

uurien
Copy link
Collaborator

@uurien uurien commented Jun 20, 2023

What does this PR do?

Reduce the request made in the IAST benchmarks

Motivation

Reduce the time that test need to execute to prevent timeouts

@github-actions
Copy link

github-actions bot commented Jun 20, 2023

Overall package size

Self size: 4.36 MB
Deduped: 60.72 MB
No deduping: 60.77 MB

Dependency sizes

name version self size total size
@datadog/pprof 2.2.1 14.24 MB 15.12 MB
@datadog/native-iast-taint-tracking 1.5.0 14.86 MB 14.86 MB
@datadog/native-appsec 3.2.0 13.38 MB 13.39 MB
protobufjs 7.1.2 2.76 MB 6.55 MB
@datadog/native-iast-rewriter 2.0.1 2.09 MB 2.1 MB
@opentelemetry/core 1.3.1 784.66 kB 1.37 MB
@datadog/native-metrics 2.0.0 898.77 kB 1.3 MB
@opentelemetry/api 1.4.1 780.32 kB 780.32 kB
opentracing 0.14.7 194.81 kB 194.81 kB
semver 7.3.8 88.2 kB 118.6 kB
@datadog/sketches-js 2.1.0 109.9 kB 109.9 kB
lodash.sortby 4.7.0 75.76 kB 75.76 kB
lru-cache 7.14.0 74.95 kB 74.95 kB
ipaddr.js 2.0.1 59.52 kB 59.52 kB
ignore 5.2.0 48.87 kB 48.87 kB
import-in-the-middle 1.3.5 34.34 kB 38.81 kB
istanbul-lib-coverage 3.2.0 29.34 kB 29.34 kB
retry 0.10.1 27.44 kB 27.44 kB
lodash.uniq 4.5.0 25.01 kB 25.01 kB
limiter 1.1.5 23.17 kB 23.17 kB
lodash.kebabcase 4.1.1 17.75 kB 17.75 kB
lodash.pick 4.4.0 16.33 kB 16.33 kB
node-abort-controller 3.0.1 14.33 kB 14.33 kB
crypto-randomuuid 1.0.0 11.18 kB 11.18 kB
diagnostics_channel 1.1.0 7.07 kB 7.07 kB
path-to-regexp 0.1.7 6.78 kB 6.78 kB
koalas 1.0.2 6.47 kB 6.47 kB
methods 1.1.2 5.29 kB 5.29 kB
module-details-from-path 1.0.3 4.47 kB 4.47 kB

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@codecov
Copy link

codecov bot commented Jun 20, 2023

Codecov Report

Merging #3260 (6622977) into master (5aaa178) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3260   +/-   ##
=======================================
  Coverage   86.03%   86.03%           
=======================================
  Files         194      194           
  Lines        7517     7517           
  Branches       33       33           
=======================================
  Hits         6467     6467           
  Misses       1050     1050           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@pr-commenter
Copy link

pr-commenter bot commented Jun 20, 2023

Benchmarks

Comparing candidate commit 6622977 in PR branch ugaitz/reduce-iast-benchmark with baseline commit 5aaa178 in branch master.

Found 44 performance improvements and 0 performance regressions! Performance is the same for 422 metrics, 26 unstable metrics.

scenario:appsec-iast-no-vulnerability-control-16

  • 🟩 cpu_user_time [-177.763ms; -160.591ms] or [-23.591%; -21.312%]
  • 🟩 execution_time [-174.638ms; -147.373ms] or [-19.480%; -16.439%]
  • 🟩 instructions [+0; +0] or [-17.289%; -17.103%]
  • 🟩 max_rss_usage [-6.568KB; -6.336KB] or [-8.769%; -8.459%]

scenario:appsec-iast-with-vulnerability-control-16

  • 🟩 cpu_user_time [-0.590s; -0.542s] or [-35.453%; -32.587%]
  • 🟩 execution_time [-0.902s; -0.781s] or [-36.923%; -31.949%]
  • 🟩 instructions [-1; -1] or [-33.791%; -33.445%]
  • 🟩 max_rss_usage [-7.851KB; -7.335KB] or [-9.424%; -8.805%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-16

  • 🟩 cpu_user_time [-0.344s; -0.304s] or [-9.681%; -8.550%]
  • 🟩 execution_time [-0.377s; -0.325s] or [-10.199%; -8.815%]
  • 🟩 instructions [+0; +0] or [-20.286%; -19.454%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-16

  • 🟩 cpu_user_time [-0.396s; -0.347s] or [-10.733%; -9.404%]
  • 🟩 execution_time [-0.427s; -0.369s] or [-11.160%; -9.654%]
  • 🟩 instructions [+0; +0] or [-23.264%; -22.869%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-16

  • 🟩 cpu_user_time [-0.663s; -0.590s] or [-14.173%; -12.626%]
  • 🟩 execution_time [-1.223s; -1.007s] or [-20.576%; -16.933%]
  • 🟩 instructions [-1; -1] or [-32.097%; -31.337%]
  • 🟩 max_rss_usage [-13.949KB; -10.989KB] or [-10.668%; -8.405%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-16

  • 🟩 cpu_user_time [-0.841s; -0.715s] or [-17.023%; -14.482%]
  • 🟩 execution_time [-1.486s; -1.231s] or [-23.141%; -19.176%]
  • 🟩 instructions [-2; -2] or [-33.822%; -33.402%]
  • 🟩 max_rss_usage [-10.264KB; -7.317KB] or [-7.627%; -5.437%]

scenario:appsec-iast-no-vulnerability-control-18

  • 🟩 cpu_user_time [-164.650ms; -149.094ms] or [-22.208%; -20.110%]
  • 🟩 execution_time [-197.163ms; -160.394ms] or [-21.868%; -17.790%]
  • 🟩 instructions [+0; +0] or [-17.108%; -16.917%]
  • 🟩 max_rss_usage [-4.795KB; -4.267KB] or [-5.890%; -5.242%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-18

  • 🟩 cpu_user_time [-0.214s; -0.192s] or [-16.507%; -14.813%]
  • 🟩 execution_time [-0.248s; -0.196s] or [-16.970%; -13.368%]
  • 🟩 instructions [+0; +0] or [-18.576%; -17.674%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-18

  • 🟩 cpu_user_time [-0.243s; -0.214s] or [-17.613%; -15.504%]
  • 🟩 execution_time [-0.280s; -0.227s] or [-18.107%; -14.705%]
  • 🟩 instructions [+0; +0] or [-23.298%; -22.962%]

scenario:appsec-iast-with-vulnerability-control-18

  • 🟩 cpu_user_time [-0.580s; -0.524s] or [-35.235%; -31.841%]
  • 🟩 execution_time [-0.960s; -0.800s] or [-38.484%; -32.077%]
  • 🟩 instructions [-1; -1] or [-33.672%; -33.410%]
  • 🟩 max_rss_usage [-7.928KB; -7.484KB] or [-8.875%; -8.377%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-18

  • 🟩 cpu_user_time [-0.668s; -0.584s] or [-28.902%; -25.258%]
  • 🟩 execution_time [-1.231s; -0.968s] or [-34.846%; -27.411%]
  • 🟩 instructions [-1; -1] or [-33.643%; -32.625%]
  • 🟩 max_rss_usage [-11.197KB; -8.003KB] or [-9.144%; -6.535%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-18

  • 🟩 cpu_user_time [-0.806s; -0.680s] or [-30.697%; -25.902%]
  • 🟩 execution_time [-1.465s; -1.155s] or [-36.475%; -28.757%]
  • 🟩 instructions [-2; -2] or [-34.902%; -34.511%]
  • 🟩 max_rss_usage [-9.509KB; -6.465KB] or [-7.601%; -5.168%]

@uurien
Copy link
Collaborator Author

uurien commented Jun 20, 2023

With 200 request by iteration, it expend 9'40", trying with 150 (it should be enough)

@uurien uurien marked this pull request as ready for review June 20, 2023 09:51
@uurien uurien requested a review from a team as a code owner June 20, 2023 09:51
@uurien uurien merged commit bce0ee1 into master Jun 20, 2023
@uurien uurien deleted the ugaitz/reduce-iast-benchmark branch June 20, 2023 11:35
nsavoire pushed a commit that referenced this pull request Jun 20, 2023
* Reduce request per iteration in IAST benchmarks

* Reduce a bit more

* 30 iterations 200 requests
nsavoire pushed a commit that referenced this pull request Jun 20, 2023
* Reduce request per iteration in IAST benchmarks

* Reduce a bit more

* 30 iterations 200 requests
nsavoire pushed a commit that referenced this pull request Jun 21, 2023
* Reduce request per iteration in IAST benchmarks

* Reduce a bit more

* 30 iterations 200 requests
This was referenced Jun 21, 2023
tlhunter pushed a commit that referenced this pull request Jun 23, 2023
* Reduce request per iteration in IAST benchmarks

* Reduce a bit more

* 30 iterations 200 requests
tlhunter pushed a commit that referenced this pull request Jun 23, 2023
* Reduce request per iteration in IAST benchmarks

* Reduce a bit more

* 30 iterations 200 requests
tlhunter pushed a commit that referenced this pull request Jun 23, 2023
* Reduce request per iteration in IAST benchmarks

* Reduce a bit more

* 30 iterations 200 requests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants