Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add E-Divisive means algorithm to the available change detection models #1007

Closed
smalleni opened this issue Jan 2, 2024 · 2 comments · Fixed by #1494
Closed

Add E-Divisive means algorithm to the available change detection models #1007

smalleni opened this issue Jan 2, 2024 · 2 comments · Fixed by #1494
Assignees
Milestone

Comments

@smalleni
Copy link

smalleni commented Jan 2, 2024

Feature idea.

There is a lot of research and existing applications around using E-Divisive means for identifying change points in software performance data. See: https://arxiv.org/pdf/2003.00584.pdf Hence, it would be good to have that added to the current available changepoint detection models in Horreum.
Describe the solution you'd like

Add E-Divisive means based change point detection model here: https://github.com/Hyperfoil/Horreum/tree/master/horreum-backend/src/main/java/io/hyperfoil/tools/horreum/changedetection

@smalleni smalleni added the type/feature A new feature label Jan 2, 2024
Copy link

github-actions bot commented Jan 2, 2024

👋 @smalleni
Thanks for opening your first issue here! Be sure to follow the issue template!

@johnaohara
Copy link
Member

Hi @smalleni thanks for the suggestion.

This is an algorithm that we have discussed implementing in the past. and have recoded some of the disucssion in various discussions/issues;

#913
#38

The implementation (hunter: https://github.com/datastax-labs/hunter) that we were planning on using is detailed in this paper: https://arxiv.org/pdf/2301.03034.pdf and has been used in production with mongoDB for a number of years.

There are a number of implementation details that we need to perform before we integrate Hunter;

  • Polyglot language support: a lot of algorithms/libraries are created in python/R, we need to support multiple languages in order to not re-create each algorithm in Java
  • Batch processing: currently, Horreum supports a continual monotonic time-series of data. Algorithms like eDivisive Mean require batch processing of the entire dataset and we need to design into Horreum how to handle deltas in historic change point detection and the implications for notifications/change storage etc. I.e. if we run eDivisive one week and get 4 changes detected, then re-run a week later and get 3, then how do we handle the differences in the UI/reporting etc. The same applies to if the algorithm detects different changes between runs.

@johnaohara johnaohara added this to the 0.13 milestone Jan 2, 2024
@johnaohara johnaohara self-assigned this Mar 5, 2024
johnaohara added a commit to johnaohara/Horreum that referenced this issue Apr 15, 2024
…ixes Hyperfoil#1007

 - run bulk change detection once per fingerprint
 - Allow option to recalc datapoints for variables, or re-use
 - added end-to-end and injectable tests

blocked by : Hyperfoil#1380
johnaohara added a commit to johnaohara/Horreum that referenced this issue Apr 24, 2024
…ixes Hyperfoil#1007

 - run bulk change detection once per fingerprint
 - Allow option to recalc datapoints for variables, or re-use
 - added end-to-end and injectable tests

blocked by : Hyperfoil#1380
johnaohara added a commit to johnaohara/Horreum that referenced this issue Apr 29, 2024
…ixes Hyperfoil#1007

 - run bulk change detection once per fingerprint
 - Allow option to recalc datapoints for variables, or re-use
 - added end-to-end and injectable tests
 - Disable eDivisive tests unless enabled with 'ci' profile
johnaohara added a commit to johnaohara/Horreum that referenced this issue Apr 29, 2024
…ixes Hyperfoil#1007

 - run bulk change detection once per fingerprint
 - Allow option to recalc datapoints for variables, or re-use
 - added end-to-end and injectable tests
 - Disable eDivisive tests unless enabled with 'ci' profile
johnaohara added a commit that referenced this issue May 1, 2024
…ixes #1007

 - run bulk change detection once per fingerprint
 - Allow option to recalc datapoints for variables, or re-use
 - added end-to-end and injectable tests
 - Disable eDivisive tests unless enabled with 'ci' profile
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants