Move scripted-metrics agg to plugin #63096
Labels
:Analytics/Aggregations
Aggregations
>refactoring
Team:Analytics
Meta label for analytical engine team (ESQL/Aggs/Geo)
We'd like to move the scripted-metrics aggregation to a plugin. This can be included by default so that users can continue to use it without disruption, but will prevent "stack" components from using it.
The rationale behind this is that scripted-metrics is very powerful and offers a lot of flexibility... but it is also a very sharp tool and difficult to use well. Because scripted-metrics is essentially a scripted map-reduce job, the user needs to be aware of how to build a robust distributed algorithm, not leak memory or take excessive runtime, and do it inside a scripting language. It is very easy to (accidentally) write bad scripted-metrics for these reasons. Worse yet, many of these "bad" implementations will work fine at small scale but can blow up catastrophically when encountering adversarial situations in large clusters.
For these reasons, we wish to discourage our own stack features from using scripted-metric. If there is functionality that is only doable with scripted-metric, it's a good indicator that we are missing important functionality in regular aggregations and we can instead work on adding "core" functionality. This will ensure that it is properly tested for performance and stability, and maintained by the analytics team.
By moving scripted-metric to a plugin, we can add a forcing function for "internal" uses to find (or build) alternate solutions, while still allowing users of Elasticsearch to use scripted-metric unchanged.
The text was updated successfully, but these errors were encountered: