Use absolute features when doing training/prediction.

[vkalintiris]

Summary

Add a final step to the preprocessing of feature vectors to make them always be absolute values only. This follows on from some internal research showing the impact of this on the distance measure used to generate anomaly scores (illustrative colab notebook).

In below chart we add some contamination to the metric in last part of the chart.

Orange line below then shows impact of ensuring feature vectors are all absolute as opposed to allowing both positive and negative which can lead to a bias in distance measures of metics such as system.cpu that tend to randomly move up and down from second to second.

In this case the orange line behaves better as an anomaly score and jumps consistently in the period of contaminated data. Whereas the implementation that does not force absolute values actually performs worse. Essentially this is because allowing positive and negative values "naturally" suppresses distance measures since elements of the feature vector can essentially cancel each other out when you try to compare distances.

Component Name

area/ml

Test Plan

CI & verification with the ML team.

[andrewm4894]

LGTM. Tested and verified it behaves as expected on one of our devml nodes.

[andrewm4894]

LGTM