MQE: fix incorrect query results or "found duplicate series for the m…

…atch group" errors when binary operation has unsorted labels in `on` (#9482) (#9484)

* MQE: fix incorrect query results or "found duplicate series for the match group" errors when binary operation has unsorted labels in `on`

* Add changelog entry

* Address PR feedback: explain purpose of tests

(cherry picked from commit e8e1e13)

# Conflicts:
#	CHANGELOG.md

CHANGELOG.md

@@ -34,7 +34,7 @@
 * [CHANGE] Distributor: reject incoming requests until the distributor service has started. #9317
 * [FEATURE] Alertmanager: Added `-alertmanager.log-parsing-label-matchers` to control logging when parsing label matchers. This flag is intended to be used with `-alertmanager.utf8-strict-mode-enabled` to validate UTF-8 strict mode is working as intended. The default value is `false`. #9173
 * [FEATURE] Alertmanager: Added `-alertmanager.utf8-migration-logging-enabled` to enable logging of tenant configurations that are incompatible with UTF-8 strict mode. The default value is `false`. #9174
-* [FEATURE] Querier: add experimental streaming PromQL engine, enabled with `-querier.query-engine=mimir`. #8422 #8430 #8454 #8455 #8360 #8490 #8508 #8577 #8660 #8671 #8677 #8747 #8850 #8872 #8838 #8911 #8909 #8923 #8924 #8925 #8932 #8933 #8934 #8962 #8986 #8993 #8995 #9008 #9017 #9018 #9019 #9120 #9121 #9136 #9139 #9140 #9145 #9191 #9192 #9194 #9196 #9201 #9212 #9225 #9260 #9272 #9277 #9278 #9280 #9281 #9342 #9343 #9371
+* [FEATURE] Querier: add experimental streaming PromQL engine, enabled with `-querier.query-engine=mimir`. #8422 #8430 #8454 #8455 #8360 #8490 #8508 #8577 #8660 #8671 #8677 #8747 #8850 #8872 #8838 #8911 #8909 #8923 #8924 #8925 #8932 #8933 #8934 #8962 #8986 #8993 #8995 #9008 #9017 #9018 #9019 #9120 #9121 #9136 #9139 #9140 #9145 #9191 #9192 #9194 #9196 #9201 #9212 #9225 #9260 #9272 #9277 #9278 #9280 #9281 #9342 #9343 #9371 #9482
 * [FEATURE] Experimental Kafka-based ingest storage. #6888 #6894 #6929 #6940 #6951 #6974 #6982 #7029 #7030 #7091 #7142 #7147 #7148 #7153 #7160 #7193 #7349 #7376 #7388 #7391 #7393 #7394 #7402 #7404 #7423 #7424 #7437 #7486 #7503 #7508 #7540 #7621 #7682 #7685 #7694 #7695 #7696 #7697 #7701 #7733 #7734 #7741 #7752 #7838 #7851 #7871 #7877 #7880 #7882 #7887 #7891 #7925 #7955 #7967 #8031 #8063 #8077 #8088 #8135 #8176 #8184 #8194 #8216 #8217 #8222 #8233 #8503 #8542 #8579 #8657 #8686 #8688 #8703 #8706 #8708 #8738 #8750 #8778 #8808 #8809 #8841 #8842 #8845 #8853 #8886 #8988
   * What it is:
     * When the new ingest storage architecture is enabled, distributors write incoming write requests to a Kafka-compatible backend, and the ingesters asynchronously replay ingested data from Kafka. In this architecture, the write and read path are de-coupled through a Kafka-compatible backend. The write path and Kafka load is a function of the incoming write traffic, the read path load is a function of received queries. Whatever the load on the read path, it doesn't affect the write path.

pkg/streamingpromql/operators/vector_vector_binary_operation.go

@@ -386,6 +386,8 @@ func (b *VectorVectorBinaryOperation) groupKeyFunc() func(labels.Labels) []byte
 	buf := make([]byte, 0, 1024)
 
 	if b.VectorMatching.On {
+		slices.Sort(b.VectorMatching.MatchingLabels)
+
 		return func(l labels.Labels) []byte {
 			return l.BytesWithLabels(buf, b.VectorMatching.MatchingLabels...)
 		}

pkg/streamingpromql/testdata/ours/binary_operators.test

@@ -98,11 +98,27 @@ eval range from 0 to 24m step 6m left_side - on(env, pod) right_side
   {env="test", pod="a"} -9 -18 -27
   {env="test", pod="b"} -36 -45 -54
 
+# Test the same thing again with the grouping labels in a different order.
+# (The implementation of binary operations relies on grouping labels being sorted in some places,
+# so this test exists to ensure this is done correctly.)
+eval range from 0 to 24m step 6m left_side - on(pod, env) right_side
+  {env="prod", pod="a"} -63 -72 -81
+  {env="test", pod="a"} -9 -18 -27
+  {env="test", pod="b"} -36 -45 -54
+
 eval range from 0 to 24m step 6m left_side - ignoring(env, pod) right_side
   {group="baz"} -33 -42 -51
   {group="bar"} -6 -15 -24
   {group="foo"} -69 -78 -87
 
+# Test the same thing again with the grouping labels in a different order.
+# (The implementation of binary operations relies on grouping labels being sorted in some places,
+# so this test exists to ensure this is done correctly.)
+eval range from 0 to 24m step 6m left_side - ignoring(pod, env) right_side
+  {group="baz"} -33 -42 -51
+  {group="bar"} -6 -15 -24
+  {group="foo"} -69 -78 -87
+
 clear
 
 # One-to-one matching, but different series match at different time steps, or not at all