From 699069fd111706fa78cf75ddfe13fbf86ab8bd22 Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Sun, 15 May 2022 07:50:37 -0400 Subject: [PATCH 01/12] Create v2-1.md --- docs/sources/release-notes/v2-1.md | 88 ++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 docs/sources/release-notes/v2-1.md diff --git a/docs/sources/release-notes/v2-1.md b/docs/sources/release-notes/v2-1.md new file mode 100644 index 00000000000..198ff86da94 --- /dev/null +++ b/docs/sources/release-notes/v2-1.md @@ -0,0 +1,88 @@ +--- +title: "Grafana Mimir version 2.0 release notes" +menuTitle: "V2.0 release notes" +description: "Release notes for Grafana Mimir version 2.0" +weight: 100 +--- + +# Grafana Mimir version 2.0 release notes + +Grafana Labs is excited to announce the first release of Grafana Mimir, the most scalable, most performant open source time series database in the world. In customer tests, we’ve shown that a single cluster can support more than one billion active time series. + +Besides massive scale, Grafana Mimir offers a host of other benefits, including easy deployment, native multi-tenancy, high availability, durable long-term storage, and exceptional query performance on even the highest-cardinality queries. + +For more information about Grafana Mimir features and capabilities, watch the **Introducing Grafana Mimir** video. +{{< vimeo 688679929 >}} + +
+ +We’re launching Grafana Mimir with a 2.0 version number to signal our respect for Cortex, the project from which Grafana Mimir was forked. The choice of 2.0 also represents our conviction that Grafana Mimir is real-world-tested, production-ready software. It has served as the backbone of our Grafana Cloud Metrics and Grafana Enterprise Metrics products since their inception. + +For more on the story of why we created Grafana Mimir, please see the following blog posts: + +- [Announcing Grafana Mimir, the most scalable open source TSDB in the world](https://grafana.com/blog/2022/03/30/announcing-grafana-mimir/) +- [Grafana Mimir Q&A with Grafana Labs CEO Raj Dutt](https://grafana.com/blog/2022/03/30/qa-with-our-ceo-about-grafana-mimir/) + +Grafana Mimir builds on Cortex 1.10.0, adding features we developed to run Grafana Enterprise Metrics and Grafana Cloud Metrics at massive scale. As a result, these listed features and enhancements, migration considerations, and bug fixes highlight changes between Cortex 1.10.0 and Grafana Mimir 2.0. + +The complete list of changes is recorded in the [Changelog](https://github.com/grafana/mimir/blob/main/CHANGELOG.md). + +## Features and enhancements + +These features and enhancements distinguish Grafana Mimir from Cortex 1.10.0: + +- **Simplified deployment experience**: These changes improve the experience of setting up and maintaining Grafana Mimir: + + - We revamped Cortex’s single process mode, renaming it [monolithic mode]({{< relref "../operators-guide/architecture/deployment-modes/index.md#monolithic-mode" >}}). Monolithic mode now runs the required compactor component, and has better configuration defaults, so that it works with fewer changes out-of-the-box. + + - We switched to `memberlist` as the default store for Grafana Mimir’s [hash rings]({{< relref "../operators-guide/architecture/hash-ring/index.md" >}}). With this, users no longer have to run Consul or etcd as external dependencies. We made performance optimizations to `memberlist` to reduce its CPU utilization, which ensures that `memberlist` runs smoothly on Grafana Mimir clusters with lots of active series. + + - We’ve included our own internal best practice dashboards, mixins, and alerts for [monitoring Grafana Mimir]({{< relref "../operators-guide/visualizing-metrics" >}}). While installing monitoring best practices such as these has typically required use of Jsonnet, we’ve eliminated this requirement. We include dashboards as JSON and alerting and recording rules as YAML which can be directly imported into your Grafana and Prometheus deployments. The alerts are accompanied by [playbooks](https://github.com/grafana/mimir/blob/main/operations/mimir-mixin/docs/playbooks.md) distilled from our own internal operations. + +- **Configuration parameter reduction and classification**: We removed 36% of the configuration parameters in Grafana Mimir. All remaining configuration parameters have been classified as basic, advanced, or experimental. This is meant to make Grafana Mimir’s configuration more approachable for new users. In a default installation, you can focus exclusively on basic configuration. As you become more familiar with Grafana Mimir and want to push your clusters further, you can choose to tune advanced parameters or use experimental parameters. Refer to [parameter categories]({{< relref "../operators-guide/configuring/reference-configuration-parameters/index.md#parameter-categories" >}}) to learn more. + +- **More-scalable metrics compaction**: Grafana Mimir's new compactor uses a split-and-merge compaction algorithm. This new compactor parallelizes the compaction of overlapping blocks across multiple machines, and splits blocks in such a way as to overcome Prometheus’ [64 GB index size limit](https://grafana.com/blog/2019/10/31/lifting-the-index-size-limit-of-prometheus-with-postings-compression/). This solves a problem that we've seen with tenants with greater than 30M time series in Cortex - namely, that the count of uncompacted blocks grew indefinitely because the compactor couldn’t work fast enough. With its new compactor, a single Grafana Mimir cluster can support over one billion active time series. Refer to [compactor]({{< relref "../operators-guide/architecture/components/compactor/index.md" >}}) to learn more. + +- **Query sharding for improved query speeds**: Grafana Mimir introduces query sharding to accelerate the execution of high-cardinality or CPU-intensive queries. Query sharding distributes the execution of a single query across multiple machines, to significantly reduce query execution time. We have seen speedups of 10 to 30x in our Grafana Cloud Metrics clusters. Refer to [query sharding]({{< relref "../operators-guide/architecture/query-sharding/index.md" >}}) to learn more. + +- **Federated rule groups**: Grafana Mimir makes it possible to write alerting and recording rules that use metrics data from multiple tenants. For example, a user can now create a recording rule that adds metricA from tenantA to metricB from tenantB and writes the result to tenantC. This feature is experimental. For more information on how to use it, refer to [federated rule groups]({{< relref "../operators-guide/reference-http-api/index.md#federated-rule-groups" >}}). + +- **Understand your metrics cardinality**: Grafana Mimir adds two API endpoints to help users identify high-cardinality metrics. The [`label_names` endpoint]({{< relref "../operators-guide/reference-http-api/index.md#label-names-cardinality" >}}) takes a metric name and returns all label names applied to that metric, as well as the count of values for each label name. When run without a metric name, it returns the highest-cardinality label names. The [`label_values` endpoint]({{< relref "../operators-guide/reference-http-api/index.md#label-values-cardinality" >}}) returns the highest-cardinality metrics, and can be used to get a count of how many series have a given label-value pair applied. The new [custom tracker feature]({{< relref "../operators-guide/configuring/configuring-custom-trackers.md" >}}) takes this one step further by allowing you to track the count of active series over time that match a specific label matcher. + +These features are no longer considered experimental; they are stable features: + +- **Cross tenant query federation** is a feature that allows you to issue queries that aggregate series across multiple tenants, providing a way to get a global view of your data. Refer to `tenant-federation.enabled` in the [configuration parameters]({{< relref "../operators-guide/configuring/reference-configuration-parameters/index.md#configuration-parameters" >}}) block to learn more. + +- **Zone-aware replication** protects against data loss due to an outage in a specific zone by ensuring data is replicated across multiple zones. In addition to promoting this feature to stable, we’ve also released a [Kubernetes operator](https://github.com/grafana/rollout-operator) that makes it easier for those running Grafana Mimir on Kubernetes to manage multi-zone rollouts. Refer to [Configuring zone-aware replication]({{< relref "../operators-guide/configuring/configuring-zone-aware-replication.md" >}}) to learn more. + +- **Azure Blob storage and OpenStack Swift support**: Two additional object storage options are available. Grafana Mimir now supports AWS S3, GCS, Azure Blob Storage, OpenStack Swift, as well as any S3-compatible object storage. + +- **Horizontally scalable Alertmanager**: The [Alertmanager]({{< relref "../operators-guide/architecture/components/alertmanager.md" >}}) can distribute its workload among multiple Alertmanager replicas, increasing the maximum number of tenants a cluster can comfortably handle. The Alertmanager uses the tenant ID to determine which replica(s) own which tenant. The replication factor is configurable, so alerts are replicated across multiple replicas, reducing the likelihood of a dropped alert. + +- **Query-scheduler component**: The query-scheduler component solves the scalability limitations of the query-frontend component. We strongly encourage users operating larger Grafana Mimir clusters to run this optional component. Refer to the [query-scheduler]({{< relref "../operators-guide/architecture/components/query-scheduler" >}}) to learn more about this component. + +## Migration considerations + +This information is relevant only to users migrating from Cortex to Grafana Mimir. Those who are not currently using Cortex today can follow our tutorial to [Get Mimir up and running in 10 minutes](https://grafana.com/tutorials/play-with-grafana-mimir/). + +Grafana Mimir is 100% compatible with Cortex, and the migration from Cortex to Mimir can be done without any data loss or downtime. + +Grafana Mimir 2.0 is a major version upgrade, and therefore it introduces several breaking changes for those migrating from Cortex. To make this upgrade as easy as possible, we’ve added functionality to [`mimirtool`]({{< relref "../operators-guide/tools/mimirtool.md" >}}) to automate the required configuration changes. We’ve paired this tooling with detailed [migration guides]({{< relref "../migration-guide/" >}}) that walk through all required changes. + +For a demonstration that illustrates how easy this migration can be, please see the following video: + +{{< vimeo 691929138 >}} + +
+ +## Bug fixes + +### 2.0.0 bug fixes + +Version 2.0.0 fixes these bugs: + +- [PR 424](https://github.com/grafana/mimir/pull/424): Fixed a bug where cached results for step-aligned queries were used to respond to unaligned queries, causing the user to receive incorrect results. The new configuration parameter `query-frontend.cache-unaligned-requests` safely enables the caching of query results, even when the query step is not aligned. + +- [PR 224](https://github.com/grafana/mimir/pull/224): Fixed a race condition in the Alertmanager, which sometimes resulted in a tenant’s configuration being replaced with the blank fallback configuration. + +- [PR 551](https://github.com/grafana/mimir/pull/551): Fixed a bug in `memberlist` that caused large messages to be corrupted. This bug fix, in addition to the memberlist performance optimizations, enabled us to switch to `memberlist` as the default store for Grafana Mimir’s hash rings. From c1536afe99ffa46724923d4890100b6da0db30bb Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Sun, 15 May 2022 07:56:26 -0400 Subject: [PATCH 02/12] Update and rename v2-1.md to v2.1.md updated the header and renamed the file. --- .../release-notes/{v2-1.md => v2.1.md} | 36 ++++--------------- 1 file changed, 7 insertions(+), 29 deletions(-) rename docs/sources/release-notes/{v2-1.md => v2.1.md} (68%) diff --git a/docs/sources/release-notes/v2-1.md b/docs/sources/release-notes/v2.1.md similarity index 68% rename from docs/sources/release-notes/v2-1.md rename to docs/sources/release-notes/v2.1.md index 198ff86da94..5ef46ba37c3 100644 --- a/docs/sources/release-notes/v2-1.md +++ b/docs/sources/release-notes/v2.1.md @@ -1,44 +1,22 @@ --- -title: "Grafana Mimir version 2.0 release notes" -menuTitle: "V2.0 release notes" -description: "Release notes for Grafana Mimir version 2.0" -weight: 100 +title: "Grafana Mimir version 2.1 release notes" +menuTitle: "V2.1 release notes" +description: "Release notes for Grafana Mimir version 2.1" +weight: 200 --- -# Grafana Mimir version 2.0 release notes +# Grafana Mimir version 2.1 release notes -Grafana Labs is excited to announce the first release of Grafana Mimir, the most scalable, most performant open source time series database in the world. In customer tests, we’ve shown that a single cluster can support more than one billion active time series. - -Besides massive scale, Grafana Mimir offers a host of other benefits, including easy deployment, native multi-tenancy, high availability, durable long-term storage, and exceptional query performance on even the highest-cardinality queries. - -For more information about Grafana Mimir features and capabilities, watch the **Introducing Grafana Mimir** video. -{{< vimeo 688679929 >}} +Grafana Labs is excited to announce version 2.1 of Grafana Mimir, the most scalable, most performant open source time series database in the world.
-We’re launching Grafana Mimir with a 2.0 version number to signal our respect for Cortex, the project from which Grafana Mimir was forked. The choice of 2.0 also represents our conviction that Grafana Mimir is real-world-tested, production-ready software. It has served as the backbone of our Grafana Cloud Metrics and Grafana Enterprise Metrics products since their inception. - -For more on the story of why we created Grafana Mimir, please see the following blog posts: - -- [Announcing Grafana Mimir, the most scalable open source TSDB in the world](https://grafana.com/blog/2022/03/30/announcing-grafana-mimir/) -- [Grafana Mimir Q&A with Grafana Labs CEO Raj Dutt](https://grafana.com/blog/2022/03/30/qa-with-our-ceo-about-grafana-mimir/) - -Grafana Mimir builds on Cortex 1.10.0, adding features we developed to run Grafana Enterprise Metrics and Grafana Cloud Metrics at massive scale. As a result, these listed features and enhancements, migration considerations, and bug fixes highlight changes between Cortex 1.10.0 and Grafana Mimir 2.0. +Below we highlight the top features, enhancements and bugfixes in this release, as well as relevant callouts for those upgrading from Grafana Mimir 2.0. The complete list of changes is recorded in the [Changelog](https://github.com/grafana/mimir/blob/main/CHANGELOG.md). ## Features and enhancements -These features and enhancements distinguish Grafana Mimir from Cortex 1.10.0: - -- **Simplified deployment experience**: These changes improve the experience of setting up and maintaining Grafana Mimir: - - - We revamped Cortex’s single process mode, renaming it [monolithic mode]({{< relref "../operators-guide/architecture/deployment-modes/index.md#monolithic-mode" >}}). Monolithic mode now runs the required compactor component, and has better configuration defaults, so that it works with fewer changes out-of-the-box. - - - We switched to `memberlist` as the default store for Grafana Mimir’s [hash rings]({{< relref "../operators-guide/architecture/hash-ring/index.md" >}}). With this, users no longer have to run Consul or etcd as external dependencies. We made performance optimizations to `memberlist` to reduce its CPU utilization, which ensures that `memberlist` runs smoothly on Grafana Mimir clusters with lots of active series. - - - We’ve included our own internal best practice dashboards, mixins, and alerts for [monitoring Grafana Mimir]({{< relref "../operators-guide/visualizing-metrics" >}}). While installing monitoring best practices such as these has typically required use of Jsonnet, we’ve eliminated this requirement. We include dashboards as JSON and alerting and recording rules as YAML which can be directly imported into your Grafana and Prometheus deployments. The alerts are accompanied by [playbooks](https://github.com/grafana/mimir/blob/main/operations/mimir-mixin/docs/playbooks.md) distilled from our own internal operations. - - **Configuration parameter reduction and classification**: We removed 36% of the configuration parameters in Grafana Mimir. All remaining configuration parameters have been classified as basic, advanced, or experimental. This is meant to make Grafana Mimir’s configuration more approachable for new users. In a default installation, you can focus exclusively on basic configuration. As you become more familiar with Grafana Mimir and want to push your clusters further, you can choose to tune advanced parameters or use experimental parameters. Refer to [parameter categories]({{< relref "../operators-guide/configuring/reference-configuration-parameters/index.md#parameter-categories" >}}) to learn more. - **More-scalable metrics compaction**: Grafana Mimir's new compactor uses a split-and-merge compaction algorithm. This new compactor parallelizes the compaction of overlapping blocks across multiple machines, and splits blocks in such a way as to overcome Prometheus’ [64 GB index size limit](https://grafana.com/blog/2019/10/31/lifting-the-index-size-limit-of-prometheus-with-postings-compression/). This solves a problem that we've seen with tenants with greater than 30M time series in Cortex - namely, that the count of uncompacted blocks grew indefinitely because the compactor couldn’t work fast enough. With its new compactor, a single Grafana Mimir cluster can support over one billion active time series. Refer to [compactor]({{< relref "../operators-guide/architecture/components/compactor/index.md" >}}) to learn more. From b2e60988f956b7fb3471d3ed3e33ad80ece9caa6 Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Sun, 15 May 2022 09:00:11 -0400 Subject: [PATCH 03/12] Update v2.1.md Missing the upgrade configurations. --- docs/sources/release-notes/v2.1.md | 40 +++++------------------------- 1 file changed, 6 insertions(+), 34 deletions(-) diff --git a/docs/sources/release-notes/v2.1.md b/docs/sources/release-notes/v2.1.md index 5ef46ba37c3..de6e3c75de2 100644 --- a/docs/sources/release-notes/v2.1.md +++ b/docs/sources/release-notes/v2.1.md @@ -9,55 +9,27 @@ weight: 200 Grafana Labs is excited to announce version 2.1 of Grafana Mimir, the most scalable, most performant open source time series database in the world. -
- Below we highlight the top features, enhancements and bugfixes in this release, as well as relevant callouts for those upgrading from Grafana Mimir 2.0. The complete list of changes is recorded in the [Changelog](https://github.com/grafana/mimir/blob/main/CHANGELOG.md). ## Features and enhancements -- **Configuration parameter reduction and classification**: We removed 36% of the configuration parameters in Grafana Mimir. All remaining configuration parameters have been classified as basic, advanced, or experimental. This is meant to make Grafana Mimir’s configuration more approachable for new users. In a default installation, you can focus exclusively on basic configuration. As you become more familiar with Grafana Mimir and want to push your clusters further, you can choose to tune advanced parameters or use experimental parameters. Refer to [parameter categories]({{< relref "../operators-guide/configuring/reference-configuration-parameters/index.md#parameter-categories" >}}) to learn more. - -- **More-scalable metrics compaction**: Grafana Mimir's new compactor uses a split-and-merge compaction algorithm. This new compactor parallelizes the compaction of overlapping blocks across multiple machines, and splits blocks in such a way as to overcome Prometheus’ [64 GB index size limit](https://grafana.com/blog/2019/10/31/lifting-the-index-size-limit-of-prometheus-with-postings-compression/). This solves a problem that we've seen with tenants with greater than 30M time series in Cortex - namely, that the count of uncompacted blocks grew indefinitely because the compactor couldn’t work fast enough. With its new compactor, a single Grafana Mimir cluster can support over one billion active time series. Refer to [compactor]({{< relref "../operators-guide/architecture/components/compactor/index.md" >}}) to learn more. - -- **Query sharding for improved query speeds**: Grafana Mimir introduces query sharding to accelerate the execution of high-cardinality or CPU-intensive queries. Query sharding distributes the execution of a single query across multiple machines, to significantly reduce query execution time. We have seen speedups of 10 to 30x in our Grafana Cloud Metrics clusters. Refer to [query sharding]({{< relref "../operators-guide/architecture/query-sharding/index.md" >}}) to learn more. - -- **Federated rule groups**: Grafana Mimir makes it possible to write alerting and recording rules that use metrics data from multiple tenants. For example, a user can now create a recording rule that adds metricA from tenantA to metricB from tenantB and writes the result to tenantC. This feature is experimental. For more information on how to use it, refer to [federated rule groups]({{< relref "../operators-guide/reference-http-api/index.md#federated-rule-groups" >}}). - -- **Understand your metrics cardinality**: Grafana Mimir adds two API endpoints to help users identify high-cardinality metrics. The [`label_names` endpoint]({{< relref "../operators-guide/reference-http-api/index.md#label-names-cardinality" >}}) takes a metric name and returns all label names applied to that metric, as well as the count of values for each label name. When run without a metric name, it returns the highest-cardinality label names. The [`label_values` endpoint]({{< relref "../operators-guide/reference-http-api/index.md#label-values-cardinality" >}}) returns the highest-cardinality metrics, and can be used to get a count of how many series have a given label-value pair applied. The new [custom tracker feature]({{< relref "../operators-guide/configuring/configuring-custom-trackers.md" >}}) takes this one step further by allowing you to track the count of active series over time that match a specific label matcher. - -These features are no longer considered experimental; they are stable features: - -- **Cross tenant query federation** is a feature that allows you to issue queries that aggregate series across multiple tenants, providing a way to get a global view of your data. Refer to `tenant-federation.enabled` in the [configuration parameters]({{< relref "../operators-guide/configuring/reference-configuration-parameters/index.md#configuration-parameters" >}}) block to learn more. - -- **Zone-aware replication** protects against data loss due to an outage in a specific zone by ensuring data is replicated across multiple zones. In addition to promoting this feature to stable, we’ve also released a [Kubernetes operator](https://github.com/grafana/rollout-operator) that makes it easier for those running Grafana Mimir on Kubernetes to manage multi-zone rollouts. Refer to [Configuring zone-aware replication]({{< relref "../operators-guide/configuring/configuring-zone-aware-replication.md" >}}) to learn more. - -- **Azure Blob storage and OpenStack Swift support**: Two additional object storage options are available. Grafana Mimir now supports AWS S3, GCS, Azure Blob Storage, OpenStack Swift, as well as any S3-compatible object storage. - -- **Horizontally scalable Alertmanager**: The [Alertmanager]({{< relref "../operators-guide/architecture/components/alertmanager.md" >}}) can distribute its workload among multiple Alertmanager replicas, increasing the maximum number of tenants a cluster can comfortably handle. The Alertmanager uses the tenant ID to determine which replica(s) own which tenant. The replication factor is configurable, so alerts are replicated across multiple replicas, reducing the likelihood of a dropped alert. - -- **Query-scheduler component**: The query-scheduler component solves the scalability limitations of the query-frontend component. We strongly encourage users operating larger Grafana Mimir clusters to run this optional component. Refer to the [query-scheduler]({{< relref "../operators-guide/architecture/components/query-scheduler" >}}) to learn more about this component. - -## Migration considerations - -This information is relevant only to users migrating from Cortex to Grafana Mimir. Those who are not currently using Cortex today can follow our tutorial to [Get Mimir up and running in 10 minutes](https://grafana.com/tutorials/play-with-grafana-mimir/). +- **Mimir on ARM**: The `mimir build` command now builds both amd64 and arm64 Docker images, making it easier for those on arm-based machines to develop and run Mimir. Both amd64 and arm64 images will now be published to the [Mimir docker registry](https://hub.docker.com/r/grafana/mimir). Note that our existing integration test suite only uses the amd64 images, which means we cannot make any functional or performance guarantees about the arm64 images. -Grafana Mimir is 100% compatible with Cortex, and the migration from Cortex to Mimir can be done without any data loss or downtime. +- **`Remote` ruler mode for improved rule evaluation performance**: We've added a `remote` mode for the Grafana Mimir ruler, in which the ruler delegates rule evaluation to the [query-frontend]({{< relref "../operators-guide/architecture/components/query-frontend/index.md" >}}) rather than evaluating rules directly within the ruler process itself. This allows recording and alerting rules to benefit from the query parallelization techniques implemented in the query-frontend (like query sharding). `Remote` mode is considered experimental and is off by default. To enable, see [remote ruler]({{< relref "../operators-guide/architecture/components/ruler/#remote" >}}). -Grafana Mimir 2.0 is a major version upgrade, and therefore it introduces several breaking changes for those migrating from Cortex. To make this upgrade as easy as possible, we’ve added functionality to [`mimirtool`]({{< relref "../operators-guide/tools/mimirtool.md" >}}) to automate the required configuration changes. We’ve paired this tooling with detailed [migration guides]({{< relref "../migration-guide/" >}}) that walk through all required changes. +- **Per-tenant custom trackers for monitoring cardinality**: In Grafana Mimir 2.0, we introduced a [custom tracker feature]({{< relref "../operators-guide/configuring/configuring-custom-trackers.md" >}}) that allows you to track the count of active series over time that match a specific label matcher. In Grafana Mimir 2.1, we've made it possible to configure custom trackers via the [runtime configuration file]({{< relref "../operators-guide/configuring/about-runtime-configuration.md" >}}). This means you can now define different trackers for each tenant in your cluster and modify those trackers without an ingester restart. -For a demonstration that illustrates how easy this migration can be, please see the following video: +## Upgrade considerations -{{< vimeo 691929138 >}} -
## Bug fixes -### 2.0.0 bug fixes +### 2.1.0 bug fixes -Version 2.0.0 fixes these bugs: +Version 2.1.0 fixes these bugs: - [PR 424](https://github.com/grafana/mimir/pull/424): Fixed a bug where cached results for step-aligned queries were used to respond to unaligned queries, causing the user to receive incorrect results. The new configuration parameter `query-frontend.cache-unaligned-requests` safely enables the caching of query results, even when the query step is not aligned. From 3786902a9a64bff1b0d9ad332f5336938f8dd13e Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Sun, 15 May 2022 09:39:56 -0400 Subject: [PATCH 04/12] Update v2.1.md added bug description --- docs/sources/release-notes/v2.1.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/sources/release-notes/v2.1.md b/docs/sources/release-notes/v2.1.md index de6e3c75de2..157f759b8c3 100644 --- a/docs/sources/release-notes/v2.1.md +++ b/docs/sources/release-notes/v2.1.md @@ -21,9 +21,15 @@ The complete list of changes is recorded in the [Changelog](https://github.com/g - **Per-tenant custom trackers for monitoring cardinality**: In Grafana Mimir 2.0, we introduced a [custom tracker feature]({{< relref "../operators-guide/configuring/configuring-custom-trackers.md" >}}) that allows you to track the count of active series over time that match a specific label matcher. In Grafana Mimir 2.1, we've made it possible to configure custom trackers via the [runtime configuration file]({{< relref "../operators-guide/configuring/about-runtime-configuration.md" >}}). This means you can now define different trackers for each tenant in your cluster and modify those trackers without an ingester restart. +- **Remove unused series from Grafana Mimir's `/metrics` endpoint**: + ## Upgrade considerations +We've updated the default values for 2 parameters in Grafana Mimir to give users better out-of-the-box performance: + +- We've changed the default for `-blocks-storage.tsdb.isolation-enabled` from `true` to `false`. We've marked this flag as deprecated and will remove it, setting the value permanently to `false`, in 2 releases. Our decision to do this came from our experience running our [1 billion series load test](https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-prometheus-tsdb-grafana-mimir-to-1-billion-active-series/#prometheus-tsdb-enhancements), where we saw that disabling this setting reduced ingester 99th percentile latency by 90%. +- The store gateway attributes cache is now enabled by default (achieved by updating the default for `-blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items` from `0` to `50000`). This in-memory cache makes it faster to look up object attributes for chunk data. We've been running this optional cache internally for a while and upon a recent configuration audit, realized it made sense to do the same for all users. We estimate this change will consume an additional 6MB of memory, but we believe it is well worth the performance improvement. ## Bug fixes @@ -31,8 +37,4 @@ The complete list of changes is recorded in the [Changelog](https://github.com/g Version 2.1.0 fixes these bugs: -- [PR 424](https://github.com/grafana/mimir/pull/424): Fixed a bug where cached results for step-aligned queries were used to respond to unaligned queries, causing the user to receive incorrect results. The new configuration parameter `query-frontend.cache-unaligned-requests` safely enables the caching of query results, even when the query step is not aligned. - -- [PR 224](https://github.com/grafana/mimir/pull/224): Fixed a race condition in the Alertmanager, which sometimes resulted in a tenant’s configuration being replaced with the blank fallback configuration. - -- [PR 551](https://github.com/grafana/mimir/pull/551): Fixed a bug in `memberlist` that caused large messages to be corrupted. This bug fix, in addition to the memberlist performance optimizations, enabled us to switch to `memberlist` as the default store for Grafana Mimir’s hash rings. +- [PR 1704](https://github.com/grafana/mimir/pull/1704): Fixed a bug that previously caused Grafana Mimir to crash on startup when trying to run in monolithic mode with the results cache enabled due to duplicate metric names. From 07689da9e41a23cf1058b098342b19504194e736 Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Sun, 15 May 2022 10:09:53 -0400 Subject: [PATCH 05/12] Update v2.1.md bug fix writeup. --- docs/sources/release-notes/v2.1.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/sources/release-notes/v2.1.md b/docs/sources/release-notes/v2.1.md index 157f759b8c3..928e125f766 100644 --- a/docs/sources/release-notes/v2.1.md +++ b/docs/sources/release-notes/v2.1.md @@ -9,9 +9,7 @@ weight: 200 Grafana Labs is excited to announce version 2.1 of Grafana Mimir, the most scalable, most performant open source time series database in the world. -Below we highlight the top features, enhancements and bugfixes in this release, as well as relevant callouts for those upgrading from Grafana Mimir 2.0. - -The complete list of changes is recorded in the [Changelog](https://github.com/grafana/mimir/blob/main/CHANGELOG.md). +Below we highlight the top features, enhancements and bugfixes in this release, as well as relevant callouts for those upgrading from Grafana Mimir 2.0. The complete list of changes is recorded in the [Changelog](https://github.com/grafana/mimir/blob/main/CHANGELOG.md). ## Features and enhancements @@ -35,6 +33,7 @@ We've updated the default values for 2 parameters in Grafana Mimir to give users ### 2.1.0 bug fixes -Version 2.1.0 fixes these bugs: - - [PR 1704](https://github.com/grafana/mimir/pull/1704): Fixed a bug that previously caused Grafana Mimir to crash on startup when trying to run in monolithic mode with the results cache enabled due to duplicate metric names. +- [PR 1835](https://github.com/grafana/mimir/pull/1835): Fixed a bug that caused Grafana Mimir to crash when an invalid Alertmanager configuration was set even though the Alertmanager component was disabled. After this fix, the Alertmanager configuration is only validated if the Alertmanager component is loaded. +- [PR 1836](https://github.com/grafana/mimir/pull/1836): The ability to run Alertmanager with `local` storage broke in Grafana Mimir 2.0 when we removed the ability to run the Alertmanager without sharding. With this bugfix, we've made it possible to again run Alertmanager with `local` storage. However, for production use, we still recommend using external store since this is needed to persist Alertmanager state (e.g. silences) between replicas. +- [PR 1715](https://github.com/grafana/mimir/pull/1715): Restored Grafana Mimir's ability to use CNAME DNS records to reach memcached servers. The bug was inherited from an upstream change to Thanos; we contributed a fix to Thanos and subsequently updated our Thanos version. From 638d4e75f671b6fa06efd0929b52b544fda85e81 Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Sun, 15 May 2022 10:22:48 -0400 Subject: [PATCH 06/12] Update v2.1.md Added the series count description --- docs/sources/release-notes/v2.1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/release-notes/v2.1.md b/docs/sources/release-notes/v2.1.md index 928e125f766..dd86e30bf3d 100644 --- a/docs/sources/release-notes/v2.1.md +++ b/docs/sources/release-notes/v2.1.md @@ -19,7 +19,7 @@ Below we highlight the top features, enhancements and bugfixes in this release, - **Per-tenant custom trackers for monitoring cardinality**: In Grafana Mimir 2.0, we introduced a [custom tracker feature]({{< relref "../operators-guide/configuring/configuring-custom-trackers.md" >}}) that allows you to track the count of active series over time that match a specific label matcher. In Grafana Mimir 2.1, we've made it possible to configure custom trackers via the [runtime configuration file]({{< relref "../operators-guide/configuring/about-runtime-configuration.md" >}}). This means you can now define different trackers for each tenant in your cluster and modify those trackers without an ingester restart. -- **Remove unused series from Grafana Mimir's `/metrics` endpoint**: +- **Reduce cardinality of Grafana Mimir's `/metrics` endpoint**: While Grafana Mimir does a good job of exposing a relatively small number of series about its own state, this number can tick up when running Grafana Mimir clusters with high tenant counts or high active series counts. To reduce this number (and the accompanying cost of scraping and storing these time series), we've made [several optimizations](https://github.com/grafana/mimir/issues/1750). Our improvements have already reduced series count on the `/metrics` endpoint of one of our internal clusters by 11%, and we expect an even greater reduction on our bigger clusters once this rolls out there. ## Upgrade considerations From 287b8cb58e4d5227d845c5c9bd0efb09941b1b7a Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Mon, 16 May 2022 09:20:08 -0400 Subject: [PATCH 07/12] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Peter Štibraný Co-authored-by: Marco Pracucci --- docs/sources/release-notes/v2.1.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sources/release-notes/v2.1.md b/docs/sources/release-notes/v2.1.md index dd86e30bf3d..fa35370c41a 100644 --- a/docs/sources/release-notes/v2.1.md +++ b/docs/sources/release-notes/v2.1.md @@ -13,13 +13,13 @@ Below we highlight the top features, enhancements and bugfixes in this release, ## Features and enhancements -- **Mimir on ARM**: The `mimir build` command now builds both amd64 and arm64 Docker images, making it easier for those on arm-based machines to develop and run Mimir. Both amd64 and arm64 images will now be published to the [Mimir docker registry](https://hub.docker.com/r/grafana/mimir). Note that our existing integration test suite only uses the amd64 images, which means we cannot make any functional or performance guarantees about the arm64 images. +- **Mimir on ARM**: We now publish Docker images for both `amd64` and `arm64`, making it easier for those on arm-based machines to develop and run Mimir. Multiplaform images are available from the the [Mimir docker registry](https://hub.docker.com/r/grafana/mimir). Note that our existing integration test suite only uses the `amd64` images, which means we cannot make any functional or performance guarantees about the `arm64` images. - **`Remote` ruler mode for improved rule evaluation performance**: We've added a `remote` mode for the Grafana Mimir ruler, in which the ruler delegates rule evaluation to the [query-frontend]({{< relref "../operators-guide/architecture/components/query-frontend/index.md" >}}) rather than evaluating rules directly within the ruler process itself. This allows recording and alerting rules to benefit from the query parallelization techniques implemented in the query-frontend (like query sharding). `Remote` mode is considered experimental and is off by default. To enable, see [remote ruler]({{< relref "../operators-guide/architecture/components/ruler/#remote" >}}). - **Per-tenant custom trackers for monitoring cardinality**: In Grafana Mimir 2.0, we introduced a [custom tracker feature]({{< relref "../operators-guide/configuring/configuring-custom-trackers.md" >}}) that allows you to track the count of active series over time that match a specific label matcher. In Grafana Mimir 2.1, we've made it possible to configure custom trackers via the [runtime configuration file]({{< relref "../operators-guide/configuring/about-runtime-configuration.md" >}}). This means you can now define different trackers for each tenant in your cluster and modify those trackers without an ingester restart. -- **Reduce cardinality of Grafana Mimir's `/metrics` endpoint**: While Grafana Mimir does a good job of exposing a relatively small number of series about its own state, this number can tick up when running Grafana Mimir clusters with high tenant counts or high active series counts. To reduce this number (and the accompanying cost of scraping and storing these time series), we've made [several optimizations](https://github.com/grafana/mimir/issues/1750). Our improvements have already reduced series count on the `/metrics` endpoint of one of our internal clusters by 11%, and we expect an even greater reduction on our bigger clusters once this rolls out there. +- **Reduce cardinality of Grafana Mimir's `/metrics` endpoint**: While Grafana Mimir does a good job of exposing a relatively small number of series about its own state, this number can tick up when running Grafana Mimir clusters with high tenant counts or high active series counts. To reduce this number (and the accompanying cost of scraping and storing these time series), we've made [several optimizations](https://github.com/grafana/mimir/issues/1750). These improvements reduces series count on the `/metrics` endpoint by more than 10%. ## Upgrade considerations From 8c2af9156db8e6b818e2ca91cf0e95348649ecfc Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Mon, 16 May 2022 09:38:41 -0400 Subject: [PATCH 08/12] Update v2.1.md --- docs/sources/release-notes/v2.1.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sources/release-notes/v2.1.md b/docs/sources/release-notes/v2.1.md index fa35370c41a..c5accb15382 100644 --- a/docs/sources/release-notes/v2.1.md +++ b/docs/sources/release-notes/v2.1.md @@ -19,7 +19,7 @@ Below we highlight the top features, enhancements and bugfixes in this release, - **Per-tenant custom trackers for monitoring cardinality**: In Grafana Mimir 2.0, we introduced a [custom tracker feature]({{< relref "../operators-guide/configuring/configuring-custom-trackers.md" >}}) that allows you to track the count of active series over time that match a specific label matcher. In Grafana Mimir 2.1, we've made it possible to configure custom trackers via the [runtime configuration file]({{< relref "../operators-guide/configuring/about-runtime-configuration.md" >}}). This means you can now define different trackers for each tenant in your cluster and modify those trackers without an ingester restart. -- **Reduce cardinality of Grafana Mimir's `/metrics` endpoint**: While Grafana Mimir does a good job of exposing a relatively small number of series about its own state, this number can tick up when running Grafana Mimir clusters with high tenant counts or high active series counts. To reduce this number (and the accompanying cost of scraping and storing these time series), we've made [several optimizations](https://github.com/grafana/mimir/issues/1750). These improvements reduces series count on the `/metrics` endpoint by more than 10%. +- **Reduce cardinality of Grafana Mimir's `/metrics` endpoint**: While Grafana Mimir does a good job of exposing a relatively small number of series about its own state, this number can tick up when running Grafana Mimir clusters with high tenant counts or high active series counts. To reduce this number (and the accompanying cost of scraping and storing these time series), we've made [several optimizations](https://github.com/grafana/mimir/issues/1750). These improvements reduce series count on the `/metrics` endpoint by more than 10%. ## Upgrade considerations @@ -27,7 +27,7 @@ We've updated the default values for 2 parameters in Grafana Mimir to give users - We've changed the default for `-blocks-storage.tsdb.isolation-enabled` from `true` to `false`. We've marked this flag as deprecated and will remove it, setting the value permanently to `false`, in 2 releases. Our decision to do this came from our experience running our [1 billion series load test](https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-prometheus-tsdb-grafana-mimir-to-1-billion-active-series/#prometheus-tsdb-enhancements), where we saw that disabling this setting reduced ingester 99th percentile latency by 90%. -- The store gateway attributes cache is now enabled by default (achieved by updating the default for `-blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items` from `0` to `50000`). This in-memory cache makes it faster to look up object attributes for chunk data. We've been running this optional cache internally for a while and upon a recent configuration audit, realized it made sense to do the same for all users. We estimate this change will consume an additional 6MB of memory, but we believe it is well worth the performance improvement. +- The store gateway attributes cache is now enabled by default (achieved by updating the default for `-blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items` from `0` to `50000`). This in-memory cache makes it faster to look up object attributes for chunk data. We've been running this optional cache internally for a while and upon a recent configuration audit, realized it made sense to do the same for all users. The increase in store-gateway memory utilization from enabling this cache is negligible and easily justified given the performance gains. ## Bug fixes From 81516d08323fa4d2b8a447679b6d9b1da70525a1 Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Mon, 16 May 2022 16:32:09 -0400 Subject: [PATCH 09/12] Update v2.1.md updated tsdb isolation wording. --- docs/sources/release-notes/v2.1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/release-notes/v2.1.md b/docs/sources/release-notes/v2.1.md index c5accb15382..38e3a13e589 100644 --- a/docs/sources/release-notes/v2.1.md +++ b/docs/sources/release-notes/v2.1.md @@ -25,7 +25,7 @@ Below we highlight the top features, enhancements and bugfixes in this release, We've updated the default values for 2 parameters in Grafana Mimir to give users better out-of-the-box performance: -- We've changed the default for `-blocks-storage.tsdb.isolation-enabled` from `true` to `false`. We've marked this flag as deprecated and will remove it, setting the value permanently to `false`, in 2 releases. Our decision to do this came from our experience running our [1 billion series load test](https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-prometheus-tsdb-grafana-mimir-to-1-billion-active-series/#prometheus-tsdb-enhancements), where we saw that disabling this setting reduced ingester 99th percentile latency by 90%. +- We've changed the default for `-blocks-storage.tsdb.isolation-enabled` from `true` to `false`. We've marked this flag as deprecated and will remove it completely in 2 releases. [TSDB isolation](https://grafana.com/blog/2020/05/05/how-isolation-improves-queries-in-prometheus-2.17/) is a feature inherited from Prometheus that didn't provide any benefit given Grafana Mimir's distributed architecture and in our [1 billion series load test](https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-prometheus-tsdb-grafana-mimir-to-1-billion-active-series/#prometheus-tsdb-enhancements) we found it actually hurt performance. Disabling it reduced our ingester 99th percentile latency by 90%. - The store gateway attributes cache is now enabled by default (achieved by updating the default for `-blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items` from `0` to `50000`). This in-memory cache makes it faster to look up object attributes for chunk data. We've been running this optional cache internally for a while and upon a recent configuration audit, realized it made sense to do the same for all users. The increase in store-gateway memory utilization from enabling this cache is negligible and easily justified given the performance gains. From 5b70d63fd7da32dfb040c7b191e6682f9df57130 Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Mon, 16 May 2022 21:44:55 +0000 Subject: [PATCH 10/12] Ran make doc. --- docs/sources/release-notes/v2.1.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/sources/release-notes/v2.1.md b/docs/sources/release-notes/v2.1.md index 38e3a13e589..75c968f93e7 100644 --- a/docs/sources/release-notes/v2.1.md +++ b/docs/sources/release-notes/v2.1.md @@ -7,13 +7,13 @@ weight: 200 # Grafana Mimir version 2.1 release notes -Grafana Labs is excited to announce version 2.1 of Grafana Mimir, the most scalable, most performant open source time series database in the world. +Grafana Labs is excited to announce version 2.1 of Grafana Mimir, the most scalable, most performant open source time series database in the world. Below we highlight the top features, enhancements and bugfixes in this release, as well as relevant callouts for those upgrading from Grafana Mimir 2.0. The complete list of changes is recorded in the [Changelog](https://github.com/grafana/mimir/blob/main/CHANGELOG.md). ## Features and enhancements -- **Mimir on ARM**: We now publish Docker images for both `amd64` and `arm64`, making it easier for those on arm-based machines to develop and run Mimir. Multiplaform images are available from the the [Mimir docker registry](https://hub.docker.com/r/grafana/mimir). Note that our existing integration test suite only uses the `amd64` images, which means we cannot make any functional or performance guarantees about the `arm64` images. +- **Mimir on ARM**: We now publish Docker images for both `amd64` and `arm64`, making it easier for those on arm-based machines to develop and run Mimir. Multiplaform images are available from the the [Mimir docker registry](https://hub.docker.com/r/grafana/mimir). Note that our existing integration test suite only uses the `amd64` images, which means we cannot make any functional or performance guarantees about the `arm64` images. - **`Remote` ruler mode for improved rule evaluation performance**: We've added a `remote` mode for the Grafana Mimir ruler, in which the ruler delegates rule evaluation to the [query-frontend]({{< relref "../operators-guide/architecture/components/query-frontend/index.md" >}}) rather than evaluating rules directly within the ruler process itself. This allows recording and alerting rules to benefit from the query parallelization techniques implemented in the query-frontend (like query sharding). `Remote` mode is considered experimental and is off by default. To enable, see [remote ruler]({{< relref "../operators-guide/architecture/components/ruler/#remote" >}}). @@ -27,13 +27,13 @@ We've updated the default values for 2 parameters in Grafana Mimir to give users - We've changed the default for `-blocks-storage.tsdb.isolation-enabled` from `true` to `false`. We've marked this flag as deprecated and will remove it completely in 2 releases. [TSDB isolation](https://grafana.com/blog/2020/05/05/how-isolation-improves-queries-in-prometheus-2.17/) is a feature inherited from Prometheus that didn't provide any benefit given Grafana Mimir's distributed architecture and in our [1 billion series load test](https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-prometheus-tsdb-grafana-mimir-to-1-billion-active-series/#prometheus-tsdb-enhancements) we found it actually hurt performance. Disabling it reduced our ingester 99th percentile latency by 90%. -- The store gateway attributes cache is now enabled by default (achieved by updating the default for `-blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items` from `0` to `50000`). This in-memory cache makes it faster to look up object attributes for chunk data. We've been running this optional cache internally for a while and upon a recent configuration audit, realized it made sense to do the same for all users. The increase in store-gateway memory utilization from enabling this cache is negligible and easily justified given the performance gains. +- The store gateway attributes cache is now enabled by default (achieved by updating the default for `-blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items` from `0` to `50000`). This in-memory cache makes it faster to look up object attributes for chunk data. We've been running this optional cache internally for a while and upon a recent configuration audit, realized it made sense to do the same for all users. The increase in store-gateway memory utilization from enabling this cache is negligible and easily justified given the performance gains. ## Bug fixes ### 2.1.0 bug fixes -- [PR 1704](https://github.com/grafana/mimir/pull/1704): Fixed a bug that previously caused Grafana Mimir to crash on startup when trying to run in monolithic mode with the results cache enabled due to duplicate metric names. -- [PR 1835](https://github.com/grafana/mimir/pull/1835): Fixed a bug that caused Grafana Mimir to crash when an invalid Alertmanager configuration was set even though the Alertmanager component was disabled. After this fix, the Alertmanager configuration is only validated if the Alertmanager component is loaded. -- [PR 1836](https://github.com/grafana/mimir/pull/1836): The ability to run Alertmanager with `local` storage broke in Grafana Mimir 2.0 when we removed the ability to run the Alertmanager without sharding. With this bugfix, we've made it possible to again run Alertmanager with `local` storage. However, for production use, we still recommend using external store since this is needed to persist Alertmanager state (e.g. silences) between replicas. -- [PR 1715](https://github.com/grafana/mimir/pull/1715): Restored Grafana Mimir's ability to use CNAME DNS records to reach memcached servers. The bug was inherited from an upstream change to Thanos; we contributed a fix to Thanos and subsequently updated our Thanos version. +- [PR 1704](https://github.com/grafana/mimir/pull/1704): Fixed a bug that previously caused Grafana Mimir to crash on startup when trying to run in monolithic mode with the results cache enabled due to duplicate metric names. +- [PR 1835](https://github.com/grafana/mimir/pull/1835): Fixed a bug that caused Grafana Mimir to crash when an invalid Alertmanager configuration was set even though the Alertmanager component was disabled. After this fix, the Alertmanager configuration is only validated if the Alertmanager component is loaded. +- [PR 1836](https://github.com/grafana/mimir/pull/1836): The ability to run Alertmanager with `local` storage broke in Grafana Mimir 2.0 when we removed the ability to run the Alertmanager without sharding. With this bugfix, we've made it possible to again run Alertmanager with `local` storage. However, for production use, we still recommend using external store since this is needed to persist Alertmanager state (e.g. silences) between replicas. +- [PR 1715](https://github.com/grafana/mimir/pull/1715): Restored Grafana Mimir's ability to use CNAME DNS records to reach memcached servers. The bug was inherited from an upstream change to Thanos; we contributed a fix to Thanos and subsequently updated our Thanos version. From b9fee4bc856fa60e6bcd58f8d3ff84118bb26e13 Mon Sep 17 00:00:00 2001 From: Jennifer Villa Date: Tue, 17 May 2022 01:50:26 +0000 Subject: [PATCH 11/12] Fixed a broken relref. --- docs/sources/release-notes/v2.1.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sources/release-notes/v2.1.md b/docs/sources/release-notes/v2.1.md index 75c968f93e7..0aee38d2810 100644 --- a/docs/sources/release-notes/v2.1.md +++ b/docs/sources/release-notes/v2.1.md @@ -15,11 +15,11 @@ Below we highlight the top features, enhancements and bugfixes in this release, - **Mimir on ARM**: We now publish Docker images for both `amd64` and `arm64`, making it easier for those on arm-based machines to develop and run Mimir. Multiplaform images are available from the the [Mimir docker registry](https://hub.docker.com/r/grafana/mimir). Note that our existing integration test suite only uses the `amd64` images, which means we cannot make any functional or performance guarantees about the `arm64` images. -- **`Remote` ruler mode for improved rule evaluation performance**: We've added a `remote` mode for the Grafana Mimir ruler, in which the ruler delegates rule evaluation to the [query-frontend]({{< relref "../operators-guide/architecture/components/query-frontend/index.md" >}}) rather than evaluating rules directly within the ruler process itself. This allows recording and alerting rules to benefit from the query parallelization techniques implemented in the query-frontend (like query sharding). `Remote` mode is considered experimental and is off by default. To enable, see [remote ruler]({{< relref "../operators-guide/architecture/components/ruler/#remote" >}}). +- **`Remote` ruler mode for improved rule evaluation performance**: We've added a `remote` mode for the Grafana Mimir ruler, in which the ruler delegates rule evaluation to the [query-frontend]({{< relref "../operators-guide/architecture/components/query-frontend/index.md" >}}) rather than evaluating rules directly within the ruler process itself. This allows recording and alerting rules to benefit from the query parallelization techniques implemented in the query-frontend (like query sharding). `Remote` mode is considered experimental and is off by default. To enable, see [remote ruler]({{< relref "../operators-guide/architecture/components/ruler/index.md#remote" >}}). - **Per-tenant custom trackers for monitoring cardinality**: In Grafana Mimir 2.0, we introduced a [custom tracker feature]({{< relref "../operators-guide/configuring/configuring-custom-trackers.md" >}}) that allows you to track the count of active series over time that match a specific label matcher. In Grafana Mimir 2.1, we've made it possible to configure custom trackers via the [runtime configuration file]({{< relref "../operators-guide/configuring/about-runtime-configuration.md" >}}). This means you can now define different trackers for each tenant in your cluster and modify those trackers without an ingester restart. -- **Reduce cardinality of Grafana Mimir's `/metrics` endpoint**: While Grafana Mimir does a good job of exposing a relatively small number of series about its own state, this number can tick up when running Grafana Mimir clusters with high tenant counts or high active series counts. To reduce this number (and the accompanying cost of scraping and storing these time series), we've made [several optimizations](https://github.com/grafana/mimir/issues/1750). These improvements reduce series count on the `/metrics` endpoint by more than 10%. +- **Reduce cardinality of Grafana Mimir's `/metrics` endpoint**: While Grafana Mimir does a good job of exposing a relatively small number of series about its own state, this number can tick up when running Grafana Mimir clusters with high tenant counts or high active series counts. To reduce this number (and the accompanying cost of scraping and storing these time series), we made [several optimizations](https://github.com/grafana/mimir/issues/1750) which decreased series count on the `/metrics` endpoint by more than 10%. ## Upgrade considerations From 8dd6f4bc653192f29ff625dc12d492d316bcd248 Mon Sep 17 00:00:00 2001 From: Marco Pracucci Date: Tue, 17 May 2022 12:02:41 +0200 Subject: [PATCH 12/12] Update docs/sources/release-notes/v2.1.md --- docs/sources/release-notes/v2.1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/release-notes/v2.1.md b/docs/sources/release-notes/v2.1.md index 0aee38d2810..7af3aa42777 100644 --- a/docs/sources/release-notes/v2.1.md +++ b/docs/sources/release-notes/v2.1.md @@ -27,7 +27,7 @@ We've updated the default values for 2 parameters in Grafana Mimir to give users - We've changed the default for `-blocks-storage.tsdb.isolation-enabled` from `true` to `false`. We've marked this flag as deprecated and will remove it completely in 2 releases. [TSDB isolation](https://grafana.com/blog/2020/05/05/how-isolation-improves-queries-in-prometheus-2.17/) is a feature inherited from Prometheus that didn't provide any benefit given Grafana Mimir's distributed architecture and in our [1 billion series load test](https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-prometheus-tsdb-grafana-mimir-to-1-billion-active-series/#prometheus-tsdb-enhancements) we found it actually hurt performance. Disabling it reduced our ingester 99th percentile latency by 90%. -- The store gateway attributes cache is now enabled by default (achieved by updating the default for `-blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items` from `0` to `50000`). This in-memory cache makes it faster to look up object attributes for chunk data. We've been running this optional cache internally for a while and upon a recent configuration audit, realized it made sense to do the same for all users. The increase in store-gateway memory utilization from enabling this cache is negligible and easily justified given the performance gains. +- The store-gateway attributes cache is now enabled by default (achieved by updating the default for `-blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items` from `0` to `50000`). This in-memory cache makes it faster to look up object attributes for chunk data. We've been running this optional cache internally for a while and upon a recent configuration audit, realized it made sense to do the same for all users. The increase in store-gateway memory utilization from enabling this cache is negligible and easily justified given the performance gains. ## Bug fixes