This repository was archived by the owner on Oct 25, 2023. It is now read-only.
Support exporting for multiple clusters #24
Comments
|
+1 This will become necessary to adhere to API rate limits for environments with many Kafka clusters. |
|
Got it! I will move this issue higher in my in-memory backlog and implement it in my next major iteration. The more I think about it, the more I think ccloudexporter need a configuration file 🤔 |
Merged
Dabz
added a commit
that referenced
this issue
Jun 4, 2020
…lso address many issues:
#24 - Allow to export data from multiple clusters
#31 - Remove global grouping per partition and allow grouping per partition for pre-defined topics
#32 - Do not available all available by default for stability
#34 - Add a header to identify the ccloudexporter
I will do some rebase to cleanup the git branch before merging.
Merged
Dabz
added a commit
that referenced
this issue
Jun 5, 2020
* This PR introduces a configuration file to the ccloudexporter, this also addresses many issues:
#24 - Allow to export data from multiple clusters
#31 - Remove global grouping per partition and allow grouping per partition for pre-defined topics
#32 - Do not available all available by default for stability
#34 - Add a header to identify the ccloudexporter
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
It would be useful to be able to support exporting for multiple clusters. The Metrics API can do this by using an OR:
"filter": { "filters": [ { "field": "metric.label.cluster_id", "op": "EQ", "value": "lkc-XXXX1" }, { "field": "metric.label.cluster_id", "op": "EQ", "value": "lkc-XXXX2" } ], "op": "OR" },Results from the query can also be grouped by both the cluster id and the topic name to avoid collisions of topic names across clusters:
"group_by": [ "metric.label.cluster_id", "metric.label.topic" ],Finally, you can also use the new
GROUPEDformat by passing in theformatparameter to make the grouping more explicit if it is helpful.The text was updated successfully, but these errors were encountered: