Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use snake case on Azure Storage config #1883

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,14 @@
## main / unreleased
* [CHANGE] **BREAKING CHANGE** Use snake case on Azure Storage config [#1879](https://github.com/grafana/tempo/issues/1879) (@faustodavid)
Example of using snake case on Azure Storage config:
```
# config.yaml
storage:
azure:
storage_account_name:
storage_account_key:
container_name:
```
* [ENHANCEMENT] Support global and wildcard overrides in generic forwarder feature [#1871](https://github.com/grafana/tempo/pull/1871) (@Blinkuu)
* [FEATURE] Add `/status/usage-stats` endpoint to show usage stats data [#1782](https://github.com/grafana/tempo/pull/1782) (@electron0zero)
* [FEATURE] Add generic forwarder and implement otlpgrpc forwarder [#1775](https://github.com/grafana/tempo/pull/1775) (@Blinkuu)
Expand Down
22 changes: 11 additions & 11 deletions docs/tempo/website/configuration/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -635,40 +635,40 @@ storage:
# store traces in this container.
# Tempo requires a dedicated bucket since it maintains a top-level object structure and does not support
# a custom prefix to nest within a shared bucket.
[container-name: <string>]
[container_name: <string>]

# optional.
# Azure endpoint to use, defaults to Azure global(core.windows.net) for other
# regions this needs to be changed e.g Azure China(blob.core.chinacloudapi.cn),
# Azure German(blob.core.cloudapi.de), Azure US Government(blob.core.usgovcloudapi.net).
[endpoint-suffix: <string>]
[endpoint_suffix: <string>]

# Name of the azure storage account
[storage-account-name: <string>]
[storage_account_name: <string>]

# optional.
# access key when using access key credentials.
[storage-account-key: <string>]
[storage_account_key: <string>]

# optional.
# use Azure Managed Identity to access Azure storage.
[use-managed-identity: <bool>]
[use_managed_identity: <bool>]

# optional.
# The Client ID for the user-assigned Azure Managed Identity used to access Azure storage.
[user-assigned-id: <bool>]
[user_assigned_id: <bool>]

# Optional. Default is 0 (disabled)
# Example: "hedge-requests-at: 500ms"
# Example: "hedge_requests_at: 500ms"
# If set to a non-zero value a second request will be issued at the provided duration. Recommended to
# be set to p99 of Axure Blog Storage requests to reduce long tail latency. This setting is most impactful when
# used with queriers and has minimal to no impact on other pieces.
[hedge-requests-at: <duration>]
[hedge_requests_at: <duration>]

# Optional. Default is 2
# Example: "hedge-requests-up-to: 2"
# The maximum number of requests to execute when hedging. Requires hedge-requests-at to be set.
[hedge-requests-up-to: <int>]
# Example: "hedge_requests_up_to: 2"
# The maximum number of requests to execute when hedging. Requires hedge_requests_at to be set.
[hedge_requests_up_to: <int>]

# How often to repoll the backend for new blocks. Default is 5m
[blocklist_poll: <duration>]
Expand Down
8 changes: 4 additions & 4 deletions docs/tempo/website/configuration/azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ weight: 30

Tempo requires the following configuration to authenticate to and access Azure blob storage:

- Storage Account name specified in the configuration file as `storage-account-name` or in the environment variable `AZURE_STORAGE_ACCOUNT`
- Storage Account name specified in the configuration file as `storage_account_name` or in the environment variable `AZURE_STORAGE_ACCOUNT`
- Credentials for accessing the Storage Account; can be one of the following
- Storage Account access key specified in the configuration file as `storage-account-key` or in the environment variable `AZURE_STORAGE_KEY`
- An Azure Managed Identity; either system or user assigned. To use Azure Managed Identities, you'll need to set `use-managed-identity` to `true` in the configuration file or set `user-assigned-id` to the client ID for the managed identity you'd like to use.
- Storage Account access key specified in the configuration file as `storage_account_key` or in the environment variable `AZURE_STORAGE_KEY`
- An Azure Managed Identity; either system or user assigned. To use Azure Managed Identities, you'll need to set `use_managed_identity` to `true` in the configuration file or set `user_assigned_id` to the client ID for the managed identity you'd like to use.
- For a system-assigned managed identity, no additional configuration is required.
- For a user-assigned managed identity, you'll need to set `user-assigned-id` to the client ID for the managed identity in the configuration file.
- For a user-assigned managed identity, you'll need to set `user_assigned_id` to the client ID for the managed identity in the configuration file.

## Azure blocklist polling

Expand Down
20 changes: 10 additions & 10 deletions docs/tempo/website/configuration/manifest.md
Original file line number Diff line number Diff line change
Expand Up @@ -492,16 +492,16 @@ storage:
storage_class: ""
metadata: {}
azure:
storage-account-name: ""
storage-account-key: ""
use-managed-identity: false
user-assigned-id: ""
container-name: ""
endpoint-suffix: blob.core.windows.net
max-buffers: 4
buffer-size: 3145728
hedge-requests-at: 0s
hedge-requests-up-to: 2
storage_account_name: ""
storage_account_key: ""
use_managed_identity: false
user_assigned_id: ""
container_name: ""
endpoint_suffix: blob.core.windows.net
max_buffers: 4
buffer_size: 3145728
hedge_requests_at: 0s
hedge_requests_up_to: 2
cache: ""
cache_min_compaction_level: 0
cache_max_block_age: 0s
Expand Down
8 changes: 4 additions & 4 deletions example/docker-compose/azure/tempo-azure.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ storage:
path: /tmp/tempo/wal # where to store the the wal locally
encoding: snappy # wal encoding/compression. options: none, gzip, lz4-64k, lz4-256k, lz4-1M, lz4, snappy, zstd, s2
azure:
container-name: tempo # how to store data in azure
endpoint-suffix: azurite:10000
storage-account-name: "devstoreaccount1" # Default Azurite storage account
storage-account-key: "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==" # Default azurite sorage key
container_name: tempo # how to store data in azure
endpoint_suffix: azurite:10000
storage_account_name: "devstoreaccount1" # Default Azurite storage account
storage_account_key: "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==" # Default azurite sorage key
pool:
max_workers: 100 # worker pool determines the number of parallel requests to the object store backend
queue_depth: 10000
Expand Down
8 changes: 4 additions & 4 deletions integration/e2e/config-all-in-one-azurite.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@ storage:
blocklist_poll: 1s
backend: azure
azure:
container-name: tempo # how to store data in azure
endpoint-suffix: tempo_e2e-azurite:10000
storage-account-name: "devstoreaccount1"
storage-account-key: "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
container_name: tempo # how to store data in azure
endpoint_suffix: tempo_e2e-azurite:10000
storage_account_name: "devstoreaccount1"
storage_account_key: "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
pool:
max_workers: 10
queue_depth: 100
Expand Down
8 changes: 4 additions & 4 deletions modules/storage/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,11 @@ func (cfg *Config) RegisterFlagsAndApplyDefaults(prefix string, f *flag.FlagSet)
cfg.Trace.Block.RowGroupSizeBytes = 100_000_000 // 100 MB

cfg.Trace.Azure = &azure.Config{}
f.StringVar(&cfg.Trace.Azure.StorageAccountName, util.PrefixConfig(prefix, "trace.azure.storage-account-name"), "", "Azure storage account name.")
f.Var(&cfg.Trace.Azure.StorageAccountKey, util.PrefixConfig(prefix, "trace.azure.storage-account-key"), "Azure storage access key.")
f.StringVar(&cfg.Trace.Azure.ContainerName, util.PrefixConfig(prefix, "trace.azure.container-name"), "", "Azure container name to store blocks in.")
f.StringVar(&cfg.Trace.Azure.StorageAccountName, util.PrefixConfig(prefix, "trace.azure.storage_account_name"), "", "Azure storage account name.")
f.Var(&cfg.Trace.Azure.StorageAccountKey, util.PrefixConfig(prefix, "trace.azure.storage_account_key"), "Azure storage access key.")
f.StringVar(&cfg.Trace.Azure.ContainerName, util.PrefixConfig(prefix, "trace.azure.container_name"), "", "Azure container name to store blocks in.")
f.StringVar(&cfg.Trace.Azure.Endpoint, util.PrefixConfig(prefix, "trace.azure.endpoint"), "blob.core.windows.net", "Azure endpoint to push blocks to.")
f.IntVar(&cfg.Trace.Azure.MaxBuffers, util.PrefixConfig(prefix, "trace.azure.max-buffers"), 4, "Number of simultaneous uploads.")
f.IntVar(&cfg.Trace.Azure.MaxBuffers, util.PrefixConfig(prefix, "trace.azure.max_buffers"), 4, "Number of simultaneous uploads.")
cfg.Trace.Azure.BufferSize = 3 * 1024 * 1024
cfg.Trace.Azure.HedgeRequestsUpTo = 2

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ data:
storage:
trace:
azure:
container-name: tempo
container_name: tempo
backend: gcs
blocklist_poll: 5m
cache: memcached
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ data:
storage:
trace:
azure:
container-name: tempo
container_name: tempo
backend: gcs
blocklist_poll: "0"
cache: memcached
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ data:
storage:
trace:
azure:
container-name: tempo
container_name: tempo
backend: gcs
blocklist_poll: "0"
cache: memcached
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ data:
storage:
trace:
azure:
container-name: tempo
container_name: tempo
backend: gcs
blocklist_poll: "0"
cache: memcached
Expand Down
2 changes: 1 addition & 1 deletion operations/jsonnet-compiled/ConfigMap-tempo-querier.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ data:
storage:
trace:
azure:
container-name: tempo
container_name: tempo
backend: gcs
blocklist_poll: 5m
cache: memcached
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ data:
storage:
trace:
azure:
container-name: tempo
container_name: tempo
backend: gcs
blocklist_poll: "0"
cache: memcached
Expand Down
2 changes: 1 addition & 1 deletion operations/jsonnet-compiled/Deployment-compactor.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ spec:
template:
metadata:
annotations:
config_hash: 04977ed074a986f446c93be0dbcd59cc
config_hash: 4b31dabe47e305e29ccc562cf6212642
labels:
app: compactor
name: compactor
Expand Down
2 changes: 1 addition & 1 deletion operations/jsonnet-compiled/Deployment-distributor.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ spec:
template:
metadata:
annotations:
config_hash: b18e8715d189e60852d11f32653473a9
config_hash: cb68a21b4c17d720811c006122d45156
labels:
app: distributor
name: distributor
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ spec:
template:
metadata:
annotations:
config_hash: 264025c7ed58d6666fe3c7278e8993f2
config_hash: 68ecabe76a76226ca514106156b173a8
labels:
app: metrics-generator
name: metrics-generator
Expand Down
2 changes: 1 addition & 1 deletion operations/jsonnet-compiled/Deployment-querier.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ spec:
template:
metadata:
annotations:
config_hash: 15acd5511dbb9b46b5e58e3fa1059843
config_hash: 282a5956d7363ec1a10ec681e5589158
labels:
app: querier
name: querier
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ spec:
template:
metadata:
annotations:
config_hash: df848edaed79a9620eb44a18be4ce4f5
config_hash: f2c7c344a113c8db82f5bb70cb58391d
labels:
app: query-frontend
name: query-frontend
Expand Down
2 changes: 1 addition & 1 deletion operations/jsonnet-compiled/StatefulSet-ingester.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ spec:
template:
metadata:
annotations:
config_hash: df848edaed79a9620eb44a18be4ce4f5
config_hash: f2c7c344a113c8db82f5bb70cb58391d
labels:
app: ingester
name: ingester
Expand Down
6 changes: 3 additions & 3 deletions operations/jsonnet-compiled/util/jsonnetfile.lock.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
"subdir": "ksonnet-util"
}
},
"version": "351bfae3eee23db305597f272ef29ec16f99ee49",
"sum": "2++XoPslyz02LRgsxREWxjLgYgiCIqhAtXCyVSvYcoE="
"version": "fd5379a1fba2d572fc314a0395dd61e7df335948",
"sum": "/pkNOLhRqvQoPA0yYdUuJvpPHqhkCLauAUMD2ZHMIkE="
},
{
"source": {
Expand All @@ -18,7 +18,7 @@
"subdir": "memcached"
}
},
"version": "351bfae3eee23db305597f272ef29ec16f99ee49",
"version": "fd5379a1fba2d572fc314a0395dd61e7df335948",
"sum": "SWywAq4U0MRPMbASU0Ez8O9ArRNeoZzb75sEuReueow="
},
{
Expand Down
2 changes: 1 addition & 1 deletion operations/jsonnet/microservices/configmap.libsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
bucket: $._config.bucket,
},
azure: {
'container-name': $._config.bucket,
container_name: $._config.bucket,
},
pool: {
queue_depth: 2000,
Expand Down
20 changes: 10 additions & 10 deletions tempodb/backend/azure/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ import (
)

type Config struct {
StorageAccountName string `yaml:"storage-account-name"`
StorageAccountKey flagext.Secret `yaml:"storage-account-key"`
UseManagedIdentity bool `yaml:"use-managed-identity"`
UserAssignedID string `yaml:"user-assigned-id"`
ContainerName string `yaml:"container-name"`
Endpoint string `yaml:"endpoint-suffix"`
MaxBuffers int `yaml:"max-buffers"`
BufferSize int `yaml:"buffer-size"`
HedgeRequestsAt time.Duration `yaml:"hedge-requests-at"`
HedgeRequestsUpTo int `yaml:"hedge-requests-up-to"`
StorageAccountName string `yaml:"storage_account_name"`
StorageAccountKey flagext.Secret `yaml:"storage_account_key"`
UseManagedIdentity bool `yaml:"use_managed_identity"`
UserAssignedID string `yaml:"user_assigned_id"`
ContainerName string `yaml:"container_name"`
Endpoint string `yaml:"endpoint_suffix"`
MaxBuffers int `yaml:"max_buffers"`
BufferSize int `yaml:"buffer_size"`
HedgeRequestsAt time.Duration `yaml:"hedge_requests_at"`
HedgeRequestsUpTo int `yaml:"hedge_requests_up_to"`
}