[filestorage] create_directory respects umask

[cbandy]

Component(s)

extension/storage/filestorage

What happened?

Description

I am using the Collector as a sidecar to parse and export log files. The application (writing logs) and Collector (reading logs) run with the same GID and/but differing UIDs. I would like the filelog receiver to store its offsets on the same disk as the log files so they share that failure domain.

When I enable create_directory and set directory_permissions to 0775, the directories do not have group-write permission. When the collector starts before the application, these permissions prevent the application from writing its logs.

Steps to Reproduce

Configure a file_storage extension with directory_permissions: '0775' and start the collector.

Expected Result

Directories with 0775 permissions.

Actual Result

Directories with 0755 permissions:

$ umask
0022
$ ./otelcol-contrib --config otel.yaml
…

$ ls -ld * */* */*/*
drwxr-xr-x  cbandy  staff         96 Feb  7 09:02 logs
drwxr-xr-x  cbandy  staff         96 Feb  7 09:02 logs/storage
-rw-------  cbandy  staff     131072 Feb  7 09:02 logs/storage/receiver_filelog_
-rw-r--r--  cbandy  staff        332 Feb  7 09:00 otel.yaml
-rwxr-xr-x  cbandy  staff  299985410 Feb  4 12:01 otelcol-contrib
-rw-r--r--  cbandy  staff   75588482 Feb  7 08:37 otelcol-contrib_0.119.0_darwin_arm64.tar.gz

Collector version

otelcol-contrib version 0.119.0

Environment information

Environment

OS: macOS 15.2
otelcol-contrib_0.119.0_darwin_arm64

OpenTelemetry Collector configuration

receivers:
  filelog:
    include: ./logs/*.log
    storage: file_storage

exporters:
  debug: {}

extensions:
  file_storage:
    directory: ./logs/storage
    create_directory: true
    directory_permissions: '0775'

service:
  extensions: [file_storage]
  pipelines:
    logs:
      receivers: [filelog]
      exporters: [debug]

Log output

2025-02-07T08:49:26.068-0600    info    [email protected]/service.go:186 Setting up own telemetry...
2025-02-07T08:49:26.069-0600    info    builders/builders.go:26 Development component. May change in the future.        {"kind": "exporter", "data_type": "logs", "name": "debug"}
2025-02-07T08:49:26.071-0600    info    [email protected]/service.go:252 Starting otelcol-contrib...     {"Version": "0.119.0", "NumCPU": 12}
2025-02-07T08:49:26.071-0600    info    extensions/extensions.go:39     Starting extensions...
2025-02-07T08:49:26.072-0600    info    extensions/extensions.go:42     Extension is starting...        {"kind": "extension", "name": "file_storage"}
2025-02-07T08:49:26.072-0600    info    extensions/extensions.go:59     Extension started.      {"kind": "extension", "name": "file_storage"}
2025-02-07T08:49:26.072-0600    info    adapter/receiver.go:41  Starting stanza receiver        {"kind": "receiver", "name": "filelog", "data_type": "logs"}
2025-02-07T08:49:26.074-0600    warn    fileconsumer/file.go:49 finding files   {"kind": "receiver", "name": "filelog", "data_type": "logs", "component": "fileconsumer", "error": "no files match the configured criteria"}
2025-02-07T08:49:26.075-0600    info    [email protected]/service.go:275 Everything is ready. Begin running and processing data.
2025-02-07T08:49:27.196-0600  info    [email protected]/collector.go:331       Received signal from OS {"signal": "interrupt"}
2025-02-07T08:49:27.197-0600    info    [email protected]/service.go:317 Starting shutdown...
2025-02-07T08:49:27.197-0600    info    adapter/receiver.go:68  Stopping stanza receiver        {"kind": "receiver", "name": "filelog", "data_type": "logs"}
2025-02-07T08:49:27.198-0600    info    extensions/extensions.go:66     Stopping extensions...
2025-02-07T08:49:27.198-0600    info    [email protected]/service.go:331 Shutdown complete.

Additional context

No response

[github-actions]

Pinging code owners:

• extension/storage/filestorage: @swiatekm @VihasMakwana

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[swiatekm]

$ umask
0022

Your umask prevents applications from creating files or directories with these permissions. Thus, 0775 becomes 0755, because umask prevents making files writable by the group. If you want to exempt the otel collector from this, you need to give its user a different mask.

[swiatekm]

/label -bug

[swiatekm]

/label -needs-triage

[cbandy]

Yep. Unfortunately, none of the images produced by opentelemetry-collector-releases have a shell or umask command, so there's no way to adjust the mask before starting the collector.

• Should I open an issue there to add a shell?
• Should I open a PR to document that this file_storage configuration can only restrict permissions rather than expand them?

[swiatekm]

I would say that this is neither the concern of the extension, nor of the collector container image. If this is in Kubernetes, it'd probably be more reliable to use an init container to create this directory with the right permissions.

[cbandy]

I've added a little to the docs in #37783.