[receiver/filelog] File log receiver not parsing the log when restarted

[vasireddy99]

Component(s)

pkg/stanza, receiver/filelog

What happened?

Description

The file log receiver with storage extension is parsing the log data from a file "A". the collector is stopped, more log lines are to a file A. when the collector is restarted the loglines that were written to log A when the collector is in dead state are missing.

Certainly! This test case appears to be testing the behavior of a collector when it is restarted, specifically in the context of a storage extension.

Steps to Reproduce

Test Case reproducible summary, log lines that are written to the file when collector is stopped are not parsed when the collector is restarted

Setup:

Create a unique log stream name.
Start the OpenTelemetry collector with specific configuration and the log stream name.
Create a temporary log file.
Logging:

Write three log messages to the file.
Validation - First Round:

Validate the logs collected by the collector.
Collector Stop:

Stop the collector.
Logging After Stop:

Add more log messages to the file.
Collector Start:

Restart the collector.
Validation - Second Round:

Validate the logs again.
Collector Stop (Cleanup):

Stop the collector.

Expected Result

2023-12-01T09:56:50.594-08:00 | {"body":"First Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
-- | --
  | 2023-12-01T09:56:50.862-08:00 | {"body":"Second Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
  | 2023-12-01T09:56:50.862-08:00 | {"body":"Third Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
  | 2023-12-01T09:56:52.618-08:00 | {"body":"First Message after collector is stopped","attributes":{"log.file.name":"storageExtension.log"}}
  | 2023-12-01T09:56:52.769-08:00 | {"body":"Second Message after the collector is stopped","attributes":{"log.file.name":"storageExtension.log"}}
  | 2023-12-01T09:56:52.769-08:00 | {"body":"Third Message after the collector is stopped","attributes":{"log.file.name":"storageExtension.log"}}

Actual Result

2023-11-30T12:02:08.228-08:00 | {"body":"First Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
-- | --
  | 2023-11-30T12:02:08.476-08:00 | {"body":"Second Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
  | 2023-11-30T12:02:08.476-08:00 | {"body":"Third Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}

Collector version

v0.89.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

extensions:
  file_storage:
    directory: /logs/

receivers:
  filelog:
    include: [/tempLogs/storageExtension.log]
    storage: file_storage

exporters:
  awscloudwatchlogs:
    log_group_name: "adot-testbed/logs-component-testing/logs"
    log_stream_name: adot-test-bed
    log_retention: 7

service:
  extensions: [ file_storage ]
  pipelines:
    logs:
      receivers: [filelog]
      exporters: [awscloudwatchlogs]

Log output

No response

Additional context

This Behavior is breaking in v0.89.0, it was fine till v0.88.0

[github-actions]

Pinging code owners:

• receiver/filelog: @djaglowski
• pkg/stanza: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[djaglowski]

This may be related to behavior described in #29491. Can you how you are stopping the collector?

[vasireddy99]

This may be related to behavior described in #29491. Can you how you are stopping the collector?

Yeah, seems so. we are using the testcontainers lifecycle to stop/start the collector

[djaglowski]

I've opened #29610 which should address this.

I'd like to have a test case which ensures this works as intended, but it's not very straightforward since any organized shutdown would succeeded in persisting the checkpoints. Any chance you want to contribute an integration test to the filelog receiver using the same testcontainers mechanism?