Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Update Logstash module to support pipeline ID from 7.4 logs #14743

Merged
merged 4 commits into from
Nov 25, 2019

Conversation

adriansr
Copy link
Contributor

Starting with Logstash 7.4.0, logs include the pipeline ID:

from v7.4.x Logstash's config/log4j2.properties:

appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]} %m%n

Earlier version:

appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n

Currently this caused the module to fail parsing logs with a Grok error.

Starting with Logstash 7.4.0, logs include the pipeline ID. Currently
this caused the module to fail parsing this kind of logs.
@adriansr adriansr requested review from ycombinator and a team November 25, 2019 12:43
@adriansr
Copy link
Contributor Author

Doubts:

  • Is it worth to save the pipeline ID?
  • Should this be considered a bug or a feature? The change itself seems like just a feature, but the fact that Beats 7.4 can't parse Logstash 7.4 logs sounds like a bug.

@adriansr adriansr requested a review from ph November 25, 2019 16:00
Copy link
Contributor

@ph ph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We surely want to move that to 7.x.

@adriansr adriansr merged commit 3780428 into elastic:master Nov 25, 2019
@adriansr adriansr added the needs_backport PR is waiting to be backported to other branches. label Nov 25, 2019
adriansr added a commit to adriansr/beats that referenced this pull request Nov 25, 2019
elastic#14743)

Starting with Logstash 7.4.0, logs include the pipeline ID:

from v7.4.x Logstash's config/log4j2.properties:

> appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]} %m%n

Earlier version:

> appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n

Currently this caused the module to fail parsing logs with a Grok error.

(cherry picked from commit 3780428)
@adriansr adriansr added v7.6.0 and removed needs_backport PR is waiting to be backported to other branches. labels Nov 25, 2019
adriansr added a commit to adriansr/beats that referenced this pull request Nov 25, 2019
elastic#14743)

Starting with Logstash 7.4.0, logs include the pipeline ID:

from v7.4.x Logstash's config/log4j2.properties:

> appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]} %m%n

Earlier version:

> appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n

Currently this caused the module to fail parsing logs with a Grok error.

(cherry picked from commit 3780428)
adriansr added a commit that referenced this pull request Nov 26, 2019
#14743) (#14757)

Starting with Logstash 7.4.0, logs include the pipeline ID:

from v7.4.x Logstash's config/log4j2.properties:

> appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]} %m%n

Earlier version:

> appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n

Currently this caused the module to fail parsing logs with a Grok error.

(cherry picked from commit 3780428)
adriansr added a commit that referenced this pull request Nov 26, 2019
#14743) (#14756)

Starting with Logstash 7.4.0, logs include the pipeline ID:

from v7.4.x Logstash's config/log4j2.properties:

> appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]} %m%n

Earlier version:

> appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n

Currently this caused the module to fail parsing logs with a Grok error.

(cherry picked from commit 3780428)
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
elastic#14743) (elastic#14757)

Starting with Logstash 7.4.0, logs include the pipeline ID:

from v7.4.x Logstash's config/log4j2.properties:

> appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]} %m%n

Earlier version:

> appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n

Currently this caused the module to fail parsing logs with a Grok error.

(cherry picked from commit 9f69305)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants