[receiver/syslog] expose the syslog parser config from the syslog receiver

[namco1992]

Description

The syslog receiver relies on the stanza syslog input. Among other inputs, the syslog input is a bit special because it's a wrapper of a TCP/UDP input with a syslog parser. The user can't configure the syslog parser because it's not exposed from the syslog input config, which means that some useful features like the on_error mode or parse_from/to is unavailable if you are using syslog receiver.

This PR adds the parser config to the syslog input config under the parser field. I chose not to flatten it because the existing InputConfig.WriterConfig will conflict with the ParserConfig.TransformerConfig.WriterConfig. We can override it later in the Build func, but I guess it's a bit implicit and error-prone, so I choose to add a new parser field. I'm open to changing it if the owners think otherwise.

This change should be backward-compatible.

Testing

Tested locally with newly added config.

Documentation

Added the parser config documentation to the syslog receiver README.

[namco1992]

https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/12425577954/job/34692561655?pr=36906
It seems the TestComponentConfigStruct is unhappy with the ParserConfig. I indeed didn't find another example that exposes the ParserConfig on the receiver level, is it a controversial thing to do? Would appreciate any guidance on this.

[djaglowski]

I don't agree that there's any reason to expose most of these settings. The only one that makes any sense to me is on_error because it can provide useful behavior options.

All of the other fields suggested here are for things that instruct the parser where to find things, which is already defined due to the behavior of the tcp and udp inputs.

If you want to customize behavior further, why not use tcp or udp receivers and configure the syslog parser manually?

[namco1992]

Hi @djaglowski, thanks for the reply.

The only one that makes any sense to me is on_error because it can provide useful behavior options.

If this makes some sense, then is it possible to at least expose this config?

All of the other fields suggested here are for things that instruct the parser where to find things, which is already defined due to the behavior of the tcp and udp inputs.

Can you elaborate more on this point? The TCP/UDP inputs didn't really "define" any parser config other than init a default config. I'm also not sure why it would be a bad thing to grant users the power to config the parser.

If you want to customize behavior further, why not use tcp or udp receivers and configure the syslog parser manually?

Yes, I guess we would have to go down that route if this can't happen upstream. However, there is some config that we can't touch from the outside, like setting the SplitFuncBuilder to OctetSplitFuncBuilder:

opentelemetry-collector-contrib/pkg/stanza/operator/input/syslog/config.go

Lines 65 to 67 in f21f718

	if syslogParserCfg.EnableOctetCounting {
	tcpInputCfg.SplitFuncBuilder = OctetSplitFuncBuilder
	}

CMIIW, but it seems we can't really have a 1-1 replication of the syslog receiver by combining TCP/UDP receiver with syslog parser.

Lastly, the syslog_input documentation claims that it supports https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/f21f7183cafdb2ec61fbb23692e9ee37b4f871c2/pkg/stanza/docs/operators/syslog_parser.md#configuration-fields, but that's not true since the parser config is not configurable.

opentelemetry-collector-contrib/pkg/stanza/docs/operators/syslog_input.md

Line 13 in f21f718

| `syslog` | required | A [syslog parser config](./syslog_parser.md#configuration-fields) to defined syslog_parser operator. |

I understand that it might be a niche request, but I believe it's a reasonable one(or probably because I needed it 🙊 ). I'm ok with exposing the ones that you think make sense.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[namco1992]

Hi @djaglowski, is there anything I can do to reopen this discussion? Or should I consider it as a "no no"? Thanks!

[djaglowski]

The only one that makes any sense to me is on_error because it can provide useful behavior options.

If this makes some sense, then is it possible to at least expose this config?

Yes, I am in support of exposing this.

All of the other fields suggested here are for things that instruct the parser where to find things, which is already defined due to the behavior of the tcp and udp inputs.

Can you elaborate more on this point? The TCP/UDP inputs didn't really "define" any parser config other than init a default config.

The syslog input embeds the tcp & udp inputs, which means there is no ambiguity as to where the syslog parser should look when trying to find the syslog. (e.g. syslog comes into tcp input -> tcp input places it in the body -> syslog parser look for it in the body.)

I'm also not sure why it would be a bad thing to grant users the power to config the parser.

The settings are key to the seamless integration between the inputs and the parser. What good is exposing parse_from if the value is 100% known? It only complicates the configuration and introduces a way for things to not work correctly.

However, there is some config that we can't touch from the outside, like setting the SplitFuncBuilder to OctetSplitFuncBuilder:

opentelemetry-collector-contrib/pkg/stanza/operator/input/syslog/config.go

Lines 65 to 67 in f21f718

	if syslogParserCfg.EnableOctetCounting {
	tcpInputCfg.SplitFuncBuilder = OctetSplitFuncBuilder
	}

You can configure this by setting enable_octet_counting: true. It's right there in the code you linked: if syslogParserCfg.EnableOctetCounting

Lastly, the syslog_input documentation claims that it supports https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/f21f7183cafdb2ec61fbb23692e9ee37b4f871c2/pkg/stanza/docs/operators/syslog_parser.md#configuration-fields, but that's not true since the parser config is not configurable.

It supports all the fields in the syslog parser's "BaseConfig" which as you've noted excludes the fields that are defined in a generic parser.

[namco1992]

You can configure this by setting enable_octet_counting: true. It's right there in the code you linked: if syslogParserCfg.EnableOctetCounting

Let's rewind a bit so I can clarify my understanding. IIUC, the alternative you suggested is TCP/UDP log receiver + syslog parser, and the config could look like this:

receivers:
  tcplog:
    listen_address: 0.0.0.0:601
    operators:
      - type: syslog_parser
        protocol: rfc5424
        enable_octet_counting: true
        parse_to: body

During initialization, the tcp_input and syslog_parser are initialized separately. The SplitFuncBuilder is not exposed as a config, it will be nil and set to a default one.

opentelemetry-collector-contrib/pkg/stanza/operator/input/tcp/config.go

Lines 112 to 114 in 0788185

	if c.SplitFuncBuilder == nil {
	c.SplitFuncBuilder = c.defaultSplitFuncBuilder
	}

The syslog_parser is then initialized as usual, with enable_octet_counting: true, but that won't affect how tcp_input builds the SplitFunc.

What I linked in the previous comment is from syslog_input, which is what we want to mimic with tcp_input and syslog_parser. However, the config in syslog_parser won't affect how tcp_input is initialized. We can expose the SplitFuncBuilder config from tcp_input, but as of now I think the statement below still stands:

it seems we can't really have a 1-1 replication of the syslog receiver by combining TCP/UDP receiver with syslog parser.

For the rest of your comment, I don't really have a strong opinion, I will update my PR to only expose the on_error so we can get the ball rolling first.

[djaglowski]

What I linked in the previous comment is from syslog_input, which is what we want to mimic with tcp_input and syslog_parser.

I still cannot understand why you would need an alternative. on_error is the only use case I've comprehended here. If you can articulate why another field that is not currently exposed would ever need to be customized, then we can talk about exposing it. Otherwise, you should just be able to use syslog_input.

[namco1992]

Yup sure, I guess since you mentioned that possibility then I got too obsessed with it 😂 Will PR add the on_error field only.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.