Beat for tcp input / proxy

[philipmeadows]

https://github.com/elastic/filebeat/issues/213

Is there a way to migrate this issue into the new project? It's still a valid use case for us.

[monicasarbu]

@philipmeadows We didn't migrate all the issues from each Beat to the new repo. We are considering to leave them in their own repo until they are fixed and open only new ones in the beats repo.

[philipmeadows]

OK thanks.

[mgcrea]

👍 Would love this as well.

[ruflin]

Two related issues which I closed to continue the discussion here:

• TCP Input? #985
• https://github.com/elastic/filebeat/issues/213

[ddes]

Hi,

Any idea if this is gonna be integrated soon ?

Will be very appreciated here 👍

[igama]

👍

[pedro-n-rocha]

👍

[blizzard384]

👍

[r4ravi2008]

Any update on this?

[ruflin]

There is no update yet related to tcp, but we have now an experimental udp prospector in the filebeat 6.0 release. For data coming from local host I would expect udp to also work without data loss: https://github.com/elastic/beats/blob/6.0/filebeat/_meta/common.reference.p2.yml#L236

[abierbaum]

@ruflin I agree that UDP could work, but I am worried about data loss for long message strings.

Is there any documentation on the UDP prospector and specifically how large I can expect UDP to support for the samples sent to it? I would like to send JSON formatted lines to filebeat and have it send them on to ES. But my JSON lines will well exceed the normal UDP MTU sometimes so I expect that I will miss out on some of the messages. Because of that I am looking to output to a rotating log file and read in from there but I would much rather skip the I/O to the disk and just send the JSON directly to a filebeat TCP prospector.

[ruflin]

@abierbaum Have a look at max_message_size https://www.elastic.co/guide/en/beats/filebeat/master/configuration-filebeat-options.html#max-message-size

For the TCP part it reminds me of #5862 (comment) (@ph).

[monicasarbu]

A TCP input is currently added to Filebeat in #6266.

[candlerb]

Can this be closed now that #6266 has been merged?

I have opened #11942 for TCP output.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic]

This issue doesn't have a Team:<team> label.