Experiment with structured logger #5723
Conversation
This is a proof of concept WIP to add structured logging support to libbeat. This is powered by uber-go/zap (https://github.com/uber-go/zap). I'm opening this so we can discuss if we want to pursue this further. Some new features that will be possible (on top of structured logging): - Test cases can access logs via an in-memory store - Much more configurability w.r.t. logging. We'll need to decide if we want to expose this. - We can make it possible to adjust the levels and selectors at runtime via HTTP.
| @@ -0,0 +1,243 @@ | |||
| package file | |||
There was a problem hiding this comment.
I moved the FileRotator over to the common/file package because it's a reusable piece of code (file output and logging). And I refactored it. Plus it should be renamed b/c it stutters.
There was a problem hiding this comment.
I'm not even sure if log rotation should be a task of the beat itself or an external log rotation tool. We should definitively revisit how it works.
There was a problem hiding this comment.
We should make it possible for beats to work with an external log rotator (logrotate). The beat needs to be able to accept a signal from logrotate that triggers the logger to write to a new file. We can make it a requirement to be logrotate compatible and make this happen.
Overall I like our current behavior because it just works (especially on Windows where there is no built it tool for rotating).
| @@ -0,0 +1,157 @@ | |||
| package monitoring | |||
There was a problem hiding this comment.
I moved this code the logs the metrics out of the logp package. Separation of concerns and all...
I just dropped it in the monitoring package and hacked it to make it work. I was wondering it it would make sense to have a libbeat/monitoring/report/log package that contains this code for reporting the metrics to the log?
| err = out.rotator.CheckIfConfigSane() | ||
| if err != nil { | ||
| return err | ||
| } |
There was a problem hiding this comment.
This is an example of how the structured logger would be used.
|
@andrewkroh Seeing you closed this one, I wonder if you have any follow ups in mind? |
This is a proof of concept WIP to add structured logging support to libbeat. This is powered by uber-go/zap (https://github.com/uber-go/zap).
I'm opening this so we can discuss if we want to pursue this further.
Some new features that will be possible (on top of structured logging):