Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to use aggregate logger in pipeline #2397

Closed
pkositsyn opened this issue Jan 25, 2021 · 1 comment
Closed

Add ability to use aggregate logger in pipeline #2397

pkositsyn opened this issue Jan 25, 2021 · 1 comment
Labels
duplicate This issue or pull request already exists

Comments

@pkositsyn
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Currently we cannot globally limit number of logs, produced by the pipelines. The common mistake among the whole repository is that every request can result in a log in some consumer, which overrides logging pipelines. Yesterday I got the situation, such that collector started producing 150k logs/min because of exporter failure (which was external error actually).

Describe the solution you'd like
I think the best way to avoid such situations is adding config options for general logger, passed into every consumer.
This is already done in logging exporter and I like this as a way to limit logs

func createLogger(cfg *Config) (*zap.Logger, error) {
var level zapcore.Level
err := (&level).UnmarshalText([]byte(cfg.LogLevel))
if err != nil {
return nil, err
}
// We take development config as the base since it matches the purpose
// of logging exporter being used for debugging reasons (so e.g. console encoder)
conf := zap.NewDevelopmentConfig()
conf.Level = zap.NewAtomicLevelAt(level)
conf.Sampling = &zap.SamplingConfig{
Initial: cfg.SamplingInitial,
Thereafter: cfg.SamplingThereafter,
}
logginglogger, err := conf.Build()
if err != nil {
return nil, err
}
return logginglogger, nil
}

@andrewhsu
Copy link
Member

closing as duplicate because this issue is taken care of in #2102

@andrewhsu andrewhsu added the duplicate This issue or pull request already exists label Jan 28, 2021
hughesjj added a commit to hughesjj/opentelemetry-collector that referenced this issue Apr 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

2 participants