Skip to content
This repository has been archived by the owner on Nov 1, 2022. It is now read-only.

Reduce spam log generated by registry cache warmer #1529

Closed
stefanprodan opened this issue Nov 17, 2018 · 0 comments
Closed

Reduce spam log generated by registry cache warmer #1529

stefanprodan opened this issue Nov 17, 2018 · 0 comments
Labels
onboarding/activation Particular pertinence to getting Flux up and running UX In pursuit of a delightful user experience

Comments

@stefanprodan
Copy link
Member

Due to the fact that Flux and helm-op images are hosted on Quay, every time someone installs Flux will be faced with hundreds of errors.

These errors will increase with every Flux commit since we log an error per image tag:

ts=2018-11-17T20:11:23.752382666Z caller=warming.go:300 component=warmer canonical_name=quay.io/weaveworks/helm-operator auth={map[]} err="error parsing HTTP 429 response body: invalid character '<' looking for beginning of value: \"<html>\\r\\n<head><title>429 Too Many Requests</title></head>\\r\\n<body bgcolor=\\\"white\\\">\\r\\n<center><h1>429 Too Many Requests</h1></center>\\r\\n<hr><center>nginx/1.14.0</center>\\r\\n</body>\\r\\n</html>\\r\\n\"" ref=quay.io/weaveworks/helm-operator:master-821cc2c

We could detect the HTTP 429 and log a warning once per image or just ignore them and keep the rate limiter logs:

ts=2018-11-17T20:10:35.998697942Z caller=rate_limiter.go:71 component=ratelimiter info="reducing rate limit" host=quay.io limit=50.00

On GKE is even worse since Flux can't parse the nvidia-gpu-device-plugin tag and this adds up to the startup error count:

ts=2018-11-17T20:16:23.188597011Z caller=warming.go:192 component=warmer canonical_name=gcr.io/google-containers/nvidia-gpu-device-plugin@sha256 auth={map[]} err="requesting tags: mux: variable \"google-containers/nvidia-gpu-device-plugin@sha256\" doesn't match, expected \"^(?:(?:[a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9])(?:(?:\\\\.(?:[a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9]))+)?(?::[0-9]+)?/)?[a-z0-9]+(?:(?:(?:[._]|__|[-]*)[a-z0-9]+)+)?(?:(?:/[a-z0-9]+(?:(?:(?:[._]|__|[-]*)[a-z0-9]+)+)?)+)?$\""

Another issue is that we log a git clone error even if the git URL is setup correctly, adding up to the confusion:

ts=2018-11-17T20:10:22.299663884Z caller=loop.go:87 component=sync-loop err="git repo not ready: git repo has not been cloned yet"

For first time users this is a bad experience and makes it very hard to diagnose a git URL miss configuration see #1528
@stefanprodan stefanprodan added onboarding/activation Particular pertinence to getting Flux up and running UX In pursuit of a delightful user experience labels Nov 17, 2018
@stefanprodan stefanprodan mentioned this issue Nov 21, 2018
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
onboarding/activation Particular pertinence to getting Flux up and running UX In pursuit of a delightful user experience
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@stefanprodan