Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fluent-bit output plugin accidentally encodes all strings values inside arrays in base64 #1889

Closed
dottedmag opened this issue Apr 2, 2020 · 0 comments · Fixed by #1890
Closed

fluent-bit output plugin accidentally encodes all strings values inside arrays in base64 #1889

dottedmag opened this issue Apr 2, 2020 · 0 comments · Fixed by #1890

Comments

@dottedmag
Copy link
Contributor

Describe the bug

Any log object with strings inside arrays (directly or indirectly) gets inserted int Loki with these strings base64-encoded, e.g. (abbreviated):

{"Object":{"metadata":{"taints":[{"effect":"Tm9TY2hlZHVsZQ==","key":"bm9kZS1yb2xlLmt1YmVybmV0ZXMuaW8vbWFzdGVy"}]}}}

To Reproduce
Steps to reproduce the behavior:

  1. Started Loki (any version)
  2. Started fluent-bit from master
  3. Published a log message with a string inside a []
  4. Queried this log message, got base64-encoding.

Expected behavior

Strings inside arrays don't get base64-encoded.
@dottedmag dottedmag mentioned this issue Apr 2, 2020
Merged
2 tasks
cyriltovena pushed a commit that referenced this issue Apr 6, 2020
@dottedmag
JSON encoding: avoid base64 encoding of []byte inside other slices (#…
3786d74 
…1890)

Closes #1889

Signed-off-by: Mikhail Gusarov <[email protected]>
slim-bean pushed a commit that referenced this issue Apr 6, 2020
@dottedmag @slim-bean
JSON encoding: avoid base64 encoding of []byte inside other slices (#…
77da15c 
…1890)

Closes #1889

Signed-off-by: Mikhail Gusarov <[email protected]>
(cherry picked from commit 3786d74)
Photonios added a commit to SectorLabs/integrations-docs that referenced this issue May 19, 2021
@Photonios
Fix Fluent-Bit plugin encoding array values as Base64
4dfdcd4 
This issue is caused by the fact that the Fluent-Bit decoder
sends strings as []byte. There was a partial fix for this in
place already that recursed into maps and convertined []byte
to string. The fix was incomplete as it didn't handle []interface{}.
Hence, strings encoded as []byte inside slices weren't converted.

This issue is almost exactly the same as the one in Loki:
grafana/loki#1889

Hence, the fix is also almost the same.
@Photonios Photonios mentioned this issue May 19, 2021
Merged
Photonios added a commit to SectorLabs/integrations-docs that referenced this issue May 19, 2021
@Photonios
Fix Fluent-Bit plugin encoding array values as Base64
533c8b1 
This issue is caused by the fact that the Fluent-Bit decoder
sends strings as []byte. There was a partial fix for this in
place already that recursed into maps and converted []byte
to string. The fix was incomplete as it didn't handle []interface{}.
Hence, strings encoded as []byte inside slices weren't converted.

This issue is almost exactly the same as the one in Loki:
grafana/loki#1889

Hence, the fix is also almost the same.
EldarAliiev pushed a commit to coralogix/integrations-docs that referenced this issue May 20, 2021
@Photonios
Fix Fluent-Bit plugin encoding array values as Base64 (#41)
95da3a5 
This issue is caused by the fact that the Fluent-Bit decoder
sends strings as []byte. There was a partial fix for this in
place already that recursed into maps and converted []byte
to string. The fix was incomplete as it didn't handle []interface{}.
Hence, strings encoded as []byte inside slices weren't converted.

This issue is almost exactly the same as the one in Loki:
grafana/loki#1889

Hence, the fix is also almost the same.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fluentbit: JSON encoding: avoid base64 encoding of []byte inside other slices
1 participant
@dottedmag