Allow transform_generic to keep existing fields #156
Conversation
Codecov Report
@@ Coverage Diff @@
## main #156 +/- ##
==========================================
+ Coverage 58.47% 58.95% +0.47%
==========================================
Files 51 54 +3
Lines 2950 3055 +105
==========================================
+ Hits 1725 1801 +76
- Misses 1113 1134 +21
- Partials 112 120 +8
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
KalmanMeth
force-pushed
the
trasnform-generic-ext
branch
from
e5f96a5 to
f625bf7
Compare
|
The default is now to include all previous key/values and to enrich them. To get only the newly specified keys, we now have |
|
addresses issue #151 |
There was a problem hiding this comment.
@KalmanMeth looks good to me. You need to also update https://github.com/netobserv/flowlogs-pipeline/blob/main/pkg/confgen/flowlogs2metrics_config.go#L70 to reflect the new policy field
| To include the original keys and values in addition to those specified in the `rules`, | ||
| specify `policy: preserve_original_keys`. |
There was a problem hiding this comment.
What will happen if an original key conflicts with a key specified in rules? (i.e. they have the same name)
Is it an error? does the original key prevail? or the new key?
There was a problem hiding this comment.
The new key overwrites the old value.
There was a problem hiding this comment.
Do you think its worth noting in the docs?
| output := make([]config.GenericMap, 0) | ||
| for _, entry := range input { | ||
| outputEntry := make(config.GenericMap) | ||
| for _, transformRule := range g.Rules { | ||
| outputEntry := entry |
There was a problem hiding this comment.
if g.policy != "replace_keys" then this code is modifying the input slice. This might be a problem if the same slice is passed to a parallel transform stage.
There was a problem hiding this comment.
OK. Copied map entries individually.
| outputEntry := make(config.GenericMap) | ||
| for _, transformRule := range g.Rules { | ||
| outputEntry := entry | ||
| if g.policy == "replace_keys" { |
There was a problem hiding this comment.
Do we want to panic on an invalid policy value? I mean, if the user writes replace-keys with a dash instead of an underline, we will treat it as preserve_original_keys
There was a problem hiding this comment.
added panic for illegal policy
KalmanMeth
force-pushed
the
trasnform-generic-ext
branch
from
e4d9ee4 to
7c80873
Compare
Addresses issue #151