-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow transform_generic to keep existing fields #151
Comments
sounds like a good enhancement to do, cc @eranra @KalmanMeth @ronensc |
When the transform_generic was first written I had in mind that it would completely transform the input to some standard internal format. However, we can do whatever we now think is most appropriate, and update the unit tests accordingly. |
The problem with the current approach is that it limits the extensibility of the flow pipeline. If the flow exporter supports an additional field (through a newly defined template), we have to add it to the transform config list or else the new field will not be present in the log. |
Currently transform_generic removes all fields from the flow if you don't explicitly set them in the configuration.
This makes it's usage quite limited because you have to know exactly all the fields that the log will contain or you cannot modify a single field. It would be great to have that behavior configurable through a flag.
Another option would be to have
transform_generic
as a simple field rewrite (without removing fields that are not explicitly set) and have another transformation stage where you can select the fields you want for your log. It could have a more semantically intuitive name such asfilter
orselect
The text was updated successfully, but these errors were encountered: