Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow transform_generic to keep existing fields #151

Closed
amorenoz opened this issue Mar 18, 2022 · 4 comments
Closed

Allow transform_generic to keep existing fields #151

amorenoz opened this issue Mar 18, 2022 · 4 comments

Comments

@amorenoz
Copy link
Contributor

Currently transform_generic removes all fields from the flow if you don't explicitly set them in the configuration.
This makes it's usage quite limited because you have to know exactly all the fields that the log will contain or you cannot modify a single field. It would be great to have that behavior configurable through a flag.

Another option would be to have transform_generic as a simple field rewrite (without removing fields that are not explicitly set) and have another transformation stage where you can select the fields you want for your log. It could have a more semantically intuitive name such as filteror select

@jotak
Copy link
Member

jotak commented Mar 24, 2022

sounds like a good enhancement to do, cc @eranra @KalmanMeth @ronensc

@KalmanMeth
Copy link
Collaborator

When the transform_generic was first written I had in mind that it would completely transform the input to some standard internal format. However, we can do whatever we now think is most appropriate, and update the unit tests accordingly.

@amorenoz
Copy link
Contributor Author

The problem with the current approach is that it limits the extensibility of the flow pipeline. If the flow exporter supports an additional field (through a newly defined template), we have to add it to the transform config list or else the new field will not be present in the log.

@eranra
Copy link
Collaborator

eranra commented Apr 10, 2022

#156 provides the needed functionality.

@joel if needed you can modify NOO configuration to use this updated behavior

@eranra eranra closed this as completed Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants