Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vdk-structlog: add vdk_step_name and vdk_step_type to tests and fix filtering bugs #2875

Closed
DeltaMichael opened this issue Nov 3, 2023 · 0 comments · Fixed by #2968
Closed

vdk-structlog: add vdk_step_name and vdk_step_type to tests and fix filtering bugs #2875

DeltaMichael opened this issue Nov 3, 2023 · 0 comments · Fixed by #2968
Assignees
@gabrielgeorgiev1 @DeltaMichael
Labels
enhancement New feature or request initiative: VDK Run Logs story Task for an Epic
Milestone
VDK Run Logs: Log...

Comments

@DeltaMichael
Copy link
Contributor

DeltaMichael commented Nov 3, 2023 •
edited
Loading

Overview

There are no tests that check if filtering vdk_step_name and vdk_step_type works correctly. This is not easy to do, because vdk_step_name and file_name often have the same value. We might have to change how the step name is displayed, so that it's not the same as the file name.

Filtering these two fields doesn't work correctly for json format. This is most likely due to the fact that we're adding the fields directly in the run_step hook, which executes after everything else has been filtered in the run_job hook.

Acceptance criteria

  1. vdk_step_name and vdk_step_type added to functional tests for filtering
  2. bugs with json formatting fixed
@DeltaMichael DeltaMichael added enhancement New feature or request story Task for an Epic initiative: VDK Run Logs labels Nov 3, 2023
@DeltaMichael DeltaMichael changed the title vdk-structlog: add vdk_step_name to tests vdk-structlog: add vdk_step_name and vdk_step_type to tests Nov 3, 2023
@DeltaMichael DeltaMichael changed the title vdk-structlog: add vdk_step_name and vdk_step_type to tests vdk-structlog: add vdk_step_name and vdk_step_type to tests and fix filtering bugs Nov 3, 2023
@DeltaMichael DeltaMichael mentioned this issue Nov 3, 2023
Merged
DeltaMichael added a commit that referenced this issue Nov 14, 2023
@DeltaMichael
vdk-structlog: fix filtering of metadata fields for json (#2874)
8bc76fc 
## Why?

Filtering by timestamp, level and vdk_job_name does not work for json
formatting

## What?

- Introduce logic that inserts the correct fields in the json formatter
log record
- Move the metadata filters after the filters that add the job name
- Fix bug in ConsoleMetadataFilters
- Add timestamp to tests
- Enable json tests

## How was this tested?

Ran functional tests locally

## What kind of change is this?

Bugfix

## Follow-up

#2875

Signed-off-by: Dilyan Marinov <[email protected]>
@gabrielgeorgiev1 gabrielgeorgiev1 self-assigned this Nov 15, 2023
@DeltaMichael DeltaMichael linked a pull request Dec 7, 2023 that will close this issue
vdk-structlog: filter vdk_step_name and vdk_step_type correctly #2968
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gabrielgeorgiev1 gabrielgeorgiev1

@DeltaMichael DeltaMichael

Labels
enhancement New feature or request initiative: VDK Run Logs story Task for an Epic
Projects
None yet
Milestone
VDK Run Logs: Log Structure
Development

Successfully merging a pull request may close this issue.

vdk-structlog: filter vdk_step_name and vdk_step_type correctly vmware/versatile-data-kit
2 participants
@gabrielgeorgiev1 @DeltaMichael