Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metadata modeldecoder and generator logic #4154

Merged
merged 8 commits into from
Sep 8, 2020

Conversation

simitt
Copy link
Contributor

@simitt simitt commented Sep 5, 2020

Motivation/summary

This is the first PR introducing some ground work for the new json decoding + validation handling.
The changes are implemented for the metadata and not yet used. We should integrate with the new logic, once we also have the code generation logic ready for creating json schema for testing purposes of the agents. To keep PRs relatively short and reviewable I suggest to put up isolated PRs for yet unused logic that are tested in themselves and the put everything together when it's ready.
If interested there is a draft PR up showing the whole integration.

This PR introduces:

  • dedicated modeldecoder models for metadata with nullable values for v2 and rumv3
  • using https://github.com/json-iterator/go for json decoding for the new models
  • generated model decoding and validation functions
  • generator logic running on make update

Checklist

- [ ] I have signed the Contributor License Agreement.
- [ ] I have updated CHANGELOG.asciidoc

I have considered changes for:
- [ ] documentation
- [ ] logging (add log lines, choose appropriate log selector, etc.)
- [ ] metrics and monitoring (create issue for Kibana team to add metrics to visualizations, e.g. Kibana#44001)

How to test these changes

run make update && make test

Related issues

#3551

* Introduce modeldecoder models for metadata with nullable values
* Generate model decoding and validation functions
* Run generator logic on `make update`
@simitt simitt added the review label Sep 5, 2020
@simitt simitt requested a review from a team September 5, 2020 17:20
@apmmachine
Copy link
Contributor

apmmachine commented Sep 5, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #4154 updated]

  • Start Time: 2020-09-08T14:08:17.088+0000

  • Duration: 44 min 24 sec

Test stats 🧪

Test Results
Failed 0
Passed 3540
Skipped 145
Total 3685

Steps errors

Expand to view the steps failures

  • Name: Compress

    • Description: tar --exclude=coverage-files.tgz -czf coverage-files.tgz coverage

    • Duration: 0 min 0 sec

    • Start Time: 2020-09-08T14:24:23.785+0000

    • log

  • Name: Compress

    • Description: tar --exclude=system-tests-linux-files.tgz -czf system-tests-linux-files.tgz system-tests

    • Duration: 0 min 0 sec

    • Start Time: 2020-09-08T14:37:35.417+0000

    • log

  • Name: Test Sync

    • Description: ./script/jenkins/sync.sh

    • Duration: 3 min 52 sec

    • Start Time: 2020-09-08T14:18:39.568+0000

    • log

Copy link
Member

@axw axw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks great - I have only fairly minor comments and suggestions. Nice work!

* switch from pkg name `type` to `nullable`
* rename Decoder methods
* simplify v2 validation test logic
* minor refactorings
@simitt simitt requested a review from axw September 8, 2020 09:29
Copy link
Member

@axw axw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome :)

@simitt simitt merged commit c43501f into elastic:master Sep 8, 2020
@simitt simitt mentioned this pull request Sep 15, 2020
1 task
axw pushed a commit to axw/apm-server that referenced this pull request Sep 23, 2020
* Add metadata modeldecoder and generator logic

* Introduce modeldecoder models for metadata with nullable values

* Generate model decoding and validation functions

* Run generator logic on `make update`
axw added a commit that referenced this pull request Sep 23, 2020
* Add metadata modeldecoder and generator logic

* Introduce modeldecoder models for metadata with nullable values

* Generate model decoding and validation functions

* Run generator logic on `make update`

Co-authored-by: Silvia Mitter <[email protected]>
@simitt simitt deleted the metadata-decoder branch October 6, 2020 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants