Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: cleanup json output to be machine-readable #208

Merged
merged 5 commits into from
May 26, 2021
Merged

Conversation

newsroomdev
Copy link
Member

Summary

This change accomplishes both a fix and provides a new feature. The initial v2 made the JSON output unable to be read by machines, specifically command-line tools like jq.

Changes

  • Moves contextual info, e.g. the number of tests passed and the number of rows sampled, behind conditionals. If you're outputting JSON --json or --json-pretty then these contextual strings are not output.
  • Converts WriteFileSync to WriteFile streams, allowing for CLI pipe | functionality, e.g. | jq

Addtions

  • Adds a new flag --exit or -e to throw a CLI error if any Dataproofer tests are total failures. This allows for developers to creates automated CI workflows like GitHub Actions that can throw errors and notify others if a dataset has serious problems.

How to review

  • Pull this branch
  • Try out the sample datasets with jq queries. Maybe add some jq documentation?
  • Run Dataproofer with the new --exit flag. Does it throw errors in your console ❌

@newsroomdev newsroomdev requested a review from ejfox May 20, 2021 04:55
@newsroomdev newsroomdev marked this pull request as ready for review May 20, 2021 13:23
Copy link
Contributor

@ejfox ejfox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit: beautiful

@ejfox ejfox merged commit b93d808 into dev May 26, 2021
@ejfox ejfox deleted the grich/fix-json-output branch May 26, 2021 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants