Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic numerical validation? #69

Closed
brews opened this issue Apr 29, 2021 · 1 comment · Fixed by #118
Closed

Add basic numerical validation? #69

brews opened this issue Apr 29, 2021 · 1 comment · Fixed by #118
Labels
enhancement New feature or request

Comments

@brews
Copy link
Member

brews commented Apr 29, 2021

There is an interest in adding some kind of numerical validation, the kind that's manually called on zarr stores.

Initial ideas are for sanity checks... e.g. "Does the file have temperatures for the surface of the sun?", "Does the file have negative precipitation values?", etc.

Might be worth prototyping these checks in argo manually. Think about standardizing vocabulary here too -- there are many different kinds of "validation"...

@brews brews added the enhancement New feature or request label Apr 29, 2021
@brews
Copy link
Member Author

brews commented Apr 30, 2021

Thinking about this from CLI UX.

Maybe add universal options/flags so that people can specify checks to be run on output for any command? This is a cool idea but it's a bit tricky because most of dodola's commands are streaming data and not reading it from local disk. Users might end up unintentionally loading +GBs of data that were not needed otherwise. We're not dealing with smaller transactional data but big Zarr stores.

That leads me to think these should be separate, explicit commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant